Tensorflow: When using tf.expand_dims?

Question

Tensorflow tutorials include using tf.expand_dims to add "batch size" to the tensor. I read the docs for this feature, but for me it's still pretty cryptic. Does anyone know in what circumstances this should be used?

My code is below. My intention is to calculate the loss based on the distance between the predicted and the actual cells. (For example, if predictedBin = 10 and truthBin = 7 , then binDistanceLoss = 3 ).

 batch_size = tf.size(truthValues_placeholder) labels = tf.expand_dims(truthValues_placeholder, 1) predictedBin = tf.argmax(logits) binDistanceLoss = tf.abs(tf.sub(labels, logits)) 

In this case, do I need to apply tf.expand_dims to predictedBin and binDistanceLoss ? Thanks in advance.

Answer 1

expand_dims will not add or reduce elements in the tensor, it just changes shape, adding 1 to the dimensions. For example, a vector with 10 elements can be considered as a 10x1 matrix.

The situation I met to use expand_dims was when I tried to create ConvNet to classify grayscale images. Images in grayscale will be uploaded as a size matrix [320, 320] . However, tf.nn.conv2d requires input [batch, in_height, in_width, in_channels] , where in my data there is no in_channels size, which in this case should be 1 . So I used expand_dims to add another dimension.

In your case, I don't think you need expand_dims .

Answer 2

To add Da Tong to the answer, you can expand multiple dimensions at the same time. For example, if you perform the TensorFlow conv1d operation on vectors of rank 1, you need to send them with rank three.

Running expand_dims several times, but can lead to some overhead in the compute graph. You can get the same functionality in the same slot with reshape :

 import tensorflow as tf # having some tensor of rank 1, it could be an audio signal, a word vector... tensor = tf.ones(100) print(tensor.get_shape()) # => (100,) # expand its dimensionality to fit into conv2d tensor_expand = tf.expand_dims(tensor, 0) tensor_expand = tf.expand_dims(tensor_expand, 0) tensor_expand = tf.expand_dims(tensor_expand, -1) print(tensor_expand.get_shape()) # => (1, 1, 100, 1) # do the same in one line with reshape tensor_reshape = tf.reshape(tensor, [1, 1, tensor.get_shape().as_list()[0],1]) print(tensor_reshape.get_shape()) # => (1, 1, 100, 1) 

NOTE. If you get a TypeError: Failed to convert object of type <type 'list'> to Tensor. error TypeError: Failed to convert object of type <type 'list'> to Tensor. try passing tf.shape(x)[0] instead of x.get_shape()[0] as suggested here .

Hope this helps!
Cheers
Andres