In a tensor flow, how to iterate over a sequence of inputs stored in a tensor?

Question

I am trying to use RNN for the task of classifying a multidimensional variable-length sequence.

I defined the following function to get the output of the sequence (i.e. the output of the RNN cell after submitting the final input from the sequence)

def get_sequence_output(x_sequence, initial_hidden_state): previous_hidden_state = initial_hidden_state for x_single in x_sequence: hidden_state = gru_unit(previous_hidden_state, x_single) previous_hidden_state = hidden_state final_hidden_state = hidden_state return final_hidden_state 

Here x_sequence is the form tensor (?, ?, 10) , where first? for batch size and secondly? for the length of the sequence, and each input element has a length of 10. gru function takes the previous hidden state and current input and spits out the next hidden state (standard gated recursive unit).

I get an error: 'Tensor' object is not iterable. How can I loop through the tensor in order (reading one element at a time)?

My goal is to apply the gru function to each input from the sequence and get the final hidden state.

Answer 1

You can convert the tensor to a list using the unpack function, which converts the first dimension to a list. There is also a split function that does something similar. I am using unack in the RNN model I'm working on.

 y = tf.unstack(tf.transpose(y, (1, 0, 2))) 

In this case, y starts with the form (BATCH_SIZE, TIME_STEPS, 128). I transfer it to take time steps with an external dimension, and then unpack it into a list of tensors, one for each time step. Now each item in the list y, if it has the form (BATCH_SIZE, 128), and I can pass it to my RNN.

Answer 2

In TF> = 1.0, tf.pack and tf.unpack renamed to tf.stack and tf.unstack respectively