Tensorflow is new here. I am trying to build RNN. My input is a set of vector size instances instance_sizerepresenting (x, y) the positions of a set of particles at each time step. (Since the instances already have semantic content, they do not require an attachment.) The goal is to learn how to predict the position of particles in the next step.
After the RNN tutorial and slightly adapting the included RNN code, create a model, more or less, like this (omitting some details):
inputs, self._input_data = tf.placeholder(tf.float32, [batch_size, num_steps, instance_size])
self._targets = tf.placeholder(tf.float32, [batch_size, num_steps, instance_size])
with tf.variable_scope("lstm_cell", reuse=True):
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(hidden_size, forget_bias=0.0)
if is_training and config.keep_prob < 1:
lstm_cell = tf.nn.rnn_cell.DropoutWrapper(
lstm_cell, output_keep_prob=config.keep_prob)
cell = tf.nn.rnn_cell.MultiRNNCell([lstm_cell] * config.num_layers)
self._initial_state = cell.zero_state(batch_size, tf.float32)
from tensorflow.models.rnn import rnn
inputs = [tf.squeeze(input_, [1])
for input_ in tf.split(1, num_steps, inputs)]
outputs, state = rnn.rnn(cell, inputs, initial_state=self._initial_state)
output = tf.reshape(tf.concat(1, outputs), [-1, hidden_size])
softmax_w = tf.get_variable("softmax_w", [hidden_size, instance_size])
softmax_b = tf.get_variable("softmax_b", [instance_size])
logits = tf.matmul(output, softmax_w) + softmax_b
loss = position_squared_error_loss(
tf.reshape(logits, [-1]),
tf.reshape(self._targets, [-1]),
)
self._cost = cost = tf.reduce_sum(loss) / batch_size
self._final_state = state
Then I create saver = tf.train.Saver(), iterate over the data in order to train it using this method run_epoch()and write out the parameters with saver.save(). So far so good.
? . tf.train.Saver.restore(), , , , . , , batch_size x num_steps x instance_size. , num_steps x instance_size instance_size ( ); , , , , . , batch_size , . ?