Tensorflow understanding tf.train.shuffle_batch

Question

I have one training data file, about 100 thousand lines, and I execute a simple tf.train.GradientDescentOptimizer at each training step. The setup is done essentially directly from the TENSORflow MNIST example. The code below is:

 x = tf.placeholder(tf.float32, [None, 21]) W = tf.Variable(tf.zeros([21, 2])) b = tf.Variable(tf.zeros([2])) y = tf.nn.softmax(tf.matmul(x, W) + b) y_ = tf.placeholder(tf.float32, [None, 2]) cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1])) train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy) 

Given that I am reading training data from a file, I use tf.train.string_input_producer and tf.decode_csv to read lines from csv, and then tf.train.shuffle_batch to create packages that I train later.

I am confused by the fact that my parameters should be for tf.train.shuffle_batch . I read the Tensorflow documentation, and yet I'm still not sure what the “optimal” values ​​for batch_size, capacity, and min_after_dequeue are. Can someone help shed some light on how I'm going to choose the right values ​​for these parameters, or connect me to a resource where I can learn more? Thanks -

Here's the API link: https://www.tensorflow.org/versions/r0.9/api_docs/python/io_ops.html#shuffle_batch

Answer 1

A bit about the number of threads used in

https://www.tensorflow.org/versions/r0.9/how_tos/reading_data/index.html#batching

Unfortunately, I do not think there is a simple answer to batch size. The effective batch size for a network depends on many details about the network. In practice, if you care about optimal performance, you will need to run a trial version and an error (possibly starting from the values ​​used by a similar network).