How to use tf.data initialized iterators in tf.estimator input_fn?

Question

I would like to manage my learning with help tf.estimator.Estimator, but there are some problems with using it along with the tf.dataAPI.

I have something like this:

def model_fn(features, labels, params, mode):
  # Defines model ops.
  # Initializes with tf.train.Scaffold.
  # Returns an tf.estimator.EstimatorSpec.

def input_fn():
  dataset = tf.data.TextLineDataset("test.txt")
  # map, shuffle, padded_batch, etc.

  iterator = dataset.make_initializable_iterator()

  return iterator.get_next()

estimator = tf.estimator.Estimator(model_fn)
estimator.train(input_fn)

Since I cannot use make_one_shot_iteratorfor my use case, my problem is that it input_fncontains an iterator that needs to be initialized internally model_fn(here I use tf.train.Scaffoldto initialize local operating systems).

In addition, I realized that we can not only use input_fn = iterator.get_next, otherwise other ops will not be added to the same graph.

What is the recommended way to initialize an iterator?

Answer 1

TensorFlow 1.5, input_fn return a tf.data.Dataset, :

def input_fn():
  dataset = tf.data.TextLineDataset("test.txt")
  # map, shuffle, padded_batch, etc.
  return dataset

. c294fcfd.

---

tf.GraphKeys.TABLE_INITIALIZERS .

tf.add_to_collection(tf.GraphKeys.TABLE_INITIALIZERS, iterator.initializer)