Geek Answers Handbook

Cannot use grading set + set and train in less than one era

TensorFlow 1.4 moves the TF dataset to the kernel ( tf.data.Dataset) and doc / tutorial suggests using it tf.estimatorfor model training.

However, as recommended at the end of this page , the Dataset object and its iterator must be created inside the function input_fn. This means that iterations through the dataset will begin for each call estimator.train(input_fn, steps). Thus, the call is carried out using the steps <the number of samples in the era, will lead to the training of the model on a subset of the data set.

So my question is. Is it possible to implement something like this with Estimator + Dataset:

for i in range(num_epochs):
    # Train for some steps
    estimator.train(input_fn=train_input_fn, steps=valid_freq)

    validation_iterator.
    # Evaluate on the validation set (steps=None, we evaluate on the full validation set)
    estimator.evaluate(input_fn=valid_input_fn)

without starting learning iteration patterns from scratch with every call estimator.train(input_fn=train_input_fn, steps=valid_freq)?

, , input_fn? , , ( ) ( model_fn) .

github: https://github.com/tensorflow/tensorflow/issues/14283

+6
1

, estimator.train().

, , , train_input_fn , , .

, , [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], estimator.train.
, [0, 1, 2, 3, 4]:

train_size = 10
dataset = tf.data.Dataset.range(train_size)
x = dataset.make_one_shot_iterator().get_next()

sess = tf.Session()
for i in range(train_size // 2):
    print(sess.run(x))

, tf.data.Dataset.shuffle() buffer_size, , , . estimator.train .

train_size = 10
dataset = tf.data.Dataset.range(train_size)
dataset = dataset.shuffle(buffer_size=train_size)
x = dataset.make_one_shot_iterator().get_next()

sess = tf.Session()
for i in range(train_size // 2):
    print(sess.run(x))

, buffer_size .

0

More articles:


All Articles