Uploading Images to a Catalog as a Tensorflow Dataset

Question

I am relatively new to ML and very new to TensorfFlow. I spent a lot of time on the TensorFlow MINST tutorial as well as https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/how_tos/reading_data to try to figure out how to read my own data, but I'm a little confused.

I have a bunch of images (.png) in the / images / 0_Non / directory. I'm trying to make them in a TensorFlow dataset, so I can basically run the material from the MINST tutorial on it as a first pass.

import tensorflow as tf # Make a queue of file names including all the JPEG images files in the relative image directory. filename_queue = tf.train.string_input_producer(tf.train.match_filenames_once("../images/0_Non/*.png")) image_reader = tf.WholeFileReader() # Read a whole file from the queue, the first returned value in the tuple is the filename which we are ignoring. _, image_file = image_reader.read(filename_queue) image = tf.image.decode_png(image_file) # Start a new session to show example output. with tf.Session() as sess: # Required to get the filename matching to run. tf.initialize_all_variables().run() # Coordinate the loading of image files. coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(coord=coord) # Get an image tensor and print its value. image_tensor = sess.run([image]) print(image_tensor) # Finish off the filename queue coordinator. coord.request_stop() coord.join(threads) 

I have problems understanding what is going on here. So it seems that image is a tensor and image_tensor is a numpy array?

How to get my images in a dataset? I also tried following the example of Iris, which brought me here for CSV: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/learn/python/learn/datasets/base.py , but was not sure how to get this for my case when i have a bunch of png.

Thanks!

Answer 1

The recently added tf.data API makes this easier:

 import tensorflow as tf # Make a Dataset of file names including all the PNG images files in # the relative image directory. filename_dataset = tf.data.Dataset.list_files("../images/0_Non/*.png") # Make a Dataset of image tensors by reading and decoding the files. image_dataset = filename_dataset.map(lambda x: tf.decode_png(tf.read_file(x))) # NOTE: You can add additional transformations, like # 'image_dataset.batch(BATCH_SIZE)' or 'image_dataset.repeat(NUM_EPOCHS)' # in here. iterator = image_dataset.make_one_shot_iterator() next_image = iterator.get_next() # Start a new session to show example output. with tf.Session() as sess: try: while True: # Get an image tensor and print its value. image_array = sess.run([next_image]) print(image_tensor) except tf.errors.OutOfRangeError: # We have reached the end of 'image_dataset'. pass 

Please note that for training you will need to get shortcuts somewhere. Converting Dataset.zip() is a possible way to combine image_dataset with a set of labels from another source.

Answer 2

You may need this answer, but this tutorial helped me a lot. https://www.tensorflow.org/tutorials/load_data/images