Geek Answers Handbook

Create an object list function to save as tfrecord in a tensor stream?

How to create a tensor flow record from a list?

From the documentation here it seems possible. There's also this example where they convert a numpy array to a byte array using .tostring() from numpy. However, when I try to get through:

 labels = np.asarray([[1,2,3],[4,5,6]]) ... example = tf.train.Example(features=tf.train.Features(feature={ 'height': _int64_feature(rows), 'width': _int64_feature(cols), 'depth': _int64_feature(depth), 'label': _int64_feature(labels[index]), 'image_raw': _bytes_feature(image_raw)})) writer.write(example.SerializeToString())

I get an error message:

 TypeError: array([1, 2, 3]) has type type 'numpy.ndarray', but expected one of: (type 'int', type 'long')

This does not help me understand how to store a list of integers in tfrecord. I tried to view documents.

+7
source share
4 answers

After some time, messing with him and looking further in the documentation, I found my own answer. In the above function using the example code as a base:

 def _int64_feature(value): return tf.train.Feature(int64_list=tf.train.Int64List(value=[value])) ... 'label': _int64_feature(labels[index]),

labels [index] are passed to the list as [value], so you have [np.array ([1,2,3])], which causes an error.

The above listing was necessary in the example, because tf.train.Int64List () expects either an array of a list or numpy, and the example is passed in one whole so that they list it as such.
In this example, it was like

 label = [1,2,3,4] ... 'label': _int64_feature(label[index]) tf.train.Feature(int64_list=tf.train.Int64List(value=[value])) #Where value = [1] in this case

If you want to go to the list, do it

 labels = np.asarray([[1,2,3],[4,5,6]]) ... def _int64_feature(value): return tf.train.Feature(int64_list=tf.train.Int64List(value=value)) ... 'label': _int64_feature(labels[index]),

I will probably make a stretch request because I found the source documentation for tf.train.Feature is almost non-existent.

TL DR

Pass a list or numpy array to tf.train.Int64List (), but not a list of lists or a list of numpy arrays.

+16
source

According to my understanding, you want to keep a list of integers in tfrecord. You can store one packed BytesList, FloatList or Int64List as per the documentation https://github.com/tensorflow/tensorflow/blob/r0.9/tensorflow/core/example/example.proto

If you look at an example, they use the _int64_feature function, in which they create a list of values ​​passed to the function

  def _int64_feature(value): return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

In your case, you are trying to pass the list as the value of the _int64_feature function so that it throws an error.

so use this instead, which will solve your error for storing a list of int values ​​or modify the above function to suit your needs.

 'label': tf.train.Feature(int64_list=tf.train.Int64List(value=labels[index]))

Hope this will be helpful

+4
source

Int64List , BytesList and FloatList expect an iterator of basic elements (field repeated ). In the case of your _int64_feature function _int64_feature you use the list as an iterator.

When you pass the scalar, your _int64_feature creates in it an array of one int64 element (exactly as expected). But when you pass ndarray, you create a list from one ndarray and pass it to a function that expects an int64 list.

So just remove the array construct from your function: int64_list=tf.train.Int64List(value=value)

+1
source

One of them is to fix value = [value] to value = value, but if you want to pass a list of lists or a list of numpy.arrays, which is a very common situation if you want to keep the x, y, z coordinates for everything atoms of one molecule, in fact, you can first lay out your arrays, and then use value = value. For instance,

  array_1 = np.array([[1,2,3],[2,3,4]]).ravel()
0
source

More articles:


All Articles