Tf.contrib.layers.embedding_column from tensor flow

Question

I am going through a tensorflow tensor tutorial . I would like to find a description of the following line:

tf.contrib.layers.embedding_column 

I wonder if he uses word2vec or something else, or maybe I'm thinking in a completely wrong direction. I tried to click on GibHub but found nothing. I assume that finding GitHub will not be easy as python may reference some C ++ libraries. Can someone point me in the right direction?

Answer 1

I am also interested in this. It’s not entirely clear to me what they are doing, but that’s what I found.

The document is here , and it is implemented as a class called _EmbeddingColumn, which is a subclass of _FeatureColumn. It stores the embedding matrix inside the sparse_id_column attribute. The to_dnn_input_layer method then applies this embed matrix to create attachments for the next layer.

  def to_dnn_input_layer(self, input_tensor, weight_collections=None, trainable=True): output, embedding_weights = _create_embedding_lookup( input_tensor=self.sparse_id_column.id_tensor(input_tensor), weight_tensor=self.sparse_id_column.weight_tensor(input_tensor), vocab_size=self.length, dimension=self.dimension, weight_collections=_add_variable_collection(weight_collections), initializer=self.initializer, combiner=self.combiner, trainable=trainable) 

So, as far as I can see, it seems that the investments are formed by applying any training rule used (gradient descent, etc.) to the implementation matrix.

Answer 2

I had the same doubt about the investment.

Here is the main thing:

The ability to add an embedding layer along with traditional wide linear models allows you to accurately predict, reducing the sparse dimension to a low dimension.

Here is a good post about it!

And here is a simple example that combines implementation layers. Using Titanic Kaggle data to predict whether a passenger will survive based on certain attributes such as Name, Gender, what kind of ticket they had, the fare they paid for the cabin in which they stayed, etc.