Difference between variable and get_variable in TensorFlow

Question

As far as I know, Variable is the default operation for creating a variable, and get_variable is mainly used for weight distribution.

On the one hand, there are people who suggest using get_variable instead of the primitive Variable operation whenever you need a variable. On the other hand, I just see get_variable used in official docs and get_variable .

Thus, I want to know some rules of thumb on how to properly use these two mechanisms. Are there any "standard" principles?

Answer 1

I would recommend always using tf.get_variable(...) - this will make it easier to refactor your code if you need to share variables at any time, for example. in setup with several gpu (see example CIFAR with several gpu). There is no shortage of this.

Pure tf.Variable is lower level; at some point tf.get_variable() did not exist, so some code still uses the low level method.

Answer 2

tf.Variable is a class, and there are several ways to create tf.Variable, including tf.Variable .__ init__ and tf.get_variable.

tf.Variable .__ init__: Creates a new variable with an initial_value.

 W = tf.Variable(<initial-value>, name=<optional-name>) 

tf.get_variable: Gets an existing variable with these parameters or creates a new one. You can also use an initializer.

 W = tf.get_variable(name, shape=None, dtype=tf.float32, initializer=None, regularizer=None, trainable=True, collections=None) 

It is very useful to use initializers like xavier_initializer:

 W = tf.get_variable("W", shape=[784, 256], initializer=tf.contrib.layers.xavier_initializer()) 

More information at https://www.tensorflow.org/versions/r0.8/api_docs/python/state_ops.html#Variable .

Answer 3

I can find two main differences between one and the other:

• Firstly, tf.Variable always creates a new variable, regardless of whether tf.get_variable gets an existing variable with these parameters from the graph, and if it does not exist, it creates a new one.

• tf.Variable requires an initial value.

It is important to clarify that the tf.get_variable function prefix name contains the current scope of variables for performing repeated checks. For example:

 with tf.variable_scope("one"): a = tf.get_variable("v", [1]) #a.name == "one/v:0" with tf.variable_scope("one"): b = tf.get_variable("v", [1]) #ValueError: Variable one/v already exists with tf.variable_scope("one", reuse = True): c = tf.get_variable("v", [1]) #c.name == "one/v:0" with tf.variable_scope("two"): d = tf.get_variable("v", [1]) #d.name == "two/v:0" e = tf.Variable(1, name = "v", expected_shape = [1]) #e.name == "two/v_1:0" assert(a is c) #Assertion is true, they refer to the same object. assert(a is d) #AssertionError: they are different objects assert(d is e) #AssertionError: they are different objects 

The last assertion error is interesting: two variables with the same name in the same scope must be the same variable. But if you check the names of the variables d and e , you will realize that Tensorflow changed the name of the variable e :

 d.name #d.name == "two/v:0" e.name #e.name == "two/v_1:0"