About "tensorflow.initialize_all_variables ()"

Question

I was wondering what is the difference between the following two pieces of code:

import tensorflow as tf x = tf.Variable(0, name='x') model = tf.initialize_all_variables() with tf.Session() as session: for i in range(5): session.run(model) x = x + 1 print(session.run(x)) 

---

 import tensorflow as tf x = tf.Variable(0, name='x') model = tf.initialize_all_variables() with tf.Session() as session: for i in range(5): x = x + 1 session.run(model) print(session.run(x)) 

The only difference is the order of "x = x + 1" and "session.run (model)". I thought this would make a big difference in the output, since session.run (model) would initialize all the variables. However, two blocks of code display the same things ...

The code is copied from the tutorial: http://learningtensorflow.com/lesson2/

Answer 1

Yes, here is a little complicated. An important concept of Tensorflow is a lazy rating, which means that the Tensorflow node graph is computed first, and the graph is only evaluated at session.run.

For this line of code, x = x + 1 here x is of type Tensor, and + here is overloaded tf.add, so x = x + 1 actually creates a graph until the calculation is performed; and at each iteration, the graph (binary tree in this case) is added with another layer (another nested sum). session.run (model) will always initialize x to 0, session.run (x) will calculate x based on the graph so far constructed in this iteration. For example, in a 4 x iteration, 1 is added 4 times, because the graph at this iteration has 4 nested sums (or layers).

If this makes sense to you, I think that “both codes are essentially the same” will also make sense.

Note: strictly speaking, in the first iteration, x on the right side is of type Variable, but these are details, not the main point I'm trying to do ...