Delete all but a few nodes in the TensorFlow column

Question

My use of TensorFlow requires me to build a new calculation graph for each instance that needs to be processed. This leads to bloating memory requirements.

Besides a few tf.Variables , which are model parameters, I would like to remove all the other nodes. Other people with similar problems have found that tf.reset_default_graph() will be useful, but it will save you from the model parameters that I need to save.

What can I use to delete everything except these nodes?

Edit: Instances of specific computations really just mean that I am adding a lot of new operations. I believe that these operations are causing memory problems.

UPDATE: See the recently released foldorflow fold ( https://github.com/tensorflow/fold ) for dynamic calculation charts.

Answer 1

The tf.graph data structure is intended as an add-only data structure. Therefore, it is not possible to delete or modify existing nodes. This is usually not a problem, since only the necessary subgraph is processed during the session.

What you can try is to copy the Variabels of your chart into a new graph and delete the old one. For archiving, it is easy to run:

 old_graph = tf.get_default_graph() # Save the old graph for later iteration new_graph = tf.graph() # Create an empty graph new_graph.set_default() # Makes the new graph default 

If you want to iterate over all the nodes of the old graph, use:

 for node in old_graph.get_operations(): if node.type == 'Variable': # read value of variable and copy it into new Graph 

Alternatively you can use:

 for node in old_graph.get_collection('trainable_variables'): # iterates over all trainable Variabels # read and create new variable 

See also python/framework/ops.py : 1759 for more ways to manage nodes in a graph.

However, before you start talking to tf.Graph , I highly recommend considering whether this is really required. You can usually try to generalize the calculations and use common variables by plotting, so each instance that you want to process is a subgraph of that plot.