What is the default variable initializer in Tensorflow?

What is the default variable initialization method used when calling tf.get_variable() without any specifications for the initializer? The docs just say no.

+8
python deep-learning machine-learning tensorflow
source share
1 answer

From the documentation :

If the initializer is None (the default), the default initializer passed in the variable area will be used. If this is None too, glorot_uniform_initializer will be used.

The glorot_uniform_initializer function initializes values ​​from a uniform distribution.

This feature is documented as:

Glorot Unified Initializer, also called Xavier Uniform Initializer.

It draws samples from the uniform distribution inside [-limit, limit],
where limit is sqrt(6 / (fan_in + fan_out))
where fan_in is the number of input units in the weight tensor and fan_out is the number of output units in the weight tensor.

Link: http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf

+8
source share

All Articles