Discrete weights and tensor flow activations or Keras

Question

Do you know a way to limit Tensorflow or Keras to a set of discrete weights and use discrete / hard activation functions (like a sign or hard tanh, for example)?

The APIs seem to have only smooth activation functions.

What I was thinking about is also to discretize the scales using a custom regularization function, but I don’t know how to make the frameworks taken into account.

I may have to extend (for example) the Dense Layer class (of the appropriate structure) and define a custom direct distribution function (and its derivative). Do you have examples for this?

Answer 1

-, Keras. :

• : , Keras/Theano ANN, , . . , , Keras Theano .
• :, , , . , , . . 28 28 3 50 (2^dimension).

, .

Answer 2

Tensorflow, . , :

zero = tf.constant(0)
one = tf.constant(1)
neg_one = tf.constant(-1)

hard_tanh(x) = tf.minimum(tf.maximum(x, neg_one), one)) 

sign(x) = tf.greater(x, zero)

, , , , , tf.cast . .

, , . , , , .