Neural networks and XOR function

Question

I play with a neural network, which I implemented myself: this is a trivial direct network using RPROP as a learning algorithm as the only "plus" compared to the basic design.

The network decently evaluates when I test it against MNIST or when I try to compress the image, but when I try to simulate something as simple as the XOR function, sometimes during training it falls into local minima and displays the following truth table:

0 XOR 0 = 1.4598413968251171e-171 1 XOR 0 = 0.9999999999999998 0 XOR 1 = 0.9999999999999998 1 XOR 1 = 0.5 

Often the result after training is correct, but sometimes 1 XOR 1 gives 0.5 instead of 1, as expected. This does not always happen with XOR (1,1), but with other inputs. Being the XOR function of the “classic” in backpropagation literature, I wonder what is going on here, especially considering that my network seems to be studying more complex (but possibly less non-linear) tasks, just fine.

My wild guess is that something is wrong with prejudice.

Any clues?

Note 1: network location above 2 | 3 | 1, but doesn’t change much when I use more hidden units, some training attempts still go wrong.

Note 2: I introduced the implementation in Gist: https://gist.github.com/antirez/e45939b918868b91ec6fea1d1938db0d

Answer 1

The problem was due to an error in my implementation: the NN offset block just before the output block was not correctly calculated. After correcting the code, the XOR function is always calculated correctly.