Geek Answers Handbook

Neural network is not suitable for XOR

I created an Octave script to train a neural network with 1 hidden layer using backpropagation, but it does not seem to match the XOR function.

  • x 4x2 input matrix [0 0; 0 1; 1 0; 1 1]
  • y 4x1 Output Matrix [0; 1; 1; 0]
  • theta Hidden / output weight levels
  • z Weighted Amounts
  • a Activation function applies to weighted amounts.
  • mSample counter ( 4here)

My weights are initialized as follows

epsilon_init = 0.12;
theta1 = rand(hiddenCount, inputCount + 1) * 2 * epsilon_init * epsilon_init;
theta2 = rand(outputCount, hiddenCount + 1) * 2 * epsilon_init * epsilon_init;

Feed forward

a1 = x;
a1_with_bias = [ones(m, 1) a1];
z2 = a1_with_bias * theta1';
a2 = sigmoid(z2);
a2_with_bias = [ones(size(a2, 1), 1) a2];
z3 = a2_with_bias * theta2';
a3 = sigmoid(z3);

Then I calculate the logistics cost function

j = -sum((y .* log(a3) + (1 - y) .* log(1 - a3))(:)) / m;

Back distribution

delta2 = (a3 - y);
gradient2 = delta2' * a2_with_bias / m;

delta1 = (delta2 * theta2(:, 2:end)) .* sigmoidGradient(z2);
gradient1 = delta1' * a1_with_bias / m;

Gradients were checked correctly using gradient check.

, theta , Octave fminunc . ln(2) ( 0.5 ), 0.5 , .

- , ?

+4
2

, . "" , , , * 2 * epsilon_init - epsilon_init;, * 2 * epsilon_init * epsilon_init;. , .

, - :

theta1 = ( 0.5 * sqrt ( 6 / ( inputCount + hiddenCount) ) * 
    randn( hiddenCount, inputCount + 1 ) );
theta2 = ( 0.5 * sqrt ( 6 / ( hiddenCount + outputCount ) ) * 
    randn( outputCount, hiddenCount + 1 ) );

- , , , , .

, , XOR, . 10000, , . fminunc .

2 , , XOR. , , .

+5

3 , . sigmoidGradient (z2) a2. * (1-a2), .

,

+1

More articles:


All Articles