Geek Answers Handbook

Network Learning LSTM

I tried to program my own neural network LSTM (long short memory). I want to check if the basic functionality works. I developed backpropagation through the BPTT time algorithm to train a separate cellular network.

Should a single-cell LSTM network be able to learn a simple sequence or have more than one cell? It seems that the network cannot learn a simple sequence such as 1 0 0 0 1 0 0 0 0 1 0 0 0 1.

I send sequence 1 and 0 one by one, in order, to the network and feed it forward. I write down each output for sequence.

After running the entire sequence through the LSTM cell, I give the average error signals back to the cell, saving the weight changes inside the cell in a separate collection and after all the errors are completed one by one and calculating the new weights after each error, I average the new weights together so that get a new weight, for each weight in the cell.

Am I doing something wrong? I would really appreciate any advice.

Thank you very much!

+4
source share
3 answers

( ) , . 50 . : http://arxiv.org/pdf/1503.04069.pdf . , , / , , - LSTM. (Theano, mxnet, Torch ..) , , , , gpu-, lstm .

+3

1 , , 2 3 0,1,0,1,0,1. , , . .

, , . , ?

, LSTM , . .

.

  • Momentum .

  • , .

  • .

, , , .

GitHub. :

https://github.com/JANNLab/JANNLab/tree/master/examples/de/jannlab/examples

0

LSTM ( ) - , LSTM.

, , " ":

(, ). 0 1. - 0, 1.

0.23, 0 0.65, 0 ... 0.86, 0 0.13, 1 0.76, 0 ... 0.34, 0 0.43, 0 0.12, 1 0.09, 0 .. 0.83, 0 -> 0.125

( 100) 2 1, 0. - , 1. , , . . , , .

0

More articles:


All Articles