Geek Answers Handbook

Predictions Using Keras Recurrent Neural Network - Accuracy Always 1.0

TL; DR: How to use Keras RNN to predict the next value in a sequence?

I have a list of consecutive values. I want to pass them to RNN in order to predict the next value in the sequence.

[ 0.43589744 0.44230769 0.49358974 ..., 0.71153846 0.70833333 0.69230769]

I use Keras for this and can get a network with decreasing loss, but the accuracy is 1.0. This is not true. y_tests != model.predict(x_tests) .

 Epoch 0 1517/1517 [==============================] - 0s - loss: 0.0726 - acc: 1.0000 - val_loss: 0.0636 - val_acc: 1.0000 Epoch 1 1517/1517 [==============================] - 0s - loss: 0.0720 - acc: 1.0000 - val_loss: 0.0629 - val_acc: 1.0000 ...

Here is my network.

 model = Sequential() model.add(SimpleRNN(1, 100)) model.add(Dense(100, 1, activation = "sigmoid")) model.compile(loss="mean_squared_error", optimizer = "sgd")

I tried SimpleRNN, GRU and LSTM, but no luck. Here's how the data is formatted.

 # Current value y_train = [[ 0.60576923] [ 0.64102564] [ 0.66025641] ..., [ 0.71153846] [ 0.70833333] [ 0.69230769]] # Previous 10 values x_train_10 = [ [[ 0.65064103] [ 0.66346154] [ 0.66346154] ..., [ 0.72115385] [ 0.72435897] [ 0.71153846]] ..., [[ 0.66346154] [ 0.66346154] [ 0.67628205] ..., [ 0.72435897] [ 0.71153846] [ 0.70833333]] ] # Previous value x_train_1 = [[ 0.58333333] [ 0.60576923] [ 0.64102564] ..., [ 0.72435897] [ 0.71153846] [ 0.70833333]] # So here are the shapes... y_train.shape = (1895, 1) x_train_10.shape = (1895, 10, 1) x_train_1.shape = (1895, 1)

Each element in x_train_10 is a list of the previous 10 values. I formatted it to follow the Keras documentation, in which repeating layers accept form input (nb_samples, timesteps, input_dim) .

I also tried using the Embedding layer with no luck. (This may not be the right way to use it - I saw that it was not used in the classification to predict).

 model = Sequential() model.add(Embedding(1, 30)) model.add(LSTM(30, 100)) ...

pad_sequences didn't work either.

 x_train_1 = sequence.pad_sequences(x_train_1, maxlen = None, dtype = "float32")

I want RNN to work with this simple data / architecture, so I can use it for more complex problems later.

Thanks:)

+8
python neural-network
source share
2 answers

I posted a similar question on the Keras Github page and got a good answer.

lukedeo said that acc: 1.0000 means that both the true conclusion and the predicted conclusion are greater than 0.5 or vice versa. Instead, I should look at loss or mse to determine the accuracy of the model. This is because my network is a regression, not a classifier / cluster.

Root mean square error is a good measure of accuracy. accuracy_percent = 1 - np.sqrt(mse)

fchollet (creator of Keras), developed by saying that "accuracy has nothing to do with the problem of regression."

When performing the classification task, accuracy may be appropriate if you set class_mode to 'categorical' or 'binary' in model.comple(...) depending on the purpose (network output).

+7
source share

Try RMSProp as an optimizer

 model.compile(loss="mean_squared_error", optimizer = 'rmsprop')
+4
source share

More articles:


All Articles