Loss and accuracy. Are these reasonable learning curves?

Question

I study neural networks and I created a simple one in Keras to classify aperture set from the UCI machine learning repository. I used one hidden layer network with 8 hidden nodes. Adam Optimizer is used with a learning speed of 0.0005 and runs on 200 Epochs. Softmax is used at the output with loss as catagoric-crossentropy. I get the following learning curves.

As you can see, the learning curve for accuracy has many flat areas, and I don’t understand why. The error seems to be constantly decreasing, but the accuracy does not seem to be increasing equally. What do flat areas in the accuracy training curve imply? Why is accuracy not increasing in these regions, although the error seems to be decreasing?

, - ?

dataframe = pd.read_csv("iris.csv", header=None)
dataset = dataframe.values
X = dataset[:,0:4].astype(float)
y = dataset[:,4]

scalar = StandardScaler()
X = scalar.fit_transform(X)

label_encoder = LabelEncoder()
y = label_encoder.fit_transform(y)

encoder = OneHotEncoder()
y = encoder.fit_transform(y.reshape(-1,1)).toarray()

# create model
model = Sequential()
model.add(Dense(8, input_dim=4, activation='relu'))
model.add(Dense(3, activation='softmax'))

# Compile model
adam = optimizers.Adam(lr=0.0005, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='categorical_crossentropy',
              optimizer=adam, 
              metrics=['accuracy'])

# Fit the model
log = model.fit(X, y, epochs=200, batch_size=5, validation_split=0.2)

fig = plt.figure()
fig.suptitle("Adam, lr=0.0006, one hidden layer")

ax = fig.add_subplot(1,2,1)
ax.set_title('Cost')
ax.plot(log.history['loss'], label='Training')
ax.plot(log.history['val_loss'], label='Validation')
ax.legend()

ax = fig.add_subplot(1,2,2)
ax.set_title('Accuracy')
ax.plot(log.history['acc'], label='Training')
ax.plot(log.history['val_acc'], label='Validation')
ax.legend()

fig.show()

Answer 1

( ) , (. , )...

, ; () :

• y[i] - (0 1)
• p[i] - ( [0,1]),
• output[i] ( ) p[i], 0 1; , ( 0.5 ), p[i] > 0.5, output[i] = 1, , p[i] <= 0.5, output[i] = 0.

, y[k] = 1, p[k] = 0.1; , :

• loss[k] = -log(0.1) = 2.3
• p[k] < 0.5, output[k] = 0, 0 ( )

, , , p[k] = 0.22; :

• loss[k] = -log(0.22) = 1.51
• p[k] < 0.5, (output[k] = 0)

, , , , , p[k] = 0.49; :

• loss[k] = -log(0.49) = 0.71
• output[k] = 0, ..

, , .. 2,3 1,5 0,71, , : , , p[k], 0,5.

, p[k] 0,5, , , 0 1/n, n - .

, , , p[k] 0,5, , ( ), (.. 1.0) , .

, y[m] = 0 p[m] - 0,5; p[m] 0,5 (, ), 0.0 .

, , , "" , .

---

: , "" - ; ( - , 0,5). :

- ; , - , , - , () . , "" - () , , ( , , - , , , , , RMSE)...