I am writing a mini neural network backprop library from scratch and I need help writing meaningful automated tests. So far, I have automated tests that confirm that the weight and displacement gradients are correctly calculated using the backprop algorithm, but there is no test to see if the workout really works.
The code that I still have allows me to do the following:
- Define a neural network with any number of layers and neurons per layer.
- It can use any level activation features.
- Offsets are also possible.
- Layers of neurons can only be fully connected at the moment.
- Training is only BP with gradient descent.
- You must use the kit for training, testing and testing (none of these kits can be empty at the moment).
Given all this, what kind of automatic test can I write to make sure that the learning algorithm is executed correctly. What function (sin, cos, exp, quadratic, etc.) should you try to approximate? In what range and how tightly should data be selected from this function? What architecture should have NN?
Ideally, the function should be simple enough to learn, so the test does not last long (1-3 seconds), but also complex enough to provide a certain degree of confidence in the correct implementation of the algorithm.
source
share