Multi-user training AForge.NET

Question

I use ANNA AForge.NET and train it on my training kit. Since training is single-threaded, and the process can take a lot of time, I wondered if multithreaded training could be started.

Since the problem is to use streams when training the Resilient Backpropagation network, I thought about sharing my training set between different networks and once every N-era, combining the weights of all networks into one, then duplicating it into all streams (so the following the era will begin with new weights).

I cannot find a method in AForge.NET that combines two (or more) networks. Looking for some help on how to start the implementation process.

Answer 1

Combining neural networks, each N number of iterations will not work very well. It can be very difficult, just take the weight and combine them. In a way, this is how the crossover work of the Genetic algorithm works.

Indeed, the only way you are going to do this is to modify your AForge workout to support multiple threads. Basically for this you need to match the calculation of the gradient, and then do a reduction in the sum of the gradients. Then use reduced gradients to update the network.

I implemented this thing in Encog Framework, it supports multi-threaded (RPROP) and has a C # version. http://www.heatonresearch.com/encog .