Conceptual issues of training a neural network with particle swarm optimization

Question

I have a 4-input and 3-wire neural network trained in particle optimization (PSO) with mean square error (MSE) as a fitness function using the IRIS database provided by MATLAB. The fitness function is rated 50 times. The experiment is to classify functions. I have a few doubts

(1) Iterations / generation PSO = number of times the fitness function is evaluated?

(2) In many works I have seen that the MSE versus generational learning curve is a plot. In the figure, graph (a) on the left side is a model similar to NN. This is a cognitive output map 3 of hidden layer-3. Graph (b) is an NN trained by the same PSO. The aim of this work was to show the effectiveness of the new model in (a) compared to NN.

But they mention that the experiment is conducted, say, Cycles = 100 times with Generations = 300. In this case, the learning curve for (a) and (b) should be MSE vs Cycles, and not MSE and PSO generations? For example, Cycle1: Initiate PSO 1-50 → Result (Weights_1, Bias_1, MSE_1, Classification Rate_1). Cycle2: Iterate PSO 1-50 → Result (Weights_2, Bias_2, MSE_2, Classification Rate_2), etc. For 100 cycles. Why is the X axis in (a), (b) different and what do they mean?

(3) Finally, for each independent launch of the program (by running the m file several times independently, through the console), I never get the same classification speed (CR) or the same set of weights. Specifically, when I first run the program, I get the values ​​W (Weights) and CR = 100%. When I run the Matlab code program again, I can get CR = 50% and another set of weights !! As shown below for an example,

%Run1 (PSO generaions 1-50) >>PSO_NN.m Correlation = 0 Classification rate = 25 FinalWeightsBias = -0.1156 0.2487 2.2868 0.4460 0.3013 2.5761 %Run2 (PSO generaions 1-50) >>PSO_NN.m Correlation = 1 Classification rate = 100 %Run3 (PSO generaions 1-50) >>PSO_NN.m Correlation = -0.1260 Classification rate = 37.5 FinalWeightsBias = -0.1726 0.3468 0.6298 -0.0373 0.2954 -0.3254 

What should be the correct method? So, how much weight should I take, and how can I say that the network has been trained? I know that evolutionary algorithms, due to their randomness, will never give the same answer, but how can I ensure that the network is trained? Must be clarified.

Answer 1

• As with most machine learning methods, the number of iterations in a PSO is the number of times the solution is updated. In the case of PSO, this is the number of update rounds for all particles. The cost function here is estimated after each particle is updated, therefore, more than the number of iterations. Approximately (# function call functions) = (# iterations) * (# particles).
• The graphs here compare different classifiers, fuzzy cognitive maps for graph (a) and the neural network for graph (b). Thus, the X axis displays the appropriate measures for studying the iterations for each of them.
• Each time you start NN, you initialize it with various random values, so the results never match. The fact that the results vary greatly from one run to the next means that you have a convergence problem. The first thing to do in this case is to try to run more iterations. In general, convergence is a rather complex problem, and the solutions vary greatly for applications (and carefully read the answer and comments that Isaac gave you to your other question ). If your problem persists after increasing the number of iterations, you can publish it as a new question by providing a sample of your data and the actual code that you use to build and train the network.