Issue in solving XOR using neural network and genetic algorithm

Question

I'm trying to solve XOR problem using neural network. For training I'm using genetic algorithm.

But after a certain number of generations (200), the error stuck in 1. And the outputs are correct except for 1 xor 0 where the output is 0 instead of 1 I don't understand why this is happening.

population size : 100
crossover rate : 70
mutation rate : 5
elite number : 2
activation function : sigmoid
selection method : Tournament Selection with 7 participant

Mutation algorithm =

for (int i=0; i< individual.getNbrOfWeights(); i++)

    if (random(0,100) < mutationRate)
    {
        genome[i] = genome[i] + random(-0.1,0.1);
    }

Fitness calculation =

double error = 0;

error = error + feedForward({0, 1}, 1);
error = error + feedForward({1, 0}, 1);
error = error + feedForward({1, 1}, 0);
error = error + feedForward({0, 0}, 0);     

fitness = error;

where the error is the target-output

I tried in mutation to set weights in range of [-2 2], but it get even worst (the error stuck in 1.6 ). So I don't now if it is obligated to set weight in a certain range...

I really need your help, thanks in advance.

EDIT

In fact the problem was in weight initialization and mutation method.

When I set weights between [-1 1], the algorithm does not converge. But the more I enlarge the range, the more it gives better results, say between [-4 4].
for the mutation, I tried two methods (mutate one gene chosen randomly):

--> Add a random perturbation between [-0.1 0.1], with 5% mutation rate. This way, I got the best results (network outputs like desired ones) at generation 1800

--> Change the value of the gene with a new one. The new value should belong to the range. In this case, I had to put the mutation rate to 50% for that the algorithm could converge. And weights had to be between at least -7 and 7, otherwise it won't converge.

Have you checked if the population converges too quickly to a local minimum? Have a look, generation by generation, at how the population changes, if there are correct solutions (you can identify them by the fitness value), why they disappear, or why they never appear, etc. — rand

Anton Danilov Anton Danilov · Accepted Answer · 2015-02-21T10:43:20

You specified sigmoid as the activation function, but you apply the negative numbers, sigmoid works with positive numbers. I think it's the cause of the problem. You should specify another activation function e.g. hyperbolic tangent function (tanh).

EDIT: I'm pretty sure by now: the point is the fitness value. I suspected that you evaluating it as the error (i.e. minimazing it) at the same time selecting individuals as better ones with greater fitness value.

Issue in solving XOR using neural network and genetic algorithm

1 Answers