Problems Solving XOR with Genetic Algorithm

Question

I'm trying to solve XOR problem using neural network. For training I'm using genetic algorithm.

population size : 200

max_generations: 10000

crossover rate : 0,8

mutation rate : 0.1

number of weights : 9

activation function : sigmoid

selection method : high percentance for the ones with best fits

Code:

    def crossover(self,wfather,wmother):
        r = np.random.random()
        if r <= self.crossover_perc:
            new_weight= self.crossover_perc*wfather+(1-self.crossover_perc)*wmother
            new_weight2=self.crossover_perc*wmother+(1-self.crossover_perc)*wfather
            return new_weight,new_weight2
        else:
            return wfather,wmother

    def select(self,fits):
        percentuais = np.array(fits) / float(sum(fits))
        vet = [percentuais[0]]
        for p in percentuais[1:]:
            vet.append(vet[-1] + p)
        r = np.random.random()
        #print(len(vet), r)
        for i in range(len(vet)):
            if r <= vet[i]:
                return i


    def mutate(self, weight):
        r = np.random.random()
        if r <= self.mut_perc:
            mutr=np.random.randint(self.number_weights)
            weight[mutr] = weight[mutr] + np.random.normal()
        return weight

    def activation_fuction(self, net):
        return 1 / (1 + math.exp(-net))

Problem:

~5/10 tests works fine

Expected Output:

0,0 0

0,1 1

1,0 1

1,1 0

Tests:

Its inconsistent, sometimes i got four 0's, three 1's, multiple results Could you help me find the error?

**Edit

All Code:

 def create_initial_population(self):
        population = np.random.uniform(-40, 40, [self.population_size, self.number_weights])  
        return population

    def feedforward(self, inp1, inp2, weights):
        bias = 1
        x = self.activation_fuction(bias * weights[0] + (inp1 * weights[1]) + (inp2 * weights[2]))
        x2 = self.activation_fuction(bias * weights[3] + (inp1 * weights[4]) + (inp2 * weights[5]))
        out = self.activation_fuction(bias * weights[6] + (x * weights[7]) + (x2 * weights[8]))
        print(inp1, inp2, out)
        return out

    def fitness(self, weights):
        y1 = abs(0.0 - self.feedforward(0.0, 0.0, weights))
        y2 = abs(1.0 - self.feedforward(0.0, 1.0, weights))
        y3 = abs(1.0 - self.feedforward(1.0, 0.0, weights))
        y4 = abs(0.0 - self.feedforward(1.0, 1.0, weights))
        error = (y1 + y2 + y3 + y4) ** 2
        # print("Error: ", 1/error)
        return 1 / error

    def sortpopbest(self, pop):
        pop_with_fit = [(weights,self.fitness(weights)) for weights in pop]
        sorted_population=sorted(pop_with_fit, key=lambda weights_fit: weights_fit[1]) #Worst->Best One
        fits = []
        pop = []
        for i in sorted_population:
            pop.append(i[0])
            fits.append(i[1])
        return pop,fits

 def execute(self):
        pop = self.create_initial_population()
        for g in range(self.max_generations):  # maximo de geracoes
            pop, fits = self.sortpopbest(pop)
            nova_pop=[]
            for c in range(int(self.population_size/2)):
                weights =  pop[self.select(fits)]
                weights2 =  pop[self.select(fits)]
                new_weights,new_weights2=self.crossover(weights,weights2)
                new_weights=self.mutate(new_weights)
                new_weights2=self.mutate(new_weights2)
                #print(fits)
                nova_pop.append(new_weights)  # adiciona na nova_pop
                nova_pop.append(new_weights2)
            pop = nova_pop
            print(len(fits),fits)

Is that all the necessary code? Can you provide a minimal reproducible example? — AMC

maxy maxy · Accepted Answer · 2020-01-05T20:35:44

Some input:

XOR is a simple problem. With a few hundreds of random initialization, you should have some lucky ones that solve it immediately (if "solved" means that they output is correct after doing a threshold). This is a good test to see if your initialization and feed-forward pass is correct, without debugging the whole GA all at once. Or you chould just hand-craft the correct weights and biases, and see if that works.
Your initial weights (uniform -40...+40) are way too large. I guess for XOR this maybe okay-ish. But initial weights should be such that most neurons don't saturate, but aren't fully in the linear zone of sigmoid either.
After your implementation works, have a look at this numpy implementation of the feed-foward pass of a neural network for how to do it with less code.

Problems Solving XOR with Genetic Algorithm

1 Answers