Elman network not stop

Question

My Elman network training for XOR operator does not stop, it runs into millions iterations. Any help would be much appreciated!

package org.encog.example;

import org.encog.Encog;
import org.encog.engine.network.activation.ActivationSigmoid;
import org.encog.ml.train.MLTrain;
import org.encog.neural.data.NeuralDataSet;
import org.encog.neural.data.basic.BasicNeuralDataSet;
import org.encog.neural.networks.BasicNetwork;
import org.encog.neural.networks.training.propagation.back.Backpropagation;
import org.encog.neural.pattern.ElmanPattern;

public class XORRNN {

    // 4 row, 2 column
    public static double XOR_INPUT[][] = { { 0.0, 0.0 }, { 1.0, 0.0 },
            { 0.0, 1.0 }, { 1.0, 1.0 } };

    // 4 row, 1 column
    public static double XOR_IDEAL[][] = { { 0.0 }, { 1.0 }, { 1.0 }, { 0.0 } };

    public static void main(String[] args) {

        //create Elman RNN
        ElmanPattern elmanPattern = new ElmanPattern();
        elmanPattern.setInputNeurons(2);
        elmanPattern.addHiddenLayer(4);
        elmanPattern.setOutputNeurons(1);
        elmanPattern.setActivationFunction(new ActivationSigmoid());
        BasicNetwork network = (BasicNetwork) elmanPattern.generate();

        //read training data
        NeuralDataSet trainingSet = new BasicNeuralDataSet(XOR_INPUT, XOR_IDEAL);

        //set training method
        MLTrain train = new Backpropagation(network, trainingSet, 0.000001, 0.0);


        //training
        int epoch = 1;

        do{
            train.iteration();          
            System.out.println("Iteration: " + epoch + ", Error: " + train.getError());
            epoch ++;
        }while(train.getError() > 0.01);

        //shut down
        Encog.getInstance().shutdown();
    }
}

JeffHeaton JeffHeaton · Accepted Answer · 2014-11-11T00:53:23

If you want to train something that is NOT time-series, try using a feedforward neural network. You are not going to be very successful with an Elman and just 4 training set elements. If you want an example of how to structure XOR data for an Elman, see the following:

https://github.com/encog/encog-java-examples/blob/master/src/main/java/org/encog/examples/neural/recurrent/elman/ElmanXOR.java

Elman network not stop

1 Answers