I am using (encog 3.3.0 library) to build a neural network for image recognition. I have converted my images to 50x50 grayscale to avoid confusion for my neural network because I basically want to do some feature extraction from the images which is independent of colours.I have two output classes.
My Input :: A CSV file which contains 318 rows and each row has 2502 columns.Each row corresponds to an image. First 2500 columns are 50x50 pixels of the image and last 2 columns are the output classes. Input has 159 rows which have 2500 normal image pixels and then 1,0 as output and 159 rows which have 2500 normal image pixels and then 0,1 as output. 0 means it doesn't belong and 1 means it belongs to that class.
My input :: 318 rows and 2502 columns. Below is one of the rows ::
255,243,251,255,244,255,235,67,51,52,53,54,54,54,53,53,53,54,55,55,.......,53,54,54,53,53,52,54,54,54,54,54,54,54,54,57,57,57,57,57,57,57,57,57,57,0,1
the last 0,1 represents the output classes.
My Layers :: I have 3 Layers. Input layer which has 2500 neurons, a hidden layer which has 1000 neurons and a output layer which has 2 neurons.
The Problem: When I start training the network with learning rate 0.7 and momentum 0.8, even after 100 iterations error rate does not converge and continuously oscillates around 0.45-0.5.
Below is my code::
public class image_recognition {
static final int COLUMNS = 2500;
static final int OUTPUT = 2;
public BasicNetwork network;
public double[][] input;
public double[][] ideal;
public MLDataSet trainingSet;
public void createNetwork() {
network = new BasicNetwork();
//simpleFeedForward(int input, int hidden1, int hidden2, int output, boolean tanh)
network = EncogUtility.simpleFeedForward(image_recognition.COLUMNS, 1000, 0, image_recognition.OUTPUT, false);
network.reset();
}
public void train() {
//BasicMLDataSet(double[][] input, double[][] ideal)
trainingSet = new BasicMLDataSet(input, ideal);
//Backpropagation(ContainsFlat network, MLDataSet training, double learnRate, double momentum)
final Backpropagation train = new Backpropagation(network, trainingSet, 0.7, 0.8);
int epoch = 1;
do {
train.iteration();
System.out.println("Epoch #" + epoch + " Error:" + train.getError());
long time = System.currentTimeMillis();
System.out.println("after iteration time :: ");
System.out.println(time);
epoch++;
} while ((epoch < 5000) && (train.getError() > 0.3));
}
public double evaluate() {
System.out.println("Neural Network Results:");
for(MLDataPair pair: trainingSet ) {
final MLData output = network.compute(pair.getInput());
String actualoutput1 = String.format("%.6f", output.getData(0));
String idealoutput1 = String.format("%.1f", pair.getIdeal().getData(0));
String actualoutput2 = String.format("%.6f", output.getData(1));
String idealoutput2 = String.format("%.1f", pair.getIdeal().getData(1));
System.out.println("actual1 = " + actualoutput1 + ", actual2 = " + actualoutput2 + " ,ideal1 = " + idealoutput1 + " ,ideal2 = " + idealoutput2 );
}
return 0;
}
public void load(String filename) throws IOException {
int size = 0;
ReadCSV csv;
csv = new ReadCSV(filename, false, CSVFormat.DECIMAL_POINT);
while (csv.next()) {
size++;
}
csv.close();
// allocate enough space
input = new double[size][image_recognition.COLUMNS];
ideal = new double[size][image_recognition.OUTPUT];
// now load it
int index = 0;
csv = new ReadCSV(filename, false, CSVFormat.DECIMAL_POINT);
while (csv.next()) {
for(int i=0;i<image_recognition.COLUMNS;i++)
{
input[index][i] = Double.parseDouble(csv.get(i));
}
for(int i=0;i<image_recognition.OUTPUT;i++)
{
ideal[index][i] = Double.parseDouble(csv.get(image_recognition.COLUMNS+i));
}
index++;
}
csv.close();
}
public static void main(final String args[]) {
try {
image_recognition prg = new image_recognition();
long b1 = System.currentTimeMillis();
System.out.println("before loading time :: ");
System.out.println(b1);
prg.load("mycsv.csv");
long a1 = System.currentTimeMillis();
System.out.println("after loading, before creating network time :: ");
System.out.println(a1);
prg.createNetwork();
long a2 = System.currentTimeMillis();
System.out.println("after creating network, before training time :: ");
System.out.println(a2);
prg.train();
long a3 = System.currentTimeMillis();
System.out.println("after training, before testing time :: ");
System.out.println(a3);
prg.evaluate();
} catch (Throwable t) {
t.printStackTrace();
}
}
}
My Output::
Epoch #1 Error:0.48833917036172103
Epoch #2 Error:0.5
Epoch #3 Error:0.5
Epoch #4 Error:0.5
Epoch #5 Error:0.45956570930539425
.........
Epoch #23 Error:0.4744859426599884
Epoch #24 Error:0.5
Epoch #25 Error:0.5
...........
Epoch #49 Error:0.5912731593753425
Epoch #50 Error:0.5
Epoch #51 Error:0.5031968130459842
...........
Epoch #71 Error:0.5046318360708989
Epoch #72 Error:0.49357338328109024
Epoch #73 Error:0.486820369587797
...........
Epoch #103 Error:0.5155249407683976
Epoch #104 Error:0.4835673679113441
Epoch #105 Error:0.49407335871268354
.........
Epoch #142 Error:0.49038913805594664
Epoch #143 Error:0.4660191340060382
Please guide me why is the error rate not converging. I have tried running it for more iterations as well but still it doesn't converge. I need the error to be atleast 0.1 .
