Perceptron converging but returning odd results

Question

I made a simple perceptron in c++ to study AI and even following a book(pt_br) i could not make my perceptron return an expected result, i tryed to debug and find the error but i didnt succeed.

My algorithm AND gate results (A and B = Y):

0 && 0 = 0 
0 && 1 = 1
1 && 0 = 1
1 && 1 = 1

Basically its working as an OR gate or random.

I Tried to jump to Peter Norving and Russel book, but he goes fast over this and dont explain on depth one perceptron training.

I really want to learn every inch of this content, so i dont want to jump to Multilayer perceptron without making the simple one work, can you help?

The following code is the minimal code for operation with some explanations:

Sharp function:

int signal(float &sin){
    if(sin < 0)
        return 0;
    if(sin > 1)
        return 1;

    return round(sin);

}

Perceptron Struct (W are Weights):

struct perceptron{
    float w[3];
};

Perceptron training:

perceptron startTraining(){
    //- Random factory generator
    long int t = static_cast<long int>(time(NULL));
    std::mt19937 gen;
    gen.seed(std::random_device()() + t);
    std::uniform_real_distribution<float> dist(0.0, 1.0);
    //--

    //-- Samples (-1 | x | y)
    float t0[][3] = {{-1,0,0},
                     {-1,0,1},
                     {-1,1,0},
                     {-1,1,1}};

    //-- Expected result
    short d [] = {0,0,0,1};

    perceptron per;

    per.w[0] = dist(gen);
    per.w[1] = dist(gen);
    per.w[2] = dist(gen);

    //-- print random numbers
    cout <<"INIT "<< "W0: " << per.w[0]  <<" W1: " << per.w[1] << " W2: " << per.w[2] << endl;

    const float n = 0.1; // Lerning rate N
    int saida =0;        // Output Y
    long int epo = 0;    // Simple Couter
    bool erro = true;    // Loop control

    while(erro){
        erro = false;
        for (int amost = 0; amost < 4; ++amost) {           // Repeat for the number of samples x0=-1, x1,x2
            float u=0;                                      // Variable for the somatory
            for (int entrad = 0; entrad < 3; ++entrad) {    // repeat for every sinaptic weight W0=θ , W1, W2
                u = u + (per.w[entrad] * t0[amost][entrad]);// U <- Weights * Inputs
            }
            // u=u-per.w[0];                                // some references sau to take θ and subtract from U, i tried but without success
            saida = signal(u);                              // returns 1 or 0
            cout << d[amost] << " <- esperado | encontrado ->   "<< saida<< endl;
            if(saida != d[amost]){                          // if the output is not equal to the expected value
                for (int ajust = 0; ajust < 3; ++ajust) {
                    per.w[ajust] = per.w[ajust] + n * (d[amost] - saida) * t0[amost][ajust]; // W <- W + ɳ * ((d - y) x) where
                    erro = true;                                                             // W: Weights, ɳ: Learning rate
                }                                                                            // d: Desired outputs, y: outputs
            }                                                                                // x: samples
            epo++;

        }
    }
    cout << "Epocas(Loops): " << epo << endl;
    return per;
}

Main with testing part:

int main()
{
    perceptron per = startTraining();
    cout << "fim" << endl;
    cout << "W0: " << per.w[0]  <<" W1: " << per.w[1] << " W2: " << per.w[2] << endl;
    while(true){
        int x,y;
        cin >> x >> y;

        float u=0;
        u = (per.w[1] * x);
        u = u + (per.w[2] * y);
        //u=u-per.w[0];

        cout << signal(u) << endl;


}
    return 0;
}

If you need something just ask, im really happy to improve the question — Ollegn
You have TWO commented-out lines //u=u-per.w[0];. I'd say that you should re-enable this line in your main (but NOT in your training) and see what happens. — ThorngardSO

ThorngardSO ThorngardSO · Accepted Answer · 2016-01-15T23:24:53

In your main(), re-enable the line you commented out. Alternatively, you could write it like this to make it more illuminating:

float u = 0.0f;

u += (per.w[0] * float (-1));
u += (per.w[1] * float (x));
u += (per.w[2] * float (y));

The thing is that you trained the perceptron with three inputs, the first being hard-wired to a "-1" (making the first weight w[0] act like a constant "bias"). Accordingly, in your training function, your u is the sum of all THREE of those weight-input product. However, in the main() you posted, you omit w[0] completely, thus producing a wrong result.

Perceptron converging but returning odd results

1 Answers