Neural network for beginner

Question

I'm willing to create a Neural network in python, using Keras, that tells if anumber is even or odd. I know that can be done in many ways and that using NN for this is overkill but i want to this for educational purpose.

I'm running into an issue: the accuracy of my model is about 50 % that means that it's unable to tell if a number is even or odd.

I'll detail to you the step that i went through and hopefully we'll find a solution together :)

Step one creation of the data and labels: Basically my data are the number from 0 to 99(binary) and the labels are 0(odd) and 1(even)

for i in range(100):
    string = np.binary_repr(i,8)
    array = []
    for k in string:
        array.append(int(k))
    array = np.array(array)
    labels.append(-1*(i%2 - 1))

Then I'm creating the model thas is made of 3 layer. -Layer 1 (input) : one neuron that's takes any numpy array of size 8 (8 bit representation of integers) -Layer 2 (Hidden) : two neurons -Layer 3 (outuput) : one neuron

# creating a model

model = Sequential()
model.add(Dense(1, input_dim=8, kernel_initializer='uniform', activation='relu'))
model.add(Dense(2, kernel_initializer='uniform', activation='relu'))
model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))

then I'm training the model using binary_cross_entropy as a loss function since i want a binary classification of integers:

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

then I'm training the model and evaluating it:

#training
model.fit(data, labels, epochs=10, batch_size=2)

#evaluate the model
scores = model.evaluate(data, labels)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

And that's where I'm lost because of that 50 % accuracy.

I think i missundesrtood something about NN or Keras implementation so any help would be appreciated.

Thank you for reading

edit : I modified my code according to the comment of Stefan Falk

I'm quite sure the problem is that you are using raw numbers as input. You might want to extract features from your training data. Let's say.. represent each number as a binary digit in which case the network should learn that there is one bit which can tell whether a digit is even or odd. — Stefan Falk
Thank you for your quick answer Stefan, i modified my NN so it take 8 bits array as input bu im still getting my 50 % accuracy problem. I've edited my first if you want to see what i've done so far. — niklass08

Stefan Falk Stefan Falk · Accepted Answer · 2018-12-02T20:29:52

The following gives me an accuracy on the test set of 100%:

import numpy as np
from tensorflow.contrib.learn.python.learn.estimators._sklearn import train_test_split
from tensorflow.python.keras import Sequential
from tensorflow.python.keras.layers import Dense

# Number of samples (digits from 0 to N-1)
N = 10000
# Input size depends on the number of digits
input_size = int(np.log2(N)) + 1

# Generate data
y = list()
X = list()

for i in range(N):
    binary_string = np.binary_repr(i, input_size)
    array = np.zeros(input_size)
    for j, binary in enumerate(binary_string):
        array[j] = int(binary)
    X.append(array)
    y.append(int(i % 2 == 0))

X = np.asarray(X)
y = np.asarray(y)

# Make train/test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)

# Create the model
model = Sequential()
model.add(Dense(2, kernel_initializer='uniform', activation='relu'))
model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))

model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])

# Train
model.fit(X_train, y_train, epochs=3, batch_size=10)

# Evaluate    
print("Evaluating model:")
scores = model.evaluate(X_test, y_test)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

Why does it work that well?

Your problem is very simple. The network only needs to know whether the first bit is set (1) or not (0). For this you actually don't need a hidden layer or any non-linlearities. The problem can be solved with simple linear regression.

This

model = Sequential()
model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))

will do the job as well. Further, on the topic of feature engineering,

X = [v % 2 for v in range(N)]

is also enough. You'll see that X in that case will have the same content as y.

Maybe try a non-linear example such as XOR. Note that we do not have a test-set here because there's nothing to generalize or any "unseen" data which may surprise the network.

import numpy as np
from tensorflow.python.keras import Sequential
from tensorflow.python.keras.layers import Dense

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])

model = Sequential()
model.add(Dense(5, input_dim=2, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit(X, y, batch_size=1, nb_epoch=1000)

print(model.predict_proba(X))

print(model.predict_proba(X) > 0.5)

Look at this link and play around with the example.

Neural network for beginner

1 Answers

Why does it work that well?