ANN - Low performance

Question

I´m new at neural networks and I just defined my first artificial neural network as follows:

model = Sequential()
model.add(Dense(25,input_dim = 20, activation = 'relu'))
model.add(Dense(50,activation='relu'))
model.add(Dense(10,activation='relu'))
model.add(Dense(20,activation='relu')) 
model.add(Dense(4,activation='softmax'))

Therefore, it has 4 hidden layers (first one working as input) and the output layer.

Then, I have compiled the model using Adam optimizer with a learning rate of 0.2 and categorical crossentropy because I´m dealing with a multiclass problem. See below:

adam = optimizers.Adam(lr = 0.2)
model.compile(loss='categorical_crossentropy', optimizer= adam, metrics=['accuracy'])

When checking the performance of the model (accuracy & loss) is pretty low. See below the results:

Please, see results with learning rate decreased to lr = 0.001:

Please, see info about the dataset below:

    RangeIndex: 2000 entries, 0 to 1999
Data columns (total 21 columns):
battery_power    2000 non-null int64
blue             2000 non-null int64
clock_speed      2000 non-null float64
dual_sim         2000 non-null int64
fc               2000 non-null int64
four_g           2000 non-null int64
int_memory       2000 non-null int64
m_dep            2000 non-null float64
mobile_wt        2000 non-null int64
n_cores          2000 non-null int64
pc               2000 non-null int64
px_height        2000 non-null int64
px_width         2000 non-null int64
ram              2000 non-null int64
sc_h             2000 non-null int64
sc_w             2000 non-null int64
talk_time        2000 non-null int64
three_g          2000 non-null int64
touch_screen     2000 non-null int64
wifi             2000 non-null int64
price_range      2000 non-null int64
dtypes: float64(2), int64(19)

Dataset has been normalized and One Hot encoding applied to price_range attribute which is the one containing the 4 labels.

What exactly is the training data? Please show examples from your training set. How many samples does it have? Please show your entire keras code, not just the model part. Are you using early stopping? — Mathias Müller
Hi Mathias, dataset info added. I´m not using early stopping — Panri93

Roy Levy Roy Levy · Accepted Answer · 2020-03-07T12:13:26

From a first glance, the model you described is supposed to work fine for multiclass problem. So as for now, it seems like the problem is with the dataset itself - one of the following:

Not enough examples for the complexity of the problem in the provided dataset. Also make sure that the data is representing the problem both in training and test sets - try doing k-fold cross validation for this purpose.
Noisy data - if your dataset allows mistakes in the data labeling or wrong attribute values it could cause the problem when training the model.

Edit: As Dr. H. Lecter stated, in the comment - the problem also could be caused by missing local minimals for convergions, caused sometimes by large values of learning rate. You could try decreasing learning rate value in order to solve it and converge. Also trying different number of layers and nodes in each layer could help. These are common solvers for cases like yours, but if these are not the root causes - could you provide more information on the problem?

ANN - Low performance

1 Answers