How to improve accuracy with keras multi class classification?

1
votes

I am trying to do multi class classification with tf keras. I have total 20 labels and total data I have is 63952and I have tried the following code 

features = features.astype(float)
labels = df_test["label"].values

encoder = LabelEncoder()
encoder.fit(labels)
encoded_Y = encoder.transform(labels)
dummy_y = np_utils.to_categorical(encoded_Y)

Then 

def baseline_model():
    model = Sequential()
    model.add(Dense(50, input_dim=3, activation='relu'))
    model.add(Dense(40, activation='softmax'))
    model.add(Dense(30, activation='softmax'))
    model.add(Dense(20, activation='softmax'))

    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

finally 

history = model.fit(data,dummy_y, 
                    epochs=5000,
                    batch_size=50,
                    validation_split=0.3,
                    shuffle=True,
                    callbacks=[ch]).history

I have a very poor accuray with this. How can I improve that ?

2
machine-learningkerasmulticlass-classification

2 Answers

2
votes

softmax activations in the intermediate layers do not make any sense at all. Change all of them to relu and keep softmax only in the last layer.

Having done that, and should you still be getting unsatisfactory accuracy, experiment with different architectures (different numbers of layers and nodes) with a short number of epochs (say ~ 50), in order to get a feeling of how your model behaves, before going for a full fit with your 5,000 epochs.

1
votes

You did not give us vital information, but here are some guidelines:
1. Reduce the number of Dense layer - you have a complicated layer with a small amount of data (63k is somewhat small). You might experience overfitting on your train data.
2. Did you check that the test has the same distribution as your train?
3. Avoid using softmax in middle Dense layers - softmax should be used in the final layer, use sigmoid or relu instead.
4. Plot a loss as a function of epoch curve and check if it is reduces - you can then understand if your learning rate is too high or too small.