Sentiment classifier training with Keras

Question

I am using keras (backend tensorflow) to classify sentiments from Amazon review.

It starts with an embedding layer (which uses GloVe), then LSTM layer and finally a Dense layer as output. Model summary below:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, None, 100)         2258700   
_________________________________________________________________
lstm_1 (LSTM)                (None, 16)                7488      
_________________________________________________________________
dense_1 (Dense)              (None, 5)                 85        
=================================================================
Total params: 2,266,273
Trainable params: 2,266,273
Non-trainable params: 0
_________________________________________________________________
Train on 454728 samples, validate on 113683 samples

When training the train and eval accuracy is about 74% and loss (train and eval) around 0.6.

I've tried with changing amount of elements in LSTM layer, as well as including dropout, recurrent dropout, regularizer, and with GRU (instead of LSTM). Then the accuracy increased a bit (~76%).

What else could I try in order to improve my results?

Mohamed Elzarei Mohamed Elzarei · Accepted Answer · 2018-03-20T00:56:09

I have had a great a better success with sentiment analysis using Bidirectional LSTM also stacking two layers vertically i.e 2 LSTMS together forming a deep network also helped and try to increase the number of lstm elements to be around 128.

Sentiment classifier training with Keras

1 Answers