keras add starting weights based on columns

Question

I'm using Keras with the LSTM to make a neural network . The network doesn't predict that well and i want to help it by adding some starting weights.

For exemple I want to add some starting weight to col4 so that the neural network knows that col4 is 3 times more important that col2 and col3 for predicting col_y

timestamp,      col_y,  co2, col3, col4
9-1-2019 00:00, 136228, 4.9, 19,    0
9-1-2019 01:00, 123012, 4.7, 17,    0
9-1-2019 02:00, 117309, 4.6, 23,    0
9-1-2019 03:00, 114310, 4.5, 12,    0
9-1-2019 04:00, 114096, 4.3, 5,     0
9-1-2019 05:00, 119260, 4.1, 2,     0
9-1-2019 06:00, 136643, 3.9, 13,    5
9-1-2019 07:00, 177303, 3.7, 23,    11
9-1-2019 08:00, 187407, 3.7, 5,     17
9-1-2019 09:00, 173752, 4.0, 7,     55

That is not a matter of architecture but it's preprocessing part. — Minh-Long Luu
oke, so that means that I have to normilize the data? (because in the real dataset i've done that and that also didn't change) — Ultiseeker
Yeah, try normalizing/standardizing the data, because they are vulnerable to data scaling and outliers. — Minh-Long Luu
so if I want to say col4 is 3 times more important; I need to normalize the data and then do col2 and col3 devided by 3? or is there an other step to it? — Ultiseeker
You don't have to. The network auto decides which feature is more important during training and adjusts the weights according to it. You only need to make sure the data is well processed though. — Minh-Long Luu

Tfer3 Tfer3 · Accepted Answer · 2020-12-03T10:39:34

You can achieve this with the preprocessing technique. Instead of just randomly assigning the weight to a certain feature you can perform normalization on your entire dataset with some of the standard normalization techniques like Mean subtraction, PCA etc this will help your model to converge with less amount of training.

You can also do Feature importance to check how individual feature is contributing to your model. Based on that you can make changes in the model to improve accuracy. There are several methods to check feature importance like LIME, SHAP any many others to choose from.

keras add starting weights based on columns

1 Answers