Predicting time series data with Neural Network in python

Question

I'm a beginner in Neural Network and trying to predict values which are temperature values(output) with 5 inputs in python. I used keras package in python to work Neural Network.

Also, I used two algorithms which are feedforward Neural Network(Regression) and Recurrent Neural Network(LSTM) to predict values. However, both of algorithms didn't work well for forecasting.

In my case of Feedforward Neural Network(Regression), I used 3 hidden layers(with 100, 200, 300 neurons) like code below,

def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(100, input_dim=5, kernel_initializer='normal', activation='sigmoid'))
    model.add(Dense(200, kernel_initializer = 'normal', activation='sigmoid'))
    model.add(Dense(300, kernel_initializer = 'normal', activation='sigmoid'))
    model.add(Dense(1, kernel_initializer='normal'))
    # Compile model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model


df = DataFrame({'Time': TIME_list, 'input1': input1_list, 'input2': input2_list, 'input3': input3_list, 'input4': input4_list, 'input5': input5_list, 'output': output_list})
df.index = pd.to_datetime(df.Time)
df = df.values

#Setting training data and test data
train_size_x = int(len(df)*0.8)     #The user can change the range of training data
print(train_size_x)
X_train = df[0:train_size_x, 0:5]
t_train = df[0:train_size_x, 6]
X_test = df[train_size_x:int(len(df)), 0:5]
t_test = df[train_size_x:int(len(df)), 6]

# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

scale = StandardScaler()
X_train = scale.fit_transform(X_train)
X_test = scale.transform(X_test)

#Regression in Keras package
clf = KerasRegressor(build_fn=baseline_model, nb_epoch=50, batch_size=5, verbose=0)

clf.fit(X_train,t_train)
res = clf.predict(X_test)

However, the error was quite big. The maximum absolute error was 78.4834. So I tried to minimize that error by changing number of hidden layer or neurons in hidden layer, but the error stayed around same.

After feedforward NN, secondly, I used Recurrent Neural Network(LSTM) algorithm which can predict by using only one input. In my case, the input is temperature. It gives me much less error than the feedforward NN, but I was lost in deep thought that Recurrent Nueral Network(LSTM) I implemented is little ambiguous in my case because it didn't use 5 inputs that affect the output(temperature value) such as feedforward regression that I implemented above.

And now I got lost what other kinds of algorithm I should use.

Any suggestions or ideas for my case..?

Thanks in advance.

It feels to me after reading your question, that you don't really have a grasp on machine learning, particularly on neural networks. Perhaps read up on the topic before deciding if you actually need to use them? Regardless of this, there is not enough context given to answer your question. To begin with: what kind of features do you have? What does your input data look like? How much data do you have? Your problem seems to lie within the machine learning methodology, not keras itself. — Tommalla
Hm.. Thanks for replying me back. I think I have start studying deep learning from basics first and then implement code or something.. — paulc1111

Fury Fury · Accepted Answer · 2017-08-02T01:54:40

I have to agree with the commenter to your question, you are jumping a little ahead of yourself. Neural networks can seem like black magic at times and its worth taking the time to understand whats actually going on under the hood. A good place to start learning and experimenting is with sklearn. Sklearn is a good place to start because you can try different techniques easily, this will help you learn quickly how to structure your problems. There is also an abundance of info and tutorials.

From there, you will be better equipped to tackling your own NN from scratch. Additionally, sklearn has many useful functions to pre-process/normalize your training data, which is a whole art in itself.

There are tons of good networks already available for common situations. Most of the work is in choosing the right structure for your problem, getting good data to train on, and massaging that data so it can be utilized properly.

Check it out... http://scikit-learn.org/stable/

Predicting time series data with Neural Network in python

1 Answers