Why isn't the deep learning model converging on California Housing Price data?

Question

I'm new to deep learning, and machine learning in general. I was trying to work with the California Housing Prices dataset by passing each of the 8 features to a 5 layer network and training on the output of a price. I use MSE loss and accuracy as metric with 2000 epochs. The loss starts off as understandably large, reduces to a certain a degree and gets stuck around the same value. The accuracy is stuck at 0 for all of 2000 epochs.

I have seen a few solutions online that do things like divide the total rooms with the number of households to find the average number of rooms per household. I see that doing such feature engineering would help the model converge faster, but I had hoped the model to converge with no feature engineering at all.

From what I understood, Neural Networks are function approximators such that it builds a function from input to output of the dataset. Ideally I hoped that it would also find complex features like the ones manually computed in the online solutions. Am I wrong is having these expectations? What is the reason that the model isn't converging?

train_data = pd.read_csv('./sample_data/california_housing_train.csv', sep=',')
test_data = pd.read_csv('./sample_data/california_housing_test.csv', sep=',')

model = tf.keras.models.Sequential([
  layers.Dense(8),
  layers.Dense(5, activation=tf.nn.relu),  
  layers.Dense(7, activation=tf.nn.relu),  
  layers.Dense(5, activation=tf.nn.relu),
  layers.Dropout(0.2),
  layers.Dense(1)
])

model.compile(optimizer='adam', loss='mean_squared_error', shuffle=True, metrics=['accuracy'])

model.fit(train_data.values[:, 0:8], train_data.values[:, 8], epochs=2000)

Output

This sounds like a regression problem. You can't measure accuracy - only error. Can you show us the output of your training? — rayryeng

Parthasarathy Subburaj Parthasarathy Subburaj · Accepted Answer · 2019-06-23T06:51:19

You are solving a regression problem here, so accuracy cannot be used as a metric for evaluating your model. Instead, you could use performance measure like mae , mape, mse which are more suitable for evaluating a model which predicts a continuous target variable.

Moreover I wouldn't say that your model is not converging, if you look at the loss it is reducing constantly, so there is no problem with convergence, you just need to change your evaluation measure to something which I have mentioned above.

Please refer to How to Use Metrics for Deep Learning with Keras in Python or Usage of metrics for more details on how to implement these performance measures.

Hope this helps!

Why isn't the deep learning model converging on California Housing Price data?

2 Answers