Prediction Failed on keras model in Google ML-Engine

Question

I am working through a tutorial at the moment which covers using a Keras based model on Google's cloud services with their ML-Engine.

At this stage I have the model working fine for local predictions etc, and have successfully placed the exported model into a GC-bucket. I have also created the google cloud ML-Engine model successfully.

When I try to run a prediction off the cloud hosted model, I the following error is produced.

Error:

C:\mydir>gcloud ml-engine predict --model=[mymodel] --json-instances=sample_input_prescaled.json

        {
      "error": "Prediction failed: Error during model execution: AbortionError(code=StatusCode.FAILED_PRECONDITION, details=\"Attempting to use uninitialized value dense_4/bias\n\t [[Node: dense_4/bias/read = Identity[T=DT_FLOAT, _class=[\"loc:@dense_4/bias\"], _output_shapes=[[1]], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](dense_4/bias)]]\")"
    }

I can see that this error is refering to an uninitialised value 'dense_4', which looks like the last layer within the Keras model, but I'm not sure if/why this would be tripping up the process?

Does anyone have some insight to the cause of this error message?

Below is the keras model which I am using from the tutorial, and the json file for the test prediction.

export_model.py

import pandas as pd
import keras
from keras.models import Sequential
from keras.layers import *
import tensorflow as tf

training_data_df = pd.read_csv("sales_data_training_scaled.csv")

X = training_data_df.drop('total_earnings', axis=1).values
Y = training_data_df[['total_earnings']].values

# Define the model
model = Sequential()
model.add(Dense(50, input_dim=9, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam')



# Create a TensorBoard logger
logger = keras.callbacks.TensorBoard(
    log_dir='logs',
    histogram_freq=5,
    write_graph=True
)

# Train the model
model.fit(
    X,
    Y,
    epochs=50,
    shuffle=True,
    verbose=2
)

# Load the separate test data set
test_data_df = pd.read_csv("sales_data_test_scaled.csv")

X_test = test_data_df.drop('total_earnings', axis=1).values
Y_test = test_data_df[['total_earnings']].values

test_error_rate = model.evaluate(X_test, Y_test, verbose=0)
print("The mean squared error (MSE) for the test data set is: {}".format(test_error_rate))


model_builder = tf.saved_model.builder.SavedModelBuilder("exported_model")

inputs = {
    'input': tf.saved_model.utils.build_tensor_info(model.input)
}
outputs = {
    'earnings': tf.saved_model.utils.build_tensor_info(model.output)
}

signature_def = tf.saved_model.signature_def_utils.build_signature_def(
    inputs=inputs,
    outputs=outputs,
    method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)

model_builder.add_meta_graph_and_variables(
    K.get_session(),
    tags=[tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature_def
    }
)

model_builder.save()

sample_input_prescaled.json

{ "input": [0.4999, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.5] }

Answer 1

Following the same tutorial, I found changing:

inputs = {
    'input': tf.saved_model.utils.build_tensor_info(model.input)
}
outputs = {
    'earnings': tf.saved_model.utils.build_tensor_info(model.input)
}

to:

inputs = {
    'input': tf.compat.v1.saved_model.utils.build_tensor_info(model.input)
}
outputs = {
    'earnings': tf.compat.v1.saved_model.utils.build_tensor_info(model.output)
}

When exporting the model, solved the problem for me, the former are deprecated.

Hope this helps.

Answer 2

The Above Code with the following inputs worked for me.

X = np.random.rand(1000,9)
Y = np.random.rand(1000,1)

Then I used following code.

from keras import backend as K    
sess = K.get_session()
input_tensor = model.input
output_tensor = model.output
output_tensor.eval(feed_dict={input_tensor: np.random.rand(1,9)}, 
session=sess) 

Next, I export the model. Please make sure that the exported model works fine for you before using serving functions.

 export_dir = ...
      with tf.Session(graph=tf.Graph()) as sess:
      tf.saved_model.loader.load(sess, [tag_constants.TRAINING], export_dir)

And it worked. Then, I used the following serving function in a task.py to serve the JSON input and again it worked.

 def json_serving_input_fn():
   inputs = {}
   for feat in 9:
      inputs[feat.name] = tf.placeholder(shape=[None], dtype=feat.dtype)
   return tf.estimator.export.ServingInputReceiver(inputs, inputs)

So, I suspect that your input has not been fed correctly.