No such file or directory: '/tmp/model/0001' google cloud ai platform when using the custom prediction routine

Question

The /tmp/model folder on the gcloud instance seems to be empty. It looks like the files are not been copied over correctly from the storage account even though the stderr log says otherwise.

Please how do I go about this and what am I missing. I am able to create the model version successfully when I make a request to predict.

error message

Here is the command used to create the model version:

gcloud beta ai-platform versions create $VERSION_NAME  --model $MODEL_NAME  --runtime-version 1.15  --python-version 3.7  --origin gs://$BUCKET_NAME/custom_prediction_routine/model/  --package-uris gs://$BUCKET_NAME/custom_prediction_routine/custom_predict_code-0.1.tar.gz  --prediction-class predictor.MyPredictor

And this is the from_path class method:

@classmethod
def from_path(cls, model_dir):
    sys.stderr.write(str(model_dir))
    return cls(model_dir)

I don't think the AI platform, but can you show us how you've set it up to copy files over? if this is the stderr log and you meant the '/tmp/model/0001/ there I'd guess that's just tracing out what directory it's expecting to find the model in, and isn't confirming that it's copied the files - unless you can show us the code that's producing that log? — Rup
@Rup, When creating a model version, you are to specify a google cloud storage bucket link to the models. They handle the copying automatically on their end, I do not have to specify how it is copied. At the point of predicting, they, in turn, give you a dir that your model was copied to. The current issue now is that the directory they return at the point of prediction is actually empty — Oyinkansola Ariyo
OK thanks. I was mostly asking for you to show us what you’re actually doing, so someone who knows about this could tell you what you’ve done wrong, since you haven’t shown us any configuration or code. But it sounds like it’s entirely out of your hands then, so you’ll have to ask Google support I think. — Rup
Can you provide the command used to deploy the version, as well as the code of the "from_path" class method? — Tlaquetzal
@Tlaquetzal , Here is the command I used in creating the model version gcloud beta ai-platform versions create $VERSION_NAME --model $MODEL_NAME --runtime-version 1.15 --python-version 3.7 --origin gs://$BUCKET_NAME/custom_prediction_routine/model/ --package-uris gs://$BUCKET_NAME/custom_prediction_routine/custom_predict_code-0.1.tar.gz --prediction-class predictor.MyPredictor And this is the from path class method def from_path(cls, model_dir): sys.stderr.write(str(model_dir)) return cls(model_dir) This is the method giving the wrong path — Oyinkansola Ariyo

Tlaquetzal Tlaquetzal · Accepted Answer · 2020-08-24T19:56:40

Based on the code you shared, it seems that you are not loading the model directly in from_path and you're sending the path directly to your Predictor instance.

Try instead loading the model directly in the from_path, for example, using a keras model:

@classmethod
def from_path(cls, model_dir):
    model = keras.models.load_model(model_dir) # Load with Keras, or the appropriate format
    return cls(model)

If the issue persists, try making a warm up predict request from inside the from_path function. For example:

@classmethod
def from_path(cls, model_dir):
    model = keras.models.load_model(model_dir)
    predictor = cls(model)
    outputs = predictor.predict([[1,2,3,4,5]]) # Here goes your warm up prediction request 
    return predictor

EDIT

The issue was reported here in a Google Issue Tracker. The response was that both the model and all the artifacts that will be used from model_dir needs to be loaded during the execution of from_path and stored in the predictor class if they're going to be used during the predictions.

No such file or directory: '/tmp/model/0001' google cloud ai platform when using the custom prediction routine

1 Answers