How to do Inference with Custom Preprocessing and Data Files on Google Cloud ML

Question

I want to use a model that I have trained for inference on Google Cloud ML. It is a NLP model, and I want my node.js server to interact with the model to get predictions at train time.

I have a process for running inference on the model manually, that I would like to duplicate in the cloud:

1. Use Stanford Core NLP to tokenize my text and generate data files that store my tokenized text.
2. Have the model use those data files, create Tensorflow Examples out of it, and run the model.
3. Have the model print out the predictions.

Here is how I think I can replicate it in the Cloud:

1. Send the text to the cloud using my node.js server.
2. Run my python script to generate the data file. It seems like I will have to do this inside of a custom prediction routine. I'm not sure how I can use Stanford Core NLP here.
3. Save the data file in a bucket in Google Cloud.
4. In the custom prediction routine, load the saved data file and execute the model.

Can anyone tell me if this process is correct? Also, how can I run Stanford CoreNLP on Google Cloud custom prediction routine? Also, is there a way for me to just run command line scripts (for example for creating the data files I have a simple command that I normally just run to create them)?

Answer 1

You can implement a custom preprocessing method in Python and invoke the Stanford toolkit from there. See this blog and associated sample code for details: https://cloud.google.com/blog/products/ai-machine-learning/ai-in-depth-creating-preprocessing-model-serving-affinity-with-custom-online-prediction-on-ai-platform-serving