Can not run tensorflow serving from docker

Question

I'm trying to serve my model tensorflow/serving with using docker. Followed by example https://www.tensorflow.org/tfx/serving/docker I do almost the same with my simple model.

In WindowsPowerShell:

set-variable -name "PATH" -variable "D:\Other\tf-serving-example\savedmodel"

savedmodel:

|-- 1
    |-- assets
    |-- saved_model.pb
    `-- variables
        |-- variables.data-00000-of-00001
        `-- variables.index

Next command:

docker run -p 8501:8501 --mount type=bind,source=$PATH,target=/models/simple_cnn_model -e MODEL_NAME=simple_cnn_model -t tensorflow/serving

But it's not working:

2021-01-19 15:32:30.681935: I tensorflow_serving/model_servers/server.cc:88] Building single TensorFlow model file config:  model_name: simple_cnn_model model_base_path: /models/simple_cnn_model
2021-01-19 15:32:30.682158: I tensorflow_serving/model_servers/server_core.cc:464] Adding/updating models.
2021-01-19 15:32:30.682192: I tensorflow_serving/model_servers/server_core.cc:587]  (Re-)adding model: simple_cnn_model
2021-01-19 15:32:30.783407: I tensorflow_serving/core/basic_manager.cc:740] Successfully reserved resources to load servable {name: simple_cnn_model version: 1}
2021-01-19 15:32:30.783506: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: simple_cnn_model version: 1}
2021-01-19 15:32:30.783531: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: simple_cnn_model version: 1}
2021-01-19 15:32:30.783643: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: simple_cnn_model version: 1} failed: Not found: Specified file path does not appear to contain a SavedModel bundle (should have a file called `saved_model.pb`) Specified file path: /models/simple_cnn_model/1

Interesing that everything is working with models from example. But my model almost the same and saved correctly. So what's wrong? Seems like --mount command does not copy content from savedmodel/1, but as fat as I understood it should make a copy of content to container.

Vlad Timofeev Vlad Timofeev · Accepted Answer · 2021-01-20T12:59:04

This commad works fine

docker run -t --rm -d -p 8501:8501 -v "D:\Other\tf-serving-example\savedmodel:/models/simple_cnn_model" -e MODEL_NAME=simple_cnn_model tensorflow/serving

Can not run tensorflow serving from docker

1 Answers