TensorFlow serving S3 and Docker

0
votes

I’m trying to find a way to use Tensorflow serving with the ability to add new models and new versions of models. Can I point tensorflow serving to an S3 bucket?

Also I need it to run as a container? Is this possible or do I need to implement another program to pull down the model and add it to a shared volume and ask tensorflow to update models in the file system?

Or do I need to build my own docker image to be able to pull the content from s3?

2
tensorflowtensorflow-serving
I spent some time yesterday coming up with a partial solution. I needed to share a volume between containers and then call the ReloadConfigRequest GRPC endpoint. An external app pulls the model from S3 and adds to the volume then calls the endpoint. It seemed a little heavy handed but it works. I would prefer to do something not so heavy handed. - Joel Holmes

2 Answers

5
votes

I found that I could use the TF S3 connection information (even though it isn't outlined in the TF Serving Docker Container). Example docker run command:

docker run -p 8501:8501 -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -e MODEL_BASE_PATH=s3://path/bucket/models -e MODEL_NAME=model_name -e S3_ENDPOINT=s3.us-west-1.amazonaws.com -e AWS_REGION=us-west-1 -e TF_CPP_MIN_LOG_LEVEL=3 -t tensorflow/serving

Note Log level was set because of this bug

2
votes

I've submitted a very detailed answer (but using DigitalOcean Spaces instead of S3), here:

How to deploy TensorFlow Serving using Docker and DigitalOcean Spaces

Since the implementation piggy-backs off an S3-like interface, I thought I'd add the link here in case someone needs a more comprehensive example.