Tensorflow Serving in Amazon SageMaker

2
votes

I am facing an issue with serving tensorflow models on AWS SageMaker. I have trained the model outside of the SageMaker environment, now I have a savedmodel.pb file and I need to deploy it on a SageMaker endpoint. So I simply zipped the model file and uploaded it to an S3 bucket. Now, when trying to create an endpoint, I get the following error in my Cloudwatch log:

tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path /opt/ml/model/export/Servo for servable generic_model

I believe SageMaker is looking for the tar.gz to follow a particular directory structure. However, all I have is a .pb file.

1
pythonamazon-web-servicestensorflowtensorflow-servingamazon-sagemaker
Can you verify that you also have a non-empty variables folder next to your .pb file? - sdcbr
The variables folder is empty. - Vedant Bajaj
Not sure, but the error suggests that you are pointing to the wrong path when setting up the server or so. - sdcbr
@sdcbr does the tar.gz file need to follow any particular directory structure? Currently my tar.gz file only has the pb file (in root) - Vedant Bajaj
@VedantBajaj did you get it to work? Do you mind sharing how? I am stuck on this too. - Pramesh Bajracharya

1 Answers

0
votes

TensorFlow Serving expects the following folder structure:

export/Servo/{version_number} where the version number is any valid positive number.

SageMaker expects the same directory format of TFS, there is a GH issue about this https://github.com/aws/sagemaker-python-sdk/issues/599