Deploying Mask RCNN object detection model to GCP AI Platform that accepts base64 encoded images

Question

Please help! I have been stuck for weeks on getting any type of prediction response for my mask rcnn object detection model in GCP ai platform. So far, i have trained a simple model on about 200 images which outputs a weights file in h5 format using the matterport repo. In a new python script, I load those weights like this:

# LOAD MODEL
from config import mask_config
from model import MaskRCNN

config = get_config()

model = MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
model.load_weights(H5_WEIGHT_PATH, by_name=True)

then, I created a frozen graph .pb file using the following code:

def freeze_model(model, name):
    frozen_graph = freeze_session(
        sess,
        output_names=[out.op.name for out in model.outputs][:4])
    directory = PATH_TO_SAVE_FROZEN_PB
#     directory = './'
    tf.train.write_graph(frozen_graph, directory, name , as_text=False)
    print("*"*80)
    print("Finish converting keras model to Frozen PB")
    print('PATH: ', PATH_TO_SAVE_FROZEN_PB)
#     print('PATH: ', './')
    print("*" * 80)

freeze_model(model.keras_model, FROZEN_NAME)

So far so good! I then continue to make my model tensorflow serving ready as follows:

def make_serving_ready(model_path, save_serve_path, version_number):
    import tensorflow as tf

    export_dir = os.path.join(save_serve_path, str(version_number))
    graph_pb = model_path

    builder = tf.saved_model.builder.SavedModelBuilder(export_dir)

    with tf.gfile.GFile(graph_pb, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())

    sigs = {}
    
    # tf.import_graph_def(graph_model_def, name='', input_map={"input_image": img_uint8})
    with tf.Session(graph=tf.Graph()) as sess:
        # name="" is important to ensure we don't get spurious prefixing
        tf.import_graph_def(graph_def, name="")
        g = tf.get_default_graph()
        input_image = g.get_tensor_by_name("input_image:0")
        input_image_meta = g.get_tensor_by_name("input_image_meta:0")
        input_anchors = g.get_tensor_by_name("input_anchors:0")

        output_detection = g.get_tensor_by_name("mrcnn_detection/Reshape_1:0")
        output_mask = g.get_tensor_by_name("mrcnn_mask/Reshape_1:0")

        sigs[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY] = \
            tf.saved_model.signature_def_utils.predict_signature_def(
                {"input_image": input_image, 'input_image_meta': input_image_meta, 'input_anchors': input_anchors},
#                 {"image_bytes": img_uint8, 'input_image_meta': input_image_meta, 'input_anchors': input_anchors},
                {"mrcnn_detection/Reshape_1": output_detection, 'mrcnn_mask/Reshape_1': output_mask})

        builder.add_meta_graph_and_variables(sess,
                                             [tag_constants.SERVING],
                                             signature_def_map=sigs)

    builder.save()
    print("*" * 80)
    print("FINISH CONVERTING FROZEN PB TO SERVING READY")
    print("PATH:", PATH_TO_SAVE_TENSORFLOW_SERVING_MODEL)
    print("*" * 80)

# Now convert frozen graph to Tensorflow Serving Ready
make_serving_ready(os.path.join(PATH_TO_SAVE_FROZEN_PB, FROZEN_NAME),
                     PATH_TO_SAVE_TENSORFLOW_SERVING_MODEL,
                     VERSION_NUMBER)

print("COMPLETED")

I then deploy the output of the above code (saved_model.pb) to ai platform models

The thing i'm trying to figure out is - how can i modify the above code to accept base64 encoded images? I have deployed the model successfully onto GCP AI Platform but when I do a sample input to test the predictions, i need to use the gcp requested format:

{"instances":[
{"image_bytes":{"b64":abcdefg},{"key":"1"}
]}

So when i convert the image to a base64 encoded image and input the above format, i get this error:

{
  "error": "{ \"error\": \"Failed to process element: 0 key: image_bytes of \\'instances\\' list. Error: Invalid argument: JSON object: does not have named input: image_bytes\" }"
}

So then I went back to my code and tried to change my input_image variable to accept decoded image format:

                # concatenate decoder graph and original graph
                image = tf.map_fn(decode_and_resize, image_str_tensor, back_prop=False, dtype=tf.uint8)
                tf.import_graph_def(graph_def, name="", input_map={'input_image:0':image})

but then i get this error:

ValueError: Input 0 of node zero_padding2d_1/Pad_1 was passed uint8 from decoder/map/TensorArrayStack/TensorArrayGatherV3:0 incompatible with expected float.

So i'm officially completely clueless on how to get this thing running. Is there anyone out there that can fix this??!!!

Morgan Du Morgan Du · Accepted Answer · 2020-12-15T00:20:42

According to https://cloud.google.com/ai-platform/prediction/docs/online-predict#for---json-request

The encoded string must be formatted as a JSON object with a single key named b64.

{
  "instances": [
    {
      "image_bytes": {"b64": "ASa8asdf"}
    },
    {
      "image_bytes": {"b64": "JLK7ljk3"}
    }
  ]
}

And in your TensorFlow model code, you must name the aliases for your binary input and output tensors so that they end with '_bytes'.

Deploying Mask RCNN object detection model to GCP AI Platform that accepts base64 encoded images

1 Answers