Making a Postman request to Tensorflow Serving predict REST API

Question

Windows 10 and Docker tensorflow/serving. Having problem structuring JSON to request prediction and hoping that someone could help me out.

I have tried different formats of json, none of them successful.

Information about the model served by Tensorflow Serving:

http://localhost:8501/v1/models/my_model/metadata:

{
    "model_spec": {
        "name": "my_model",
        "signature_name": "",
        "version": "1"
    },
    "metadata": {
        "signature_def": {
            "signature_def": {
                "__saved_model_init_op": {
                    "inputs": {},
                    "outputs": {
                        "__saved_model_init_op": {
                            "dtype": "DT_INVALID",
                            "tensor_shape": {
                                "dim": [],
                                "unknown_rank": true
                            },
                            "name": "NoOp"
                        }
                    },
                    "method_name": ""
                },
                "serving_default": {
                    "inputs": {
                        "lstm_input": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "50",
                                        "name": ""
                                    },
                                    {
                                        "size": "5",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "serving_default_lstm_input:0"
                        },
                        "tech_input": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "1",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "serving_default_tech_input:0"
                        }
                    },
                    "outputs": {
                        "dense_out": {
                            "dtype": "DT_FLOAT",
                            "tensor_shape": {
                                "dim": [
                                    {
                                        "size": "-1",
                                        "name": ""
                                    },
                                    {
                                        "size": "1",
                                        "name": ""
                                    }
                                ],
                                "unknown_rank": false
                            },
                            "name": "StatefulPartitionedCall:0"
                        }
                    },
                    "method_name": "tensorflow/serving/predict"
                }
            }
        }
    }
}

The JSON body I believed to be correct (but it isnt) when post requesting:

{ 
   "inputs":{ 
      "lstm_input":[ 
         [ 
            0.178988,
            0.172787,
            0.182751,
            0.177997,
            0.138975
         ],
         .. ommitted, 50 in total ..
         [ 
            0.248429,
            0.244069,
            0.248119,
            0.245594,
            0.404607
         ]
      ],
      "tech_input":[ 
         [ 
            0.256009
         ]
      ]
   }
}

I was expecting a prediction (float) to be returned, instead I get the error message:

{"error": "transpose expects a vector of size 2. But input(1) is a vector of size 3\n\t [[{{node model_3/lstm_0/transpose}}]]"}

Urdar Urdar · Accepted Answer · 2019-11-03T21:57:00

I now get the expected prediction

Either of the two changes made the trick, probably the first one.

Added an extra [] surrounding the 50 elements in lstm_input

{ 
   "inputs":{ 
      "lstm_input":[ 
         [ 
            [ 
               0.178988,
               0.172787,
               0.182751,
               0.177997,
               0.138975
            ],
            .. ommitted, 50 in total ..
            [ 
               0.248429,
               0.244069,
               0.248119,
               0.245594,
               0.404607
            ]
         ]
      ],
      "tech_input":[ 
         [ 
            0.256009
         ]
      ]
   }
}

Tensorflow 2.0, created the SavedModel straight after training with model.save('YourSavedModelName', save_format='tf'). In my previous (failed attempt) I saved the model as h5, reloaded it and then saved it as SavedModel

Hopefully this information could be helpful to someone else as well

Making a Postman request to Tensorflow Serving predict REST API

1 Answers