tensorflow serving restful request how to post a list Object

Question

I made a tensorflow Model and i use tensorflow serving to deploy this model, but when i can build the restful request params the model need

curl -d '{"instances": [[1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]}'
-X POST http://localhost:8501/v1/models/shipping_predict:predict

it call back "error": "instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"

this is my model

# prepare each input head
in_layers = list()
em_layers = list()
#
# customer
in_layer_customer = Input(shape=(1,))
em_layer_customer = Embedding(5000, 10)(in_layer_customer)
em_layer_customer = layers.Reshape([1 * 10])(em_layer_customer)
in_layers.append(in_layer_customer)
em_layers.append(em_layer_customer)

# salesman
in_layer_sale = Input(shape=(1,))
em_layer_sale = Embedding(500, 10)(in_layer_sale)
em_layer_sale = layers.Reshape([1 * 10])(em_layer_sale)
in_layers.append(in_layer_sale)
em_layers.append(em_layer_sale)

# business_type
in_layer_businessType = Input(shape=(1,))
em_layer_businessType = Embedding(100, 10)(in_layer_businessType)
em_layer_businessType = layers.Reshape([1 * 10])(em_layer_businessType)
in_layers.append(in_layer_businessType)
em_layers.append(em_layer_businessType)

# 20
in_layer_20 = Input(shape=(1,))
em_layer_20 = layers.Dense(16, activation='relu')(in_layer_20)
in_layers.append(in_layer_20)
em_layers.append(em_layer_20)

# 40
in_layer_40 = Input(shape=(1,))
em_layer_40 = layers.Dense(16, activation='relu')(in_layer_40)
in_layers.append(in_layer_40)
em_layers.append(em_layer_40)

# 45
in_layer_45 = Input(shape=(1,))
em_layer_45 = layers.Dense(16, activation='relu')(in_layer_45)
in_layers.append(in_layer_45)
em_layers.append(em_layer_45)

# other 
in_layer_other = Input(shape=(1,))
em_layer_other = layers.Dense(16, activation='relu')(in_layer_other)
in_layers.append(in_layer_other)
em_layers.append(em_layer_other)

# dischargingPort
in_layer_dischargingPortName = Input(shape=(1,))
em_layer_dischargingPortName = Embedding(5000, 10)(in_layer_dischargingPortName)
em_layer_dischargingPortName = layers.Reshape([1 * 10])(em_layer_dischargingPortName)
in_layers.append(in_layer_dischargingPortName)
em_layers.append(em_layer_dischargingPortName)

# MBL Method
in_layer_mbl = Input(shape=(1,))
em_layer_mbl = layers.Dense(16, activation='relu')(in_layer_mbl)
in_layers.append(in_layer_mbl)
em_layers.append(em_layer_mbl)

# atdMouth
in_layer_atdMouth = Input(shape=(1,))
em_layer_atdMouth = Embedding(100, 10)(in_layer_atdMouth)
em_layer_atdMouth = layers.Reshape([1 * 10])(em_layer_atdMouth)
in_layers.append(in_layer_atdMouth)
em_layers.append(em_layer_atdMouth)

merge = Concatenate()(em_layers)
dense = Dense(32, activation='relu')(merge)

output = Dense(1)(dense)
model = Model(inputs=in_layers, outputs=output)

Eric McLachlan Eric McLachlan · Accepted Answer · 2020-12-18T10:51:36

instances is a plain list, but expecting list of objects as multiple input tensors

It sounds like your model, for whatever reason, is expecting named tensors. This is not something I've worked with before but there appears to be another way of sending requests to your your model.

curl -X POST -i 'http://192.168.1.16:8501/v1/models/export:predict' --data '
{
  "signature_name":  "serving_default",
  "inputs": [
    {
      "tokens_0" :["text text text text text text text text text text"],
      "length_0": [1],
      "tokens_1": ["01 01 01 01 01 01 01 01 01 01"],
      "length_1": [1],
      "tokens_2": ["4 4 4 1 1 4 4 4 4 4"],
      "length_2": [1]
    }
  ]
}'

I've just copied this example from here (credit to @ishaan-sharma).

Your model is non-trivial so I won't try and create the exact request for you. If you're unsure about the tensor names etc, you can check out the expected shape using the saved_model_cli:

saved_model_cli show [-h] --dir DIR [--all]

tensorflow serving restful request how to post a list Object

1 Answers