Google Cloud Platform - AI Platform: why do I get different response body when calling API?

2
votes

I created 2 models on Google Cloud AI Platform and I am wondering why do I get different response body when calling REST API with Python?
To be more specific:

  • In the first case, I get 2 dictionaries (keys: "predictions" and "dense_1", the latter is the output layer name of my tensorflow model)
    {'predictions': [{'dense_1': [9.130606807519459e-23, 4.872276949885089e-23, 0.002939987927675247, 0.957423210144043, 0.0, 7.103511528994133e-11, 6.0420668887672946e-05, 0.039576299488544464, 3.989315388447379e-12, 8.409963248741903e-32]}]}
  • In the second case, I get 1 dictionary (key: "predictions").
    {'predictions': [[9.13060681e-23, 4.87227695e-23, 0.00293998793, 0.95742321, 0.0, 7.10351153e-11, 6.04206689e-05, 0.0395763, 3.98931539e-12, 8.40996325e-32]]}

This is weird because I am using the exact same model from GCS. The only difference between those 2 models is that the second one has a region endpoint in Europe and they don't run on same machine type (but I don't think there is a link with my issue).

EDIT : Here is my request method. I used regional_endpoint = None in case 1 and regional_endpoint = "europe-west1" in case 2

project_id = "my_project_id"
model_id = "my_model_id"
version_id = None # if None, default version is used
regional_endpoint = None # "europe-west1"

def predict(project, model, instances, version=None, regional_endpoint=None):
'''
Send JSON data to a deployed model for prediction.
Args:
    - project (str): Project ID where the AI Platform model is deployed
    - model (str): Model ID
    - instances (tensor): model's expected inputs
    - version (str): Optional. Version ID
    - regional_endpoint (str): Optional. See https://cloud.google.com/dataflow/docs/concepts/regional-endpoints
Returns:
    - dictionary of prediction results

'''
input_data_json = {"signature_name": "serving_default", "instances": instances.tolist()}

model_path = "projects/{}/models/{}".format(project_id, model_id)
if version is not None:
    model_path += "/versions/{}".format(version)

if regional_endpoint is not None:
    endpoint = 'https://{}-ml.googleapis.com'.format(regional_endpoint)
    regional_endpoint = ClientOptions(api_endpoint=endpoint)

ml_ressource = googleapiclient.discovery.build("ml", "v1", client_options=regional_endpoint).projects()

request = ml_ressource.predict(name=model_path, body=input_data_json)
response = request.execute()

if "error" in response:
    raise RuntimeError(response["error"])

return response["predictions"]

I get the same result using gcloud command:

$ gcloud ai-platform predict --model=my_model_id --json-request=data.json --region=europe-west1
Using endpoint [https://europe-west1-ml.googleapis.com/]
[[5.64439188e-06, 1.11136234e-09, 4.66703168e-05, 1.34729596e-08, 2.34136132e-05, 1.52856941e-07, 0.999924064, 3.328397e-10, 3.32789263e-08, 3.37864092e-09]]

$ gcloud ai-platform predict --model=my_model_id --json-request=data.json
Using endpoint [https://ml.googleapis.com/]
DENSE_1
[5.644391876558075e-06, 1.1113623354930269e-09, 4.6670316805830225e-05, 1.3472959636828818e-08, 2.341361323487945e-05, 1.528569413267178e-07, 0.9999240636825562, 3.328397002455574e-10, 3.327892628135487e-08, 3.378640922591103e-09]
1
pythontensorflowgoogle-cloud-platformgoogle-ai-platform
Can you share the request that you perform? – guillaume blaquiere
Could you share both the exact endpoints you use for each model in order to try to reproduce? Do you get the same difference in results when using the gcloud command? – Ksign
I do get the same result using cloud command (see edited post) – mlm

1 Answers

1
votes

I have reproduced the same behavior.
From the list of endpoints, I have already tested the following:

  • europe-west1
  • asia-east1
  • us-east1
  • australia-southeast1

And neither of them returns the output layer’s name like the global endpoint does.

I have already informed the AI Platform product team of this behavior and created a public issue on issuetracker to track their progress.
Therefore, I suggest that all future communication about it should be done on issuetracker.