Why is the carousel not showing in the console simulator?

Question

I am trying to figure out how I can embed Google Actions responses, such as the carousel, in a webhook response for DialogFlow.

I am using V2 of the REST protocol, so I am filling ACTIONS_ON_GOOGLE in the source field and the payload field contains the Google Actions field as specified (as per How can I integrate the Google Actions responses in a webhook response in Dialogflow?). I am sending the following response:

{
   "fulfillmentText":"This is a carousel.",
   "source":"ACTIONS_ON_GOOGLE",
   "payload":{
      "conversationToken":"",
      "expectUserResponse":true,
      "expectedInputs":[
         {
            "inputPrompt":{
               "initialPrompts":[
                  {
                     "textToSpeech":"This is a carousel."
                  }
               ],
               "noInputPrompts":[
               ]
            },
            "possibleIntents":[
               {
                  "intent":"actions.intent.OPTION",
                  "inputValueData":{,
                     "@type":"type.googleapis.com/google.actions.v2.OptionValueSpec"
                     "carouselSelect":{
                        "items":[
                           {
                              "optionInfo":{
                                 "key":"key1",
                                 "synonyms":[
                                    "Option 1"
                                 ]
                              },
                              "title":"Option 1",
                              "description":"Option 2"
                           },
                           {
                              "optionInfo":{
                                 "key":"key2",
                                 "synonyms":[
                                    "Option 2"
                                 ]
                              },
                              "title":"Option 2",
                              "description":"Option 2"
                           }
                        ]
                     }
                  }
               }
            ]
         }
      ]
   }
}

When trying this out in the console, no carousel is shown. Only the text This is a carousel. is displayed. For your information, I did not include the image field, as it is optional according to the specification, but even with images there is no carousel displayed.

It's hard to debug this, as my actions.intent.OPTION intent is not displayed in possibleIntents[] array in the response tab. I expected this actions.intent.OPTION intent to be merged with the other intents (such assistant.intent.action.TEXT) as generated by the Dialogflow response.

What am I doing wrong here? Am I maybe shooting myself in the foot by using V2 instead of V1 of the Dialogflow REST protocol?

update after initial feedback by Prisoner

I tried with the following response, but still not getting any carousel:

{
   "fulfillmentText":"Here you go.",
   "source":"ACTIONS_ON_GOOGLE",
   "payload":{
      "expectUserResponse":true,
      "richResponse":{
         "items":[
            {
               "simpleResponse":{
                  "textToSpeech":"Here are your results."
               }
            }
         ]
      },
      "systemIntent":{
         "intent":"actions.intent.OPTION",
         "data":{
            "carouselSelect":{
               "items":[
                  {
                     "optionInfo":{
                        "key":"Option1",
                        "synonyms":[
                           "Option2"
                        ]
                     },
                     "title":"Option3",
                     "description":"Option4"
                  },
                  {
                     "optionInfo":{
                        "key":"Option5",
                        "synonyms":[
                           "Option6"
                        ]
                     },
                     "title":"Option7",
                     "description":"Option8"
                  }
               ]
            },
            "@type":"type.googleapis.com/google.actions.v2.OptionValueSpec"
         }
      }
   }
}

I also tried to manually create an intent in Dialogflow with returns a 'hardcoded' carousel (that is, without fulfillment callback) and this carousel is shown perfectly. So I am sure that the console is correctly configured.

I am also comparing my response with the ones in Google Assistant flow with multiple actions_intent_OPTION handlers, but without success so far.

Prisoner Prisoner · Accepted Answer · 2018-03-06T12:42:35

You haven't shot yourself in the foot - but you have made something that was already a bit complex even more complex. There are two things to look out for:

CORRECTION: Make sure the payload is the same as what used to be data, but other fields have changed format.
You need to make sure the simulator is in Phone mode and not Speaker mode.

Details follow.

Documented Dialogflow Differences

The Actions on Google documentation for the Dialogflow response is a little confusing. Instead of providing the full example, it just says that the response that would be under expectedInputs.possibleIntents should, instead, be under data.google.systemIntent. For V2, that would be payload.google.systemIntent.

The inputPrompt object is also restructured somewhat so you should be sending a richResponse that contains a simpleResponse object.

UPDATE I've tested this. This is the entire JSON that should be returned. Note the contents of payload is exactly what the contents of data used to be. The source field is ignored and has nothing to do with the payload, apparently.

See also https://github.com/dialogflow/fulfillment-webhook-json which contain some examples.

{
    "payload": {
        "google": {
            "expectUserResponse": true,
            "richResponse": {
                "items": [
                    {
                        "simpleResponse": {
                            "textToSpeech": "This is a carousel"
                        }
                    }
                ]
            },
            "systemIntent": {
                "intent": "actions.intent.OPTION",
                "data": {
                    "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                    "carouselSelect": {
                        "items": [
                            {
                                "optionInfo": {
                                    "key": "key1",
                                    "synonyms": [
                                        "Option 1"
                                    ]
                                },
                                "title": "Option 1",
                                "description": "Option 2"
                            },
                            {
                                "optionInfo": {
                                    "key": "key2",
                                    "synonyms": [
                                        "Option 2"
                                    ]
                                },
                                "title": "Option 2",
                                "description": "Option 2"
                            }
                        ]
                    }
                }
            }
        }
    }
}

Simulator Surface Setting

Make sure that your simulator is set for the "Phone" surface rather than the "Speaker" surface. Options won't display on a Speaker.

The setting should look like this:

Not like this:

Why is the carousel not showing in the console simulator?

1 Answers