Dialogflow change surface from webhook

Question

I am trying to send a response from my webhook to trigger the process to change to another surface, but Actions on Google always throws:

MalformedResponse 'final_response' must be set.

And that it's not very helpful.

This is the JSON I am returning:

{
    "payload": {
        "google": {
            "expectUserResponse": true,
            "conversationToken": "{\"data\":{}}",
            "userStorage": "{\"data\":{}}",
            "expectedInputs": [
                {
                    "inputPrompt": {
                        "richInitialPrompt": {
                            "items": [
                                {
                                    "simpleResponse": {
                                        "textToSpeech": "PLACEHOLDER"
                                    }
                                }
                            ]
                        }
                    },
                    "possibleIntents": [
                        {
                            "intent": "actions.intent.NEW_SURFACE",
                            "inputValueData": {
                                "@type": "type.googleapis.com/google.actions.v2.NewSurfaceValueSpec",
                                "capabilities": [
                                    {
                                        "name": "actions.capability.SCREEN_OUTPUT"
                                    }
                                ],
                                "context": "Sure, I have some sample images for you.",
                                "notificationTitle": "Sample Images"
                            }
                        }
                    ]
                }
            ]
        }
    }
}

On Dialogflow I have setup 2 intents; one intent that returns the json specified here, and another intent with the event actions_intent_NEW_SURFACE, so I know what the user responded to the question to change surface.

I have been reading these sites:

https://developers.google.com/actions/assistant/surface-capabilities#multi-surface_conversations

https://dialogflow.com/docs/reference/api-v2/rest/Shared.Types/WebhookResponse

https://developers.google.com/actions/build/json/dialogflow-webhook-json#dialogflow-response-body

Action On Google, webhook response with actions.intent.NEW_SURFACE (This seems like the same problem has I have, but the OP didn't write the response.)

But none of them throw me some light on this matter.

Suggestion: try and use standalone AoG NodeJS client - github.com/actions-on-google/actions-on-google-nodejs OR AoG client with Dialogflow Fulfillment Client - github.com/dialogflow/dialogflow-fulfillment-nodejs These will help you in coding and it is easy to use features like Surface Switching, Permissions, RichUI through these Clients. — Abhinav Tyagi
Thanks for the suggestion, and I am with you, using NodeJS is much easier than a webhook implementation, because all of the documentation is for Node. But the webhook decision comes from the necessity to use other tools from the framework I am using. — German Casares
Looks like the documentation in developers.google.com/actions/assistant/… is old and that doesn't work. I have tried to send the NEW_SURFACE intent as a systemIntent and that works. — German Casares

Prisoner Prisoner · Accepted Answer · 2018-09-18T17:35:01

It looks like you're trying to send a full Action SDK body as part of your Dialogflow response. While the payload.google portion of the JSON contains objects that are similar to the Action SDK response, there are some differences. You can see https://developers.google.com/actions/build/json/dialogflow-webhook-json#dialogflow-response-body for a full example of what the response should be when using a helper, but I think your response needs to be something like

{
    "payload": {
        "google": {
            "expectUserResponse": true,
            "userStorage": "{\"data\":{}}",
            "richResponse": {
                "items": [
                    {
                        "simpleResponse": {
                            "textToSpeech": "PLACEHOLDER"
                        }
                    }
                ]
            },
            "systemIntent": {
                "intent": "actions.intent.NEW_SURFACE",
                "inputValueData": {
                    "@type": "type.googleapis.com/google.actions.v2.NewSurfaceValueSpec",
                    "capabilities": [
                        "actions.capability.SCREEN_OUTPUT"
                    ],
                    "context": "Sure, I have some sample images for you.",
                    "notificationTitle": "Sample Images"
                }
            }
        }
    }
}

Dialogflow change surface from webhook

1 Answers