Integration of dialogflow and google actions

Question

I am trying to integrate with google assistant/google home using Dialogflow and we are hitting some issues.

The issue is the following.

1. I have a google actions project that I have created which is linked to a corresponding dialogflow project. I have enabled the v2 APIs on both the dialogflow project and the integration of the dialogflow project and the google actions project.
2. The dialogflow project has intents that call a webhook for fulfillment. We have a service setup that responds to the webhook APIs. This service returns responses in V2 format including rich messages (like card, quick_replies, and carousel_select) under the key fulfillmentMessages:

However it appears that when dialogflow forwards this information to the google actions, it is not passing any of this information. On the other hand if I include a key called fulfillMentText in my service response, then dialogflow will forward that information to google actions as

[{"simpleResponse":{"textToSpeech":"Nice to meet you, Bob!"}}]}

It is not clear to me based on reading available docs, what I need to do so that dialogflow will also propagate the contents of fulfillmentMessages to google actions.

Thank you in advance.

Answer 1

The message objects for Actions on Google are different than the generic ones available for the other platforms Dialogflow supports. You need to also send the Actions on Google compatible messages in order to have them visible for the Assistant.

If you're using the dialogflow-fulfillment library, you can import the actions-on-google objects and add them to to the response and the library will handle them.

Don't worry about mixing the rich media types - only those that are appropriate for each platform will show.