BotFramework-WebChat - Adaptive Card

Question

Is there a way to add Onchange event to the adaptive card input field that is rendered in webchat (version V4). Example changing a quantity value (Adaptive card inputfield of type number) in the checkout screen should update the Total value (Adaptive card text field)

To keep it simple....In the below image once i change the number in the input box it should update in the below text box. everything should happen on the webchat V4(React) client side

Below are the options i tried, don't have any code to submit here:

option1: Tried to add an event to quantity input field in the card coming from the bot using middleware but not able to find an option to uniquely identify the input field to add the event (can see multiple input fields based on no of items in the card)

option2: create a new card in the frontend based on the card coming from bot and add events to that new card. Is it possible to interrupt the message going to bot and send a card from the frontend ?

option3: add an update button to the card so that the total is calculated in the backend and a update card is submitted to the user

below is the payload:

{
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.0",
    "type": "AdaptiveCard",
    "body": [
        {
            "type": "TextBlock",
            "text": "Output",
            "weight": "Bolder",
            "horizontalAlignment": "Center",
            "size": "Large",
            "id": "output",
            "color": "Good"
        },
        {
            "type": "Container",
            "items": [
                {
                    "$data": "{items}",
                    "type": "Container",
                    "items": [
                        {
                            "type": "TextBlock",
                            "text": " ",
                            "id": "line",
                            "spacing": "None"
                        },
                        {
                            "type": "Image",
                            "altText": "",
                            "id": "myimage",
                            "url": "{imgUrl}",
                            "spacing": "None",
                            "size": "Stretch",
                            "width": "1000px",
                            "height": "100px"
                        },
                        {
                            "type": "ColumnSet",
                            "id": "imgset",
                            "columns": [
                                {
                                    "type": "Column",
                                    "width": 50,
                                    "id": "desc",
                                    "items": [
                                        {
                                            "type": "TextBlock",
                                            "text": "{description}",
                                            "weight": "Bolder",
                                            "spacing": "None",
                                            "id": "desc",
                                            "wrap": true,
                                            "maxLines": 4
                                        }
                                    ],
                                    "spacing": "None"
                                }
                            ],
                            "spacing": "None"
                        },
                        {
                            "type": "ColumnSet",
                            "spacing": "None",
                            "columns": [
                                {
                                    "type": "Column",
                                    "width": 50,
                                    "id": "qty",
                                    "items": [
                                        {
                                            "type": "Input.Number",
                                            "placeholder": "Quantity",
                                            "id": "myquantity",
                                            "min": 0,
                                            "max": 100,
                                            "value": "{quantity}",
                                            "spacing": "None"
                                        }
                                    ],
                                    "horizontalAlignment": "Left",
                                    "verticalContentAlignment": "Center",
                                    "spacing": "None"
                                },
                                {
                                    "type": "Column",
                                    "id": "pricec",
                                    "items": [
                                        {
                                            "type": "TextBlock",
                                            "text": "{price}",
                                            "id": "pricet",
                                            "horizontalAlignment": "Right",
                                            "spacing": "None"
                                        }
                                    ],
                                    "verticalContentAlignment": "Center",
                                    "horizontalAlignment": "Right",
                                    "width": 50,
                                    "spacing": "None"
                                }
                            ],
                            "id": "qtypset"
                        },
                        {
                            "type": "ColumnSet",
                            "spacing": "None",
                            "columns": [
                                {
                                    "type": "Column",
                                    "width": 1,
                                    "items": [
                                        {
                                            "type": "TextBlock",
                                            "text": "Sub Total",
                                            "size": "Medium",
                                            "id": "subtotal00",
                                            "weight": "Bolder",
                                            "spacing": "None"
                                        }
                                    ],
                                    "id": "subtotal1",
                                    "spacing": "None"
                                },
                                {
                                    "type": "Column",
                                    "width": 1,
                                    "items": [
                                        {
                                            "type": "TextBlock",
                                            "horizontalAlignment": "Right",
                                            "text": "{subtotal}",
                                            "size": "Medium",
                                            "weight": "Bolder",
                                            "id": "subtotalt0",
                                            "color": "Accent",
                                            "spacing": "None"
                                        }
                                    ],
                                    "id": "subtotal200",
                                    "spacing": "None"
                                }
                            ],
                            "id": "colsetsubtot00"
                        }
                    ],
                    "id": "itemcontainer",
                    "style": "emphasis",
                    "spacing": "None"
                }
            ],
            "id": "rootcontainer",
            "style": "accent"
        },
        {
            "type": "ColumnSet",
            "id": "totalset",
            "columns": [
                {
                    "type": "Column",
                    "width": 50,
                    "id": "totalcolumn",
                    "items": [
                        {
                            "type": "TextBlock",
                            "text": "Total",
                            "size": "Medium",
                            "isSubtle": true,
                            "weight": "Bolder",
                            "id": "total",
                            "color": "Dark"
                        }
                    ]
                },
                {
                    "type": "Column",
                    "width": 50,
                    "items": [
                        {
                            "type": "TextBlock",
                            "text": "{total}",
                            "size": "Medium",
                            "id": "totaltext",
                            "horizontalAlignment": "Right",
                            "weight": "Bolder",
                            "color": "Accent"
                        }
                    ],
                    "id": "totalcol2"
                }
            ]
        }
    ],
    "id": "final"
}

I am using the below example as a starting point https://github.com/microsoft/BotFramework-WebChat/tree/master/samples/04.api/e.piping-to-redux

webchat.js:

import React from 'react';

import ReactWebChat, { createDirectLine, createStore } from 'botframework-webchat';
import directLineDisconnect from 'botframework-webchat-core/lib/actions/disconnect';
import dispatchIncomingActivityMiddleware from './dispatchIncomingActivityMiddleware';
import uuid from 'uuid';

export default class extends React.Component {
  constructor(props) {
    super(props);

    this.store = createStore({}, dispatchIncomingActivityMiddleware(props.appDispatch, this));
    this.activityMiddleware = this.setActivityMiddleware();
    this.attachmentMiddleware = this.setAttachmentMiddleware();

    this.state = {};

  }

  componentDidMount() {
    this.fetchToken();
    this.setSendBox();
  }

  componentWillUnmount(){

  }

  async fetchToken() {
    const myHeaders = new Headers();
    const userDetails = uuid.v4();
    myHeaders.append('Authorization', 'Bearer ' + 'mytoken'); 
    myHeaders.append('Content-type', 'application/json');
    const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate', { 
                        body: JSON.stringify({ user: { id: userDetails, name: userDetails }}),
                        method: 'POST', headers: myHeaders });
    const { token } = await res.json();
    console.log("My Token: " + token);
    this.setState(() => ({
      directLine: createDirectLine({ token })
    }));
  }

  setActivityMiddleware(){
    return () => next => card => {
      return children => (
        <div
          className={card.activity.attachments && (card.activity.attachments[0].content.id === "output") ? card.activity.attachments && card.activity.attachments[0].content.id : ''}
        >
          {next(card)(children)}
        </div>
      );
    };

  }


  setAttachmentMiddleware(){
    return () => next => ({ card, activity, attachment: baseAttachment }) => {
      let attachment = baseAttachment;
      if (baseAttachment.content.body){
      switch (baseAttachment.content.body[0].id) {
        case 'review':                   
         for (let i = 0; i < attachment.content.body[1].items.length; i++) {
         attachment.content.body[1].items[i].items[3].columns[0].items[0].value = baseAttachment.content.body[1].items[i].items[3].columns[0].items[0].value.toString();
                                                                           } //for loop
         break;

         default:
           break;
        }
    }
    return next({ card, activity, attachment });
    };

  }

  setSendBox() {

    this.store.dispatch({
      type: 'WEB_CHAT/SET_SEND_BOX',
      payload: { text: 'sample:redux-middleware' }
    });
/*

    this.store.dispatch({
      type: 'WEB_CHAT/SEND_EVENT',
      payload: { name: 'membersAdded',
                 value: { language: window.navigator.language }
               }  
    }); */
  }


  render() {
    return this.state.directLine ? (
      <ReactWebChat
        activityMiddleware={this.activityMiddleware}
        attachmentMiddleware={this.attachmentMiddleware}
        directLine={this.state.directLine}
        store={this.store}
        styleOptions={{
          backgroundColor: 'Transparent',
          hideUploadButton: true
        }}
      />
    ) : (
      <div>Connecting to bot&hellip;</div>
    );
  }

}

dispatchIncomingActivityMiddleware.js:

export default function(dispatch, thisvariable) {
    return () => next => action => {
      if (action.type === 'DIRECT_LINE/INCOMING_ACTIVITY') {
        const { activity } = action.payload;


        if (activity.from.role === 'bot'){
        var inputBox=document.getElementsByClassName("css-eycyw2");
        if (inputBox.length > 0){
          inputBox[inputBox.length - 1].style.display='block';
        }
                                          }

      }


      if ((action.type === 'WEB_CHAT/SEND_POST_BACK') || (action.type === 'WEB_CHAT/SEND_MESSAGE')) { 
        var inputBox=document.getElementsByClassName("css-eycyw2");
        if (inputBox.length > 0){
          inputBox[inputBox.length - 1].style.display='none';
          dispatch(setInputVisibility(true));
        }
      }

      return next(action);
    };
  }

Answer 1

The first thing to understand is that Web Chat uses the Adaptive Cards JavaScript SDK, available as an npm package. Web Chat mostly uses the out-of-the-box rendering functionality of the SDK, but one important thing it changes is how actions are handled. Without providing a customized handler, submit actions wouldn't be sent to the bot.

adaptiveCard.onExecuteAction = handleExecuteAction;

This is how applications are supposed to use Adaptive Cards. While most of the functionality is handled on the SDK side, there are a few things the application needs to do to make Adaptive Cards work for that specific app. While you can see Web Chat assigning a function to the onExecuteAction "event" property of a specific Adaptive Card instance, there is also a static counterpart of onExecuteAction that could be accessed like this:

AdaptiveCard.onExecuteAction = handleExecuteAction;

Using the static event will apply a handler for all Adaptive Cards instead of just one, but it will be overridden by any handlers applied to specific instances. The reason I'm telling you this is because there are many more static events, and there are a few in particular that will be useful for your situation:

static onAnchorClicked: (element: CardElement, anchor: HTMLAnchorElement) => boolean = null;
static onExecuteAction: (action: Action) => void = null;
static onElementVisibilityChanged: (element: CardElement) => void = null;
static onImageLoaded: (image: Image) => void = null;
static onInlineCardExpanded: (action: ShowCardAction, isExpanded: boolean) => void = null;
static onInputValueChanged: (input: Input) => void = null;
static onParseElement: (element: CardElement, json: any, errors?: Array<HostConfig.IValidationError>) => void = null;
static onParseAction: (element: Action, json: any, errors?: Array<HostConfig.IValidationError>) => void = null;
static onParseError: (error: HostConfig.IValidationError) => void = null;
static onProcessMarkdown: (text: string, result: IMarkdownProcessingResult) => void = null;

It would be possible for you to come up with a solution that uses the onInputValueChanged event, which fires every time any input in the card is changed. Your handler could search the card for other elements that it needs to use as operands for its calculation, and it would also need to search the card for the element that would display the result. Rather than doing all that work every time a character is typed, I prefer a solution that searches the card just once at the beginning for the elements it will use in its calculation. An alternative to listening to events on the Adaptive Card class or an Adaptive Card instance is to listen to events on particular elements, like inputs. So my example will use the static onParseElement event to get the elements it needs and then use the onValueChanged event for specific input instances it finds.

Before writing the code for the handler, we need to come up with a way for the code to know which elements to use for the operands and result of the calculation. For example, you could just have the code combine every input in the card (or in a container) and put the result in the last text block found. For my example, I've come up with a naming schema that the code can use. There are two keywords, "total" and "price", and the code looks for them in each element ID. I want to make it clear that this schema is totally arbitrary and that you can do something different if you want. Here's my example card:

{
  "type": "AdaptiveCard",
  "version": "1.0",
  "body": [
    {
      "type": "TextBlock",
      "text": "$10.00",
      "id": "foo_a_price"
    },
    {
      "type": "Input.Text",
      "id": "foo_a"
    },
    {
      "type": "TextBlock",
      "text": "$2.00",
      "id": "foo_b_price"
    },
    {
      "type": "Input.Text",
      "id": "foo_b"
    },
    {
      "type": "TextBlock",
      "text": "total",
      "id": "total_foo"
    }
  ],
  "actions": [
    {
      "type": "Action.Submit",
      "title": "Submit"
    }
  ]
}

You might be able to guess from looking at this that the idea is for one text block to have an ID that starts with "total_" and has some identifier after it. The quantities you want to add up start with that same identifier, and the price you want to multiply with each quantity has the same ID as the quantity but with the suffix "_price". I recommend using number inputs instead of text inputs, but this example shows that text still works. And here is the code for my example app that reads the schema:

import * as adaptiveCardsPackage from 'adaptivecards';

adaptiveCardsPackage.AdaptiveCard.onParseElement = element => {
  const PREFIX_TOTAL = 'total_';
  const SUFFIX_PRICE = '_price';

  if (element.id && element.id.startsWith(PREFIX_TOTAL)) {
    const itemPrefix = element.id.slice(PREFIX_TOTAL.length);
    const card = element.getRootElement();
    const inputs = card.getAllInputs().filter(input => input.id.startsWith(itemPrefix));
    const products = {};

    for (const input of inputs) {
      const priceElement = card.getElementById(input.id + SUFFIX_PRICE);
      const price = Number(priceElement.text.replace(/[^0-9.-]+/g, '')) || 0;

      // `sender` will be the same as `input`.
      // You could capture the input const instead of using the argument,
      // but I'm demonstrating that you don't need to.
      input.onValueChanged = sender => {
        const quantity = Number(sender.value) || 0;

        products[sender.id] = price * quantity;

        const sum = Object.values(products).reduce((a, b) => a + b);

        element.setText("$" + sum.toFixed(2));
        element.renderedElement.replaceWith(element.render());
      };
    }
  }
};

I have reason to believe that this change to the AdaptiveCard class will automatically be applied to the AdaptiveCard class in the package that Web Chat imports, since it's the same class in the same package. However, Web Chat now allows you to provide your own Adaptive Cards package as a property, so you can make sure Web Chat is using the package with your special event handler:

<ReactWebChat
  directLine={createDirectLine({secretOrToken})}
  adaptiveCardsPackage={adaptiveCardsPackage}
/>