Dialogflow Variations of phrases

Question

I am currently working on an app in Dialogflow which has some chat bot like aspects. I've trained the intents with some iterations of similar phrases. I expected the ML algorithms in Dialogflow to catch variations of sentences that were very similar so I omitted them from the training phrases.

I noticed that in some cases where sentences are almost identical, Dialogflow is not able to match them, e.g.

"Do you have any night sweats?" "Do you get any night sweats?"

AND

"Good morning" "Good morning John"

For the second instance, I experimented with "Good morning sir" and "Good morning miss" and that seemed to match to the right intent. However, processing names is going to be a requirement for the app and I don't want to have to go through every intent and add cases where a name entity is involved unless it is absolutely necessary.

Is there something that I am doing incorrectly possibly that is limiting the Machine Learning matching in my agent? Or is this simply a limitation of Dialogflow where I'm going to have to manually add these iterations in?

Answer 1

The underlying algorithm behind Dialogflow is not open sourced. So, It is really hard to say what is the best way to fix the problem. But here are some options you can consider:

• To understand meaning of the sentence with different word or phrase (your first example) is open area of research. No ML algorithm can able to solve this problem completely (until now). You can not trust on Dialogflow or other Chatbot to capture/understand paraphrase sentences for you.

• In your first example, some words are very common in most of the sentences [Do, you, get, any]. In this case, Its really hard for the algorithm to find the right intent based on these common keywords. Try to add different sentences with [night, sweats] to give the algorithm the chance to capture them.

• Classification threshold: As mentioned in documentation, you can change accuracy of the classifier. In your second example, "Good morning" & "Good morning John" this may fix the problem.

• There is another option in that link, Rule based and ML based approaches. Choose the appropriate one based on your training size.