How to Train luis which have very closely related utterence which should be trained in 2 different intents

2
votes

I have very large data that I trained in LUIS. Most of the intents have utterances that are very very closely related to each other and hence LUIS is unable to point out to correct intent after training.

For example, take two intent1 and intent2

Example 1:

intent1: Credit Card transactions are not showing up in my TE

intent2: Credit card is not showing in HR mini ( This is incorrectly predicted, directing to intent1)

Example 2:

intent1: My email is locked out

intent2: Locked out ( This is incorrectly predicted, directing to intent1)

intent1 is trained with more number of utterance approx 50 which have words like credit, transaction, email but intent2 is not trained with many.

So after training when I am asking intent 2 queries, Luis is predicting incorrectly and taking me to intent 1.

Can someone help me out in knowing like how we can carefully train the model with these kinds of utterances

2
azure-language-understandingtraining-data

2 Answers

3
votes

Given that you have a large amount of data, I would suggest you follow the best practices for building LUIS apps.

  • For example, ensure the vocabulary for each intent is just for that intent and not overlapping with a different intent.

    Book a flight and Book a hotel use the same vocabulary of book a. This format is the same so it should be the same intent with the different words of flight and hotel as extracted entities.

  • The incorrect prediction results show intents that have utterances, which are used as examples for a specific intent, but are predicted for different intents. What you can do is edit the utterance so that it is more specific to the intent and train the app or combine intents if utterances are too closely aligned and train again.

Hope this helps!!

0
votes

Typical problem. Fine tune your training data by doing cross validation so you can have a systemic result. F1 score is not always helpful especially with large model. You can use QBox tool as well to analyse the influence of your training data and words. (Note I work there)

it is still a very empirical process you need to be able to test it, make small change, test it again and compare the result