How to work with multiple intents on Microsoft's LUIS

Question

I'm working with LUIS and want to manage and deal not only with the top scoring intent but also with all others. In this specific situation occurs when someone inquires about two things in the same phrase.

For example: "I want to buy apples" ("Buy" intent) and "I want to sell bananas" ("Sell" intent) versus "I want to buy bananas and sell apples" ("buy" and "sell" intents on the same utterance).

The idea is to define a threshold that will accept as "valid" any intentions score above this confidence number.

During some tests I found out this can work if we have very few intents on the same utterance.

However if we increase the number of intents on the same utterance the results degrades very fast.

I included some examples to clarify what I mean: The output examples below were generated on a LUIS with 4 intents ("buy", "sell", "none" and "prank") and 1 entity ("fruit")

I want to buy apples ==>

{
  "query": "i want to buy apples",
  "topScoringIntent": {
    "intent": "Buy",
    "score": 0.999846
  },
  "intents": [
    {
      "intent": "Buy",
      "score": 0.999846
    },
    {
      "intent": "None",
      "score": 0.2572831
    },
    {
      "intent": "sell",
      "score": 2.32163586e-7
    },
    {
      "intent": "prank",
      "score": 2.32163146e-7
    }
  ],
  "entities": [
    {
      "entity": "apples",
      "type": "Fruit",
      "startIndex": 14,
      "endIndex": 19,
      "resolution": {
        "values": [
          "apple"
        ]
      }
    }
  ]
}

I want to sell bananas ==>

{
  "query": "i want to sell bananas",
  "topScoringIntent": {
    "intent": "sell",
    "score": 0.999886036
  },
  "intents": [
    {
      "intent": "sell",
      "score": 0.999886036
    },
    {
      "intent": "None",
      "score": 0.253938943
    },
    {
      "intent": "Buy",
      "score": 2.71893583e-7
    },
    {
      "intent": "prank",
      "score": 1.97906232e-7
    }
  ],
  "entities": [
    {
      "entity": "bananas",
      "type": "Fruit",
      "startIndex": 15,
      "endIndex": 21,
      "resolution": {
        "values": [
          "banana"
        ]
      }
    }
  ]
}

I want to eat a pizza ==>

{
  "query": "i want to eat a pizza",
  "topScoringIntent": {
    "intent": "prank",
    "score": 0.997353
  },
  "intents": [
    {
      "intent": "prank",
      "score": 0.997353
    },
    {
      "intent": "None",
      "score": 0.378299
    },
    {
      "intent": "sell",
      "score": 2.72957237e-7
    },
    {
      "intent": "Buy",
      "score": 1.54754474e-7
    }
  ],
  "entities": []
}

Now with two intents... The score of each one starts to reduce aggressively

I want to buy apples and sell bananas ==>

{
  "query": "i want to buy apples and sell bananas",
  "topScoringIntent": {
    "intent": "sell",
    "score": 0.4442593
  },
  "intents": [
    {
      "intent": "sell",
      "score": 0.4442593
    },
    {
      "intent": "Buy",
      "score": 0.263670564
    },
    {
      "intent": "None",
      "score": 0.161728472
    },
    {
      "intent": "prank",
      "score": 5.190861e-9
    }
  ],
  "entities": [
    {
      "entity": "apples",
      "type": "Fruit",
      "startIndex": 14,
      "endIndex": 19,
      "resolution": {
        "values": [
          "apple"
        ]
      }
    },
    {
      "entity": "bananas",
      "type": "Fruit",
      "startIndex": 30,
      "endIndex": 36,
      "resolution": {
        "values": [
          "banana"
        ]
      }
    }
  ]
}

and if we include the third intent, LUIS seems to collapse:

I want to buy apples, sell bananas and eat a pizza ==>

{
  "query": "i want to buy apples, sell bananas and eat a pizza",
  "topScoringIntent": {
    "intent": "None",
    "score": 0.139652014
  },
  "intents": [
    {
      "intent": "None",
      "score": 0.139652014
    },
    {
      "intent": "Buy",
      "score": 0.008631414
    },
    {
      "intent": "sell",
      "score": 0.005520768
    },
    {
      "intent": "prank",
      "score": 0.0000210663875
    }
  ],
  "entities": [
    {
      "entity": "apples",
      "type": "Fruit",
      "startIndex": 14,
      "endIndex": 19,
      "resolution": {
        "values": [
          "apple"
        ]
      }
    },
    {
      "entity": "bananas",
      "type": "Fruit",
      "startIndex": 27,
      "endIndex": 33,
      "resolution": {
        "values": [
          "banana"
        ]
      }
    }
  ]
}

Do you know/recommend any approach that I should use to train LUIS in order to mitigate this issue? Dealing with multiple intents in the same utterance is key to my case.

Thanks a lot for any help.

Each domain intent should have at least 10 utterances where each intent's utterance is different from the other intent utterances. The none intent should not have any domain-related intents and it should contain at least 10% of the total utterance count. — DFBerry
See my answer here about multiple intent in 1 LUIS query: stackoverflow.com/questions/48703996/… — Nicolas R

Adnan S Adnan S · Accepted Answer · 2018-03-02T18:39:29

You will likely need to do some pre-processing of the input using NLP to chunk the sentences and then train/submit the chunks one at a time. I doubt that LUIS is sophisticated enough to handle multiple intents in compound sentences.

Here's a sample code for preprocessing using Spacy in Python - have not tested this for more complicated sentences but this should work for your example sentence. You can use the segments below to feed to LUIS.

Multiple intents are not an easy problem to address and there may be other ways to handle them

import spacy
model = 'en'

nlp = spacy.load(model)
print("Loaded model '%s'" % model)

doc = nlp("i want to buy apples, sell bananas and eat a pizza ")

for word in doc:
    if word.dep_ in ('dobj'):
        subtree_span = doc[word.left_edge.i : word.right_edge.i + 1]
        print(subtree_span.root.head.text + ' ' + subtree_span.text)
        print(subtree_span.text, '|', subtree_span.root.head.text)
        print()

How to work with multiple intents on Microsoft's LUIS

2 Answers