LUIS - understand any person name

Question

we are building a product on LUIS / Microsoft Bot framework and one of the doubt we have is Person Name understanding. The product is set to use by anyone by just signing up to our website. Which means any company who is signing up can have any number of employees with any name obviously.

What we understood is the user entity is not able to recognize all names. We have created a phrase list but as per we know there is a limit to phrase list (10K or even if its 100K) and names in the world can never have a limit. The other way we are thinking is to not train the entity with utterances. However if we have 100s of customers with 1000s of users each, the utterances will not be a good idea in that case.

I do not see any other way of handling this situation. Probably I am missing something here? Anyone faced similar problem and how it is handled?

The worst case would be to create a separate LUIS instance for each customer but that's really a big task to do only because we cant handle names.

Which components you use apart from LUIS? Directline, Webchat or other? Or is it LUIS only directly from your web? — rudolf_franek
Not sure what you mean by component to be honest however the channel is webchat, skype, MS teams as off now and can grow later. And the dev is done in c# + MS Bot Framework Web API layer (c#) which gets direct LUIS Intend and reply accordingly. Does that answer your question? — Nitin Khubani
Considering input parameter "[FromBody]Activity activity" of POST method of your controller exposed to bot-framework you may use "activity.From.Name" for webchat, skype and teams, do you agree? And then you are free handle the names in your custom C# code outside of LUIS. — rudolf_franek

Joe Mayo Joe Mayo · Accepted Answer · 2018-01-10T21:10:43

As you might already know, a person's name could literally be anything: e.g. an animal, car, month, or color. So, there isn't any definitive way to identify something as a name. The closest you can come is via text analysis parts of speech and either taking a guess or comparing to an existing list. LUIS or any other NLP tool is unlikely to help with this. Here's one approach that might work out better. Try something like Microsoft's Text Analytics cognitive service, with a POST to the Key Phrases endpoint, like this:

https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/keyPhrases

and the body:

{
  "documents": [
    {
      "language": "en-us",
      "id": "myid",
      "text": "Please book a flight for John Smith at 2:30pm on Wednesday."
    }
  ]
}

That returns:

{
  "languageDetection": {
    "documents": [
      {
        "id": "e4263091-2d54-4ab7-b660-d2b393c4a889",
        "detectedLanguages": [
          {
            "name": "English",
            "iso6391Name": "en",
            "score": 1.0
          }
        ]
      }
    ],
    "errors": []
  },
  "keyPhrases": {
    "documents": [
      {
        "id": "e4263091-2d54-4ab7-b660-d2b393c4a889",
        "keyPhrases": [
          "John Smith",
          "flight"
        ]
      }
    ],
    "errors": []
  },
  "sentiment": {
    "documents": [
      {
        "id": "e4263091-2d54-4ab7-b660-d2b393c4a889",
        "score": 0.5
      }
    ],
    "errors": []
  }
}

Notice that you get "John Smith" and "flight" back as key phrases. "flight" is definitely not a name, but "John Smith" might be, giving you a better idea of what the name is. Additionally, if you have a database of customer names, you can compare the value to a customer name, either exact or soundex, to increase your confidence in the name.

Sometimes, the services don't give you an 100% answer and you have to be creative with work-arounds. Please see the Text Analytics API docs for more info.

LUIS - understand any person name

2 Answers