Open source library for parsing text for intent and entities like Alexa Custom Skills

Question

Are there any open source libraries that work like Amazon Alexa Custom Skills where you provide it with an intent schema and sample utterances to match and it will provide a parsed tokenized response with entities that are matched in the definition.

Here's an example Alexa Custom Skill Intent schema.

Sample utterances to train and specify how to match text and map to entities:

AnswerIntent the answer is {Answer} 
AnswerIntent my answer is {Answer} 
AnswerIntent is it {Answer} 
AnswerIntent {Answer} is my answer AnswerOnlyIntent {Answer}

AMAZON.StartOverIntent start game 
AMAZON.StartOverIntent new game 
AMAZON.StartOverIntent start 
AMAZON.StartOverIntent start new game

Another service is https://wit.ai/ which allows you to configure expressions and tokens to match on, is there any open source library that provides this level of flexibility.

Answer 1

Mycroft AI seems to have a nice stack of projects that offer functionality and programmer interface very similar to Amazon Alexa Custom Skills, and you can host and modify it yourself to get more flexibility than Alexa Voice Service (but that's also a bit of a downside because you have to scale it yourself).

• https://docs.mycroft.ai/skill.creation

• https://mycroft.ai/projects/

  • Mycroft Core - is the technology that ties natural language processing, text-to-speech, speech-to-text, and powerful APIs together to create a powerful experience allowing users to manipulate their smart devices and the Internet of Things through voice control.
  • Open STT - open source speech-to-text model (seems like they're leverage other API's for this right now like Google Speech To Text, Wit.ai, and IBM Watson)
  • Adapt Intent Parser - converting natural language into machine readable data structures
  • Mimic Text To Speech - text and reads it aloud in high quality voice (just a proposal project currently, not available yet)

Answer 2

Maybe, you can try Rasa-nlu. It is quite similar to MS Luis. You can parse the text to structured data like

{
"entities": [
    {
        "endIndex": null,
        "entity": "hello",
        "score": null,
        "startIndex": null,
        "type": "name"
    }
],
"intents": [
    {
        "intent": "greet",
        "score": 0.640747175086514
    },
    {
        "intent": "goodbye",
        "score": 0.2696910959582717
    },
    {
        "intent": "FindEmployeeLocation",
        "score": 0.05672220244026073
    },
    {
        "intent": "FindEmployee",
        "score": 0.032839526514953594
    }
],
"query": "hello",
"topScoringIntent": {
    "intent": "greet",
    "score": 0.640747175086514
}

You can train your language model with json or markdown format as well. The most important thing is that service is open source. It means you don't have to pay extra money for using it. You just set up your own nlu server then use it. https://github.com/RasaHQ/rasa_nlu

Answer 3

If I got you right, you want to provide a pattern to match with possible values and the library has to generate possible utterances list. If so, there is a alexa-app project which lets you do this aside from other features. It's under MIT, so its possible to borrow needed code from there. Example:

app.intent('sampleIntent',
    {
        "slots":{"NAME":"LITERAL","AGE":"NUMBER"}, 
        "utterances":[ "my {name is|name's} {names|NAME} and {I am|I'm} {1-100|AGE}{ years old|}" ]
    },
    function(request,response) { ... }
);

Possible utterances:

my name is John and I am 20 years old
my name's John and I'm 40
my name's Taylor and I'm 55 years old
....

Answer 4

There are many OSS parsing libraries. Most actually have far more flexibility than Alexa's utterance patterns which are just regular expressions.

You can choose from libraries that target NLP specifically like GATE, Stanford Core NLP, OpenNLP, and NLTK. If you're working with large document collections then Apache Lucene (or Solr if you prefer) is dandy (although GATE supports them too).

For something lighter weight you can use general purpose parser generator. There are too many to list (https://en.wikipedia.org/wiki/Comparison_of_parser_generators) but packrat parsers (http://bford.info/packrat/) like Antlr are performant and easy to use.