Assigning weights to intents

2
votes

I'm just getting started with Watson Conversation and I ran into the following issue.

I have a general intent for #greetings (hello, hi, greetings....) and another intent for #general-issues (I have a problem, I have an issue, stuff's not working, etc.....). If the user says: "hello, I have a problem with your product.", it matches the #greetings intent and replies back, but in this case I want to match the #general-issue intent.

As per: https://console.bluemix.net/docs/services/conversation/dialog-overview.html#dialog-overview

I would expect the nodes at the top of the list to be matched first, so I have the #greetings node at the bottom of the dialogue tree, to give a chance for a "higher weight" node to be matched first, but it doesn't seem to work every time.

Is duplicating the #greeting intents in #general-issue the only solution here?

2
ibm-cloudibm-watsonwatson-conversation
Have you tried retraining it to match it? There is the "improve dialog" link. – data_henrik
Retraining it would mean adding new intent permutations to the #general-issue to contain things like "Hello, I have a problem", "Hi, I have an issue" etc. That duplicates the #greeting intent, which is what you're not supposed to, according to the documentation anyway. – Moist_Gerbil
In the "improve" section you can tell WCS that the detected intent was wrong and what else it should have taken. By that you are retraining WCS and help the system learn ("cognitive"). – data_henrik

2 Answers

3
votes

So, trying to help you based on my experience, you can use the intents[0].confidence as your favor.

For example:

enter image description here

In my example, I create one condition with:

intents[0].confidence > 0.75

So, Watson will recognizes this intent just if the user types something very similar to their trained examples for the #greetings Intent.

As you can see, works very well:

enter image description here

2
votes

So here are two other approaches you can take.

Treat off topic as a contamination.

When building a conversational system, it's important to know what your end users are actually saying. So collect questions from real users.

You will find that not many people will say a greeting and a question. I personally haven't done the statistical chance over projects I've done, but at least anecdotal I have not seen it happen often.

Knowing this, you can try removing off topic / chit-chat from your intents. As it does not fully reflect the domains you want to train on.

To counter this, you can create a more detailed second workspace with off topic/chit-chat. If you do not get a good hit on the primary workspace, you can call out to the second one. You can improve this by adding chit-chat to counter examples in the primary workspace.

You can also mitigate this by simply wording your initial response to the user. For example, if your initial response is a hello, have the system also ask a question. Or have it progress the conversation where a hello becomes redundant.

Detect possible compounded intents.

At the moment, this is only easily possible at the application layer.

Setting alternate_intents to true will return the top 10 intents and their confidences.

Before going further, if the top intent < 0.2 then it needs more training (so no need to proceed).

If > 0.2 you can map these on a graph, you can visually see if the top two intents. For example:

graph image

To have your application see this, you can use the k-means algorithm to create two buckets (k=2). That would be relevant and irrelevant.

Once you see more then one that is relevant, you can take action to ignore the chit-chat/off-topic.

There is more details, and sample code here.