Batch Testing in LUIS incorrect prediction for List enities

Question

I am trying to run batch tests for my LUIS app. The entities labelled as List Entity are displayed as False Positive inside the chart.

I have already gone through the below question and I believe this issue has already been resolved. But I still have issues with the list entity type. I have not included list entities inside the batch file.

LUIS Batch Testing for Entity is Not Working

In the documentation, it is stated that: "Only the machine-learned entities are used: simple, hierarchical (parent-only), and composite entities. Do not add non-machine-learned entities because they are always found either through regular expressions, or explicit text matches"

I have used the following syntax inside batch test file (no labelled entites):

         [
          {
            "text": "example utterance goes here",
            "intent": "intent name goes here",
            "entities": []
          }
         ]

Do I have to exclude all the utterances containing the list entity while creating the batch file ? If so how can I test the Intent prediction for those utterances using batch tests?

mdrichardson mdrichardson · Accepted Answer · 2020-02-14T17:28:50

Do I have to exclude all the utterances containing the list entity while creating the batch file ?

Correct.

If so how can I test the Intent prediction for those utterances using batch tests?

You'll need to test them manually.

Per the docs,

Entity types allowed: only machined-learned entities of simple and composite. Batch testing is only useful for machined-learned intents and entities.

The reason for this is that batch testing is meant to test scenarios where you've made changes to your app and you want to ensure it still works. Detection of non-machine-learned entities is never going to change, so they don't really need to be batch-tested (other than maybe the first time).

That being said, I understand you'll likely still want the ability to do this. I recommend opening a feature request on the LUIS UserVoice

Batch Testing in LUIS incorrect prediction for List enities

1 Answers