QNA Maker not returning exact matches

Question

QNA Maker is not returning the Exact matches from the knowledge bases.

So we've been using the botbuilder-ai library to call the QNA maker. However QNA Maker is not returning the exact match which is there in the knowledgebase. However if I check that in the qnamaker.ai portal's test feature it works.

Here is the code that I'm using to extract the results.

const this.qnaRecognizer = new QnAMaker(
                {
                    knowledgeBaseId: keyvault.QnAMakerSecret.knowledgebaseID,
                    endpointKey: keyvault.QnAMakerSecret.qnaAuthKEY,
                    host: keyvault.QnAMakerSecret.qnaEndPointHost,
                },
                {
                    scoreThreshold: QNA_CONFIDENCE_THRESHOLD,
                    top: QNA_NUM_OF_RESULTS,
                }
            );
const qnaResult = await this.qnaRecognizer.getAnswers(step.context);

As expected, it would call the QNA Maker's endpoint and should return results. Instead its returning a blank Array.

Now, I monitored the qnamaker.ai calls and noticed that when they are calling the APIs, they are passing one more parameter which is isTest = true.

Here are the results:

Without isTest = true

URL: https://qnamaker-host.azurewebsites.net/qnamaker/knowledgebases/<kbid>/generateAnswer
Method: POST
Result: {
    "answers": [
        {
            "questions": [],
            "answer": "No good match found in KB.",
            "score": 0,
            "id": -1,
            "source": null,
            "metadata": []
        }
    ],
    "debugInfo": null
}

With isTest = true

URL: https://qnamaker-host.azurewebsites.net/qnamaker/knowledgebases/<kbid>/generateAnswer
Method: POST
Result: {
    "answers": [
        {
            "questions": [
                "Who are you?"
            ],
            "answer": "I am an intelligent bot",
            "score": 100,
            "id": 2,
            "source": "Editorial",
            "metadata": [
                {
                    "name": "_id",
                    "value": "<removed>"
                }
            ],
            "context": {
                "isContextOnly": false,
                "prompts": []
            }
        }
    ],
    "debugInfo": null
}

Now I should be expecting same behaviour without setting isTest = true in this case. Also, in each APIs I cannot pass isTest = true because I'm directly using their library to do this.

Can someone please help in this? Thanks in advance.

Tony Ju Tony Ju · Accepted Answer · 2019-05-17T01:29:16

The most likely reason is that you didn't publish the knowledge base.

With "isTest": true, you are requesting the test knowledge base instead of the published knowledge base. Refer to this document for more details.

Update:

There is a design limitation with QnA Maker when multiple KBs exist in a resource, the test environment is impacted by other KBs. The production environment is isolated from other KBs. This is due to the Azure Search indexes being shared among KBs in the test environment. One work-around is to only have one KB in the resource when using the test environment.

QNA Maker not returning exact matches

1 Answers