How to set confidence score settings/score threshold to 60% in qna maker azure web app bot using node.js

Question

I have created a web app bot for qna maker knowledge base in azure using node.js. Now I want my bot to give answers from knowledge base whose score is above 60%. For the answers below 60% the bot has to give the default answer. For this I tried changing the default parameters value in online code editor as shown below.

But on changing the const DefaultThreshold = 0.6 and running deploy.cmd in console. My bot gives same answer as previous.

How to make the bot to reply only if score is above 60%.

If you are getting 0.6 then either you can multiple with 100 and compare like ( 0.6 * 100 = 60 ) or consider as 0.6 is equal to 60% and tryout your scenario. — Rajeesh Menoth

Zeryth Zeryth · Accepted Answer · 2020-08-05T18:08:28

You can specify a minimum confidence score via the scoreThreshold option in a QnAMakerOption.

See the botbuilder-js repo for what options are available in QnAMakerOptions, including scoreThreshold.

Now where you could actually pass in the QnAMakerOptions, to ensure your answers have at least 60% confidence, you have a couple of options:

In the constructor of creating a new QnAMaker instance:

            const endpoint = {
                knowledgeBaseId: process.env.QnAKnowledgebaseId,
                endpointKey: process.env.QnAEndpointKey,
                host: process.env.QnAEndpointHostName
            };
            const options = { scoreThreshold: .6 };

            const qna = new QnAMaker(endpoint, options);

Pass in QnAMakerOptions when you call QnAMaker.getAnswers() method:

            const qna = new QnAMaker(endpoint);
            const options = { scoreThreshold: .6 }
            const qnaResults = await qna.getAnswers(context, options); // context is the TurnContext received in the callback of your bot's message handler

See sample 11.qnamaker in the official BotBuilder-Samples repo for basics on how to set up a QnA Maker bot. To deploy local changes to Azure (if you have the bot locally and aren't just using the online web editor to update your bot), follow Deploy your bot docs.

In the sample it just returns "No QnA Maker answers were found" if no QnA Maker answers were found within the scoreThreshold, but you could easily change this to instead send your desired default answer.

How to set confidence score settings/score threshold to 60% in qna maker azure web app bot using node.js

1 Answers