QnA Maker generally available vs preview: it doesn't find the answers

Question

It's a bit strange. I have created a knowledge base with the same excel file both in preview and generally available QnA Maker service but it doesn't find a lot of questions in the last one... In the preview one works fine.

I have noticed that the preview one finds the answers with any word order but not in the generally available.

What do I have to do? What is the problem?

I'm using the QnAMaker test:

EDIT 1: If i go in Azure to my Search Service -> Search Explorer it works fine and i find the answers correctly, as in QnA Preview Test, but not in QnA GA Test.

EDIT 2: I'm using node. I've created an azure SDKv3 Qna template bot, edited the QnAAuthKey/QnAEndpointHostName/QnAKnowledgebaseId and i've tried it with "Test in Web Chat" but i get always the "No match! Try changing the query terms!" message.

Also i've tried to get the “hostname” here that differs from the “host” param given by QnaMaker when I created the KB (ended with /qnamaker). Then i've used it with the code to get answers from here, but I receive a “not found” error: https://xx.azurewebsites.net/qnamaker/knowledgebases/xxx.../generateAnswer. Error: getaddrinfo ENOTFOUND https://xxx.azurewebsites.net https://xx.azurewebsites.net:443

EDIT 3: after multiple test with the different languages of my KB, i think that the problem is with language recognition changes and with Confidence scores. I'm rewriting some questions now...

The GA stack has a new and improved ranking algorithm, so it’s likely that you will see some variations in the confidence score of the response compared to the preview stack.

Answer 1

Since QnAMaker is available, the search method has changed: it is now using Azure Search and not only token comparison as you could have seen before.

New architecture is the following:

The main impact is a great improvement of the search functionality as in the past it could found answers not relevant.

For example if your KB looks like the following:

• Question1: "I want a car" / Answer1: "You could rent a car at ..."

• Question2: "How can I get a sandwich?" / Answer2: "Are you hungry? ..."

If you asked your KB "How can I get a car?", it may have answer "Are you hungry? ..."!

Regarding the performance of your search, there is no real documentation currently to explain how to improve it but it must be mostly based on indexes

Answer 2

We had similar issue and it looks like, it was already fixed on MS side. Recreation new KB from scratch really helped.

Answer from Microsoft:

It looks like your KB is effected by a recent bug that we have now fixed - it was causing the relevance of the results to be degraded. Can you recreate the Knowledge base and see if you are still facing these query matching issues?

To quickly recreate the KB, simply "Export the knowledge base" from the "Settings" tab in the knowledge base. Create a new empty KB and "Import knowledge base" in the "Settings" page.

Answer 3

After publishing when you get all the necessary settings info like knowledgebase id, authkey, host url. you can add then in code like this

QnADialog.cs

namespace Test.Qna
{
    [Serializable]
    [QnAMaker(authKey: "AuthKey", knowledgebaseId: "KnowledgebaseId", defaultMessage: "please rephrase, I could not understand.", scoreThreshold: 0.5, top: 1, endpointHostName: "https://yourAccount.azurewebsites.net/qnamaker")]
    public class QnADialog : QnAMakerDialog
    {}
}

If you are using Node js then you can add the settings like this

    var server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 3978, function () {
    console.log('%s listening to %s', server.name, server.url);
});

var connector = new builder.ChatConnector({
    appId: process.env.MICROSOFT_APP_ID,
    appPassword: process.env.MICROSOFT_APP_PASSWORD
});
var bot = new builder.UniversalBot(connector);
bot.set('storage', new builder.MemoryBotStorage()); 
server.post('/api/messages', connector.listen());

var recognizer = new cognitiveservices.QnAMakerRecognizer({
    knowledgeBaseId: '5abcde-cbfb-4yuio-92c5-052d3a806e78',
    authKey: 'eb7uy78y-8a64-4e75-98uj-7f89987b67bc',
    endpointHostName: 'https://name.azurewebsites.net/qnamaker'
    });

var basicQnAMakerDialog = new cognitiveservices.QnAMakerDialog({
    recognizers: [recognizer],
    defaultMessage: 'No match! Try changing the query terms!',
    qnaThreshold: 0.3
});

bot.dialog('/', basicQnAMakerDialog);

Hope this will help. For step by step guide follow this link