How to address high latency on Managed QnA Maker/Azure Cognitive Search Service after a period of no load

1
votes

How do I prevent the Azure cognitive search service from becoming cold after a period of no load?

I use QnaMaker in chatbots (Managed QnA Maker Service version).

QnaMaker seems to idle after not being used for a while. The first query takes 7 seconds to complete. Every query after the first one completes within a second.

The chatbots themselves (developed with Microsoft Bot Framework V4 nodejs) show no latency. Even after not being used for a while (thanks to the "Always on" feature in the bots appservice)

I use the stable GA version of QnaMaker and the managed version (in preview). In the first version, the appservice for QnaMaker has the feature "Always On" enabled. For the preview version, there is no appservice I can check.

Chatbotcode initialization QnaMaker

const { QnAMaker } = require('botbuilder-ai');
        const endpoint_fr = {
            knowledgeBaseId: process.env.QnAKbId_fr,
            endpointKey: process.env.QnaEndpointKey_fr,
            host: process.env.QnaHostName_fr
        };
        try {
            this.qnaMaker_fr = new QnAMaker(endpoint_fr, {});
        } catch (err) {
            console.warn(`QnAMaker Exception: ${err} Check your QnAMaker configuration in .env`);
        }

the actual call to QnAMaker service

   qnaResults = await this.qnaMaker_fr.getAnswers(stepContext.context);

The QnAMaker stack is linked to a Azure Search-resource. The one I am using has pricing tier Basic (1 replica, partition and search unit) 10 indexes are in use (15 are allowed) to store 7 knowledge bases. location is West Europe.

How do I prevent the Azure cognitive search service from becoming cold after a period of no load?

[update] Did some more digging and came to the conclusion that this cold start only occured for knowledge bases in the (preview) managed service. I decided to move all KB's to the stable version and the cold start problems stopped. This could also have to do with the fact that I am in Western Europe and the managed version is available in Northern Europe only

3
botframeworkazure-cognitive-searchqnamaker
Could you please elaborate a bit more on your architecture? I'm hearing that you are using QnA Maker service, the bot service, and Azure Cognitive Search, but not sure how each is being called and from where, and that will affect how to fix cold-start problems. – Jennifer Marsman - MSFT
hi @JenniferMarsman-MSFT. I did add some additional information. Hope this is what you were looking for. – Hessel

3 Answers

2
votes

I'm glad that you are using "Always On". The QnA Maker team sometimes recommends adding quick availability/web test using Azure app insights monitor (see https://docs.microsoft.com/azure/azure-monitor/app/monitor-web-app-availability). This is essentially a probing service every few seconds that can resolve the cold-start problem.

1
votes

I'm not sure about App Insights, but I do have a regular test I run against the QnA Maker service, primarily to monitor uptime, but it would double as keeping the service "warm". I set up an Azure Function (linked to the same App Service Plan as the QnA Maker app service) and just have it make a standard REST query at a defined interval (I use 15 minutes, you can use whatever and especially if you are using standard service tier there is no limit or extra costs on number of queries). You will just need to get the appropriate keys from the resource to make the request and store in your configuration or Key Vault, which I assume you are already familiar with from your bot itself. Should not create any incremental costs for you.

If it is possible to keep the service warm with just a ping, Jennifer's suggestion may work, but I can tell you that I have had 0 issues with the automated tests I've been running via Azure Functions.

0
votes

I moved all my QNA knowledge bases back to the stable version. The managed version seemed to cause the cold start. Not sure why but my bots are performing again.