Scale out Microsoft bot framework app in multi region environment

Question

I am setting up the MS Bot framework service environment in Azure. I was able to successfully set up the channel which connects to single bot service for a single app. Now, we would like to scale this environment globally (all over the world) and we would like to setup multi-region environments. When a user connects from the channel app (MS Team) then they should be able to connect to their nearest Azure region and get the response back. How can we set up the geographic load balancer for Microsoft bot framework web app bot service?

We tried to set up the traffic manager however we have constraint since Microsoft bot channel registration service has Microsoft APP ID (ClientID) and Password and it can only connect to only one messaging endpoint URL

Actual results: Microsoft Bot channel registration app cannot connect to more than one messaging endpoints of the different region and how can we load balance MS Bot Service.

Expected results: How can we load balance (latency by region) MS Bot Application?

Sample Scale out diagram

Answer 1

Amit, Azure bots typically run as Azure App Services. The Azure App Service has built in scaling capabilities. Depending on the pricing tier you select for the App Service, you can scale out to as many as 20 instances. You can go to 100 instances if you're in an 'Isolated' tier. You can also scale up to add memory and cpu. That's some really powerful resources you can bring it to.

I realize that you're trying to reduce latency but I wanted to point the scaling feature out first. You have another challenge I don't think if possible to overcome at this time.

If MS Teams is the only channel you're users will be using, then trying to manage traffic on your own is probably going to be ineffective. You're constraint is going to be where the MS Teams service is located. Teams is what's talking to your bot, not the user directly.

The path is something like this: User -> MS Teams -> Azure Bot Service -> Azure App Service.

Since you have no control over the Teams to Bot connection, you cant manage the traffic.

You could deploy multiple bots to different regions, then instruct your users to connect to the appropriate regional bot channel in Teams. This isn't an automatic traffic management but would at least provide some of the region support you're looking for.