Azure website warmup on scale out

9
votes

I am trying to make sure that my website is fully warmed up when I deploy or scale out. To do this I have taken advantage of the App Initialization config as discussed here: http://ruslany.net/2015/09/how-to-warm-up-azure-web-app-during-deployment-slots-swap/

I am trying to get it to work on scale out as it says it is supported here: https://feedback.azure.com/forums/169385-websites/suggestions/6972595-application-initialization-to-warm-up-specific-pag

My config itself is set as follows:

<applicationInitialization>
  <add initializationPage="api/status"/>
</applicationInitialization>

I have not specified a hostname on the Initialization element as all discussions seem to say it is not required. I also would be hosting this across multiple environments, and would not know the host names for the scaled out versions and so it would be difficult to have all the config transforms required.

When I deploy the web app the warm up works fine, I can see it be deployed into the staging slot, then go to my Status Page. However when I scale out I cannot see that it is warming up. In fact I can see that api calls made to this api, sometimes come back with a 503 response, which is coming from the newly scaled app.

How can I prevent the scaled out application from receiving requests until it has warmed up? How can I get it to actually do the warm up as discussed here: https://feedback.azure.com/forums/169385-websites/suggestions/6972595-application-initialization-to-warm-up-specific-pag

Thanks

1
azureweb-applications
Did you find a solution we are having the exact same problem – Kisbys
Hi, no I didn't find anything. I havent looked into this for a while though so something may have changed. Sorry I am not much more help – ObiEff
Don’t worry cheers for replying! – Kisbys
we're having the same problem also stackoverflow.com/questions/47025021/… – Lance
Do you have ssl redirect enabled? – TWilly

1 Answers

4
votes

We had a similar issue scaling-out app services. When we add new instances, request received 502 Bad Gateway for 3 minutes.

I will explain the steps we follow with a support engineer and we could find our problem.

  1. Go to KUDU https://{appservicename}.scm.azurewebsites.net/support
  2. On tab Mitigate -> Slow Request

  3. Add a rule to get a memory dump when request take longer. We add a rule for:

    Number Of Requests : 50

    Time Taken (Seconds) : 10

    Interval (Seconds) : 60

You should change this value for your current workload. Also you can add another rule for 5xx errors...

Configuration for memory dump

  1. On tab "action". Add a "custom action". Change CollectKillAnalyze for CollectLogs. Help

Action

  1. Reproduce the issue and download the .dump file. You can find the link to these files from the portal on "Diagnose and solve problems". Or from Kudu in "D:\home\data\DaaS\Logs" enter image description here
  2. Analyze this file with DebugDiag or WinDebug
  3. We did not initialize Redis connection on warm up. So basically RedisClient.get has a sync block in the code which will only allow one thread to access a time. Since the volumes is very high, so hundreds of worker threads are waiting on the sync block. – it needs to be passed from the owner to the next then the next one by one.

Stack trace

After change Redis connection to the warm up. We did not have more issue when scaling out our service.

I hope this can help you.