I setup a Sitecore infrastructure on azure (I created the same before and it worked) and when I connect to the web apps directly, they all work. I configured access to two of the apps via Application Gateway - while one works, the other one gives 502 on the health status.
I checked on the application and there it shows me that the error is "ERROR_INTERNET_CONNECTION_ABORTED".

When I completely deactivate Sitecore (rename default.aspx and web.config) and put an index.html page, the application gateway can access it.
As mentioned - I have a running deployment that I did with the same ARM template. I also deleted everything and redeployed - same issue. As mentioned - direct access to the web app works perfectly. I have NO network restrictions on the web app yet.