By default Azure VMs (even those behind an internal load balancer) can communicate with endpoints outside of Azure in public IP address space. (You can change this behavior using a Network Security Group for example.)
See Understanding outbound connections in Azure for more information.
When a scale set is created in the portal, it is created by default with a load balancer and NAT pools. NAT rules are created to route incoming load balancer connections to the SSH/RDP ports of the scale set VMs depending on the platform.
You can also edit similar templates in Azure quickstart templates, depending on whether you're looking for different NAT rules. In these cases, outgoing internet access is via the default load balancer DNS configuration (you can also set DNS configuration at the scale set level).
Quickstart template examples:
https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-windows-nat
https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-linux-nat