As far as I know, the certificate is used to tell the user(client browser) this website is secure.
Normally, there are two reasons why the browser window showed the non-secure page.
You're probably using a self-signed certificate.
You may have left out intermediate certificates when you export your certificate to the PFX file.
So if you want your custom domain doesn't show the non-secure page, you need buy a certificate from the azure or some other company.
After verifying domain ownership, then you could bind this certificate to your azure web app.
After this operation, it will work well.
More details about how to buy certificate with custom domain in the azure, you could refer to this article.
After buying the azure certificate, you could find the certificate as below image shows:
After binding the certificate well, it will not show the no-secure page in the browser.
Image 1:
Image 2:
I saw that creating the fully hosted domain also created A Records pointing to their IP address instead of Azure. Do I need to override these somehow, or how should it work? The tutorial did not show if "contoso.com" was https.
In my opinion, fully hosted domain is the hosting company creates its own web app server to hosting your web app and change the A records point to its own web app server IP address.
So you will find the custom domain show that company pages.
If you want to set the custom domain still points to the azure web app, you need change the A record's IP address as azure web app's address.
If you could find the hosting company has already create a certificate and verified your domain.
Then you could download it and upload it in the azure web app's portal.
After binding the SSL, it will work well.
If the fully hosted domain doesn't contain the certificate, you need buy the certificate by yourself again, upload it in the azure web app portal, change the A record points to your azure web app's IP address.
The problem it seems is that when using a fully hosted site, I can add an A Record but the hosting company keeps using their own A Record which I cannot change, it is part of their "uneditable settings."
Since I don't know how your hosting company's fully hosted site works.
Now, the most easily way is you connect to its support team to change the domain setting firstly, then you follow this article to buy a azure certificate and bind it.
For second option, did you mean to download the SSL from the hosting company and use it as 3rd party SSL in Azure? The A Record would still point to the hosting company right ? Also it seems I do not have an option to download the SSL either in PFX format, I can only view the certificate configuration (CSR, key, etc).
Yes, if you have permission to download the PFX format(if you don't know how to do it, please connect to your hosting company support), you could use it as third party SSL.
