In order to support multi-tenant Azure AD, you'll need to configure your ClaimsProvider in the custom policy with different values.
Use the values below, making sure you replace with client_id and IdTokenAudience.
<Item Key="DiscoverMetadataByTokenIssuer">true</Item>
<Item Key="ValidTokenIssuerPrefixes">https://sts.windows.net/</Item>
<Item Key="authorization_endpoint">https://login.microsoftonline.com/common/oauth2/authorize</Item>
<Item Key="client_id">df5b2515-a8d2-4d91-ab4f-eac6e1e416c2</Item>
<Item Key="BearerTokenTransmissionMethod">AuthorizationHeader</Item>
<Item Key="scope">openid</Item>
<Item Key="UsePolicyInRedirectUri">false</Item>
<Item Key="HttpBinding">POST</Item>
<Item Key="response_types">id_token</Item>
<Item Key="IdTokenAudience">df5b2515-a8d2-4d91-ab4f-eac6e1e416c2</Item>
CAUTION: This functionality isn't officially even in preview yet, so use with caution. Keep monitoring the official "Sign in by using Azure AD accounts" documentation to see when this is fully documented and supported.
EDIT:
This functionality is now in preview, see the official documentation for this: Allow users to sign in to a multi-tenant Azure AD identity provider using custom policies