App Service Hybrid Connections is what you're looking for. Caveman simple to setup, no firewall whack-a-mole-ing necessary.
Azure Friday video: Hybrid Connections to On-premises Resources
The Hybrid Connection Manager can very well live off another machine if you're trying to avoid installing more stuff on top of your WCF servers - see the diagram here.
You could also establish a site-to-site VPN, but you really don't need that complication in your life.
And is it possible to configure the WCF security to pass client's credential (Azure AD) to WCF service, and then authenticate it against on-premises AD?
I'm far from familiar with WCF, if it's okay reading claims from a JWT token, it should work. Azure AD never passes down credentials to your app, only a set of claims wrapped in a JSON Web Token.