Apache HttpClient able to communicate over HTTPS when DIRECT but not via PROXY error: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

Question

I have read though many different examples but I am currently having difficulties trying to communicate via a proxy using HTTPS. I have a wrapper to create a Apache HttpClient as seen in the code below.

Currently if I make my call without setting up a proxy it will use my truststore from the SSLSocketFactory and correctly allow the communication via SSL. The only certificate required is a verisign server certificate which does not require authentication.

When I setup a proxy I get an error saying:

javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

I feel that I must be missing some type of proxy setup which makes the proxy connection use the same SSLSocketFactory?

I tested with -Djavax.net.debug=ssl and I can see a lot more SSL activity when going direct. When I use direct I can see all the keys loaded and sent with the request, when I use the proxy I only see:

httpConnector.receiver.3, setSoTimeout(30000) called
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1307565311 bytes = { 184, 216, 5, 151, 154, 212, 232, 96, 69, 73, 240, 54, 236, 26, 8, 45, 109, 9, 192,
227, 193, 58, 129, 212, 57, 249, 205, 56 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_C
BC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH
_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH
_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
]
Compression Methods:  { 0 }
***
httpConnector.receiver.3, WRITE: TLSv1 Handshake, length = 73
httpConnector.receiver.3, WRITE: SSLv2 client hello message, length = 98
httpConnector.receiver.3, handling exception: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
httpConnector.receiver.3, SEND TLSv1 ALERT:  fatal, description = unexpected_message
httpConnector.receiver.3, WRITE: TLSv1 Alert, length = 2
httpConnector.receiver.3, called closeSocket()
httpConnector.receiver.3, IOException in getSession():  javax.net.ssl.SSLException: Unrecognized SSL message, plaintext conn
ection?
httpConnector.receiver.3, called close()
httpConnector.receiver.3, called closeInternal(true)
httpConnector.receiver.3, called close()
httpConnector.receiver.3, called closeInternal(true)
2011-12-20 11:11:59,401 [httpConnector.receiver.3] INFO - The JavaScript method AddEvent threw an exception of type class co
m.alarmpoint.integrationagent.soap.exception.SOAPRequestException with message "javax.net.ssl.SSLPeerUnverifiedException: pe
er not authenticated".  The exception will be propogated up the call stack.

Can anyone help out please. Here is my code for setting up the proxy and SSLSocketFactory.

var client = httpClientWrapper.getHttpClient();
var proxy = new HttpHost(PROXY_HOST, PROXY_PORT, "https"); 
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy); 

var authpref = new ArrayList();
authpref.add(AuthPolicy.BASIC);

client.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authpref);

ServiceAPI.getLogger().debug("KeyStore.getDefaultType() " + KeyStore.getDefaultType());


var trustStore  = KeyStore.getInstance(KeyStore.getDefaultType());        
var instream = new FileInputStream(new File("conf/my.truststore")); 
try {
ServiceAPI.getLogger().debug("getting trustore");
trustStore.load(instream, "changeit".split(''));
} finally {
instream.close();
}


var socketFactory = new SSLSocketFactory(trustStore);
var sch = new Scheme("https", socketFactory, 443);

client.getConnectionManager().getSchemeRegistry().register(sch);

Stack trace:

Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(Unknown Source)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:488)
at org.apache.http.conn.scheme.SchemeSocketFactoryAdaptor.connectSocket(SchemeSocketFactoryAdaptor.java:62)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:561)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)

Answer 1

Here's a variant on aaron's solution, in Java (vs Groovy). This solution also avoids the HttpClientWrapper class (where does that come from?), and loads the proxy's certificate directly. It is written against HttpClient 4.2 (but I think it should work with 4.0). As an added bonus it includes an example of proxy authentication for a Windows proxy such as Microsoft ForeFront TMG.

It took me long enough to piece this together that I figured I should share it:

    HttpParams params = new BasicHttpParams();
    DefaultHttpClient.setDefaultHttpParams( params );   // Add the default parameters to the parameter set we're building
    DefaultHttpClient client = new DefaultHttpClient( params );

    KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
    trustStore.load( null );

    InputStream certStream = new FileInputStream( "cert-file" );
    CertificateFactory cf = CertificateFactory.getInstance("X.509");
    X509Certificate cert = (X509Certificate)cf.generateCertificate(certStream);
    certStream.close();
    trustStore.setCertificateEntry( "proxy-cert", cert );

    SSLSocketFactory socketFactory = new SSLSocketFactory(trustStore);
    client.getConnectionManager().getSchemeRegistry().register( new Scheme( "https", 443, socketFactory ));

    client.getParams().setParameter( ConnRoutePNames.DEFAULT_PROXY, 
                           new HttpHost( "my-proxy", 8080 ));

    // These 3 lines are only needed if your proxy is Windows based & requires authentication
    AuthScope scope = new AuthScope( "myproxy", 8080, null, AuthPolicy.NTLM );
    Credentials credentials = new NTCredentials( "username", "changeit", "WORKSTATION", "MY-DOMAIN" );
    client.getCredentialsProvider().setCredentials( scope, credentials );

    HttpGet get = new HttpGet( "https://mysite.com/resource" );
    String result = client.execute( get, new BasicResponseHandler() );

    System.out.println( result );

Answer 2

I solved this. The problem which I found once debugging into HttpClients code was the way my proxy was configured and the scheme's available.

HttpRoute[{tls}->https://someproxy->https://some_endpoint:443]

The problem was that the proxy was setup for https scheme but it was actually running on http. This became a problem when the wrapper did not configure a http scheme. In the end I created the SSLSocketFactory for my truststore and a default http scheme and setup my proxy correctly.

// Setup the Keystore and Schemes for the HttpClient and Proxy
var trustStore  = KeyStore.getInstance(KeyStore.getDefaultType());        
var instream = new FileInputStream(new File("conf/my.truststore")); 
try {
    trustStore.load(instream, "changeit".split(''));
} finally {
    instream.close();
}

var socketFactory = new SSLSocketFactory(trustStore);
var schHttp = new Scheme("http", PROXY_PORT, PlainSocketFactory.getSocketFactory());

// Create the HttpClient wrapper which will have the truststore SSLSocketFactory and a default http scheme and proxy setup 
httpClientWrapper = new HttpClientWrapper("some_endpoint", 443, "/", socketFactory);
var client = httpClientWrapper.getHttpClient();
var proxy = new HttpHost(PROXY_HOST, PROXY_PORT, "http"); 
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);

client.getConnectionManager().getSchemeRegistry().register(schHttp);

Answer 3

Have you tried using the global -Dhttp.proxyHost=proxy.host.com -Dhttp.proxyPort=8080 when launching your java process to verify that the SSLSocketFactory isn't falling back to proxyless communications.