Apache Nifi Twitter Connection Issue

Question

I am attempting to configure the GetTwitter processor for a Nifi flow. I've set the required properties, such as access token and consumer key. However, when I turn on the processor, I get the following error:

Received error CONNECTION_ERROR: sun.security.validator.validatorexception pkix path building failed sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Will attempt to reconnect

So it is pretty clear that there is some sort of certificate/security issue going on. How would I fix this?

EDIT: Adding openssl command output

Andy Andy · Accepted Answer · 2017-01-31T18:30:45

Yes, currently your processor is saying that it receives a certificate identifying https://www.twitter.com (or whatever the actual URL is) but it cannot build a complete chain between the presented certificate and a known CA/trusted certificate. This is because by default, NiFi doesn’t know any trusted certificates.

Can you try to establish that your machine can validate the Twitter certificate outside of Java? You can use this OpenSSL command to do so:

$ openssl s_client -connect <host:port> -debug -state

You should see a result that is quite long but contains:

depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Twitter, Inc.", OU = Twitter Security, CN = api.twitter.com
verify return:1
SSL_connect:SSLv3 read server certificate A

and eventually:

SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 7FD9B2...F2A0CD
    Session-ID-ctx:
    Master-Key: 5847F71...0C2599
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 129600 (seconds)
    TLS session ticket:
    0000 - 0a c4 e2 31 be 96 ac 47-87 a4 38 98 0f 39 cf 24   ...1...G..8..9.$
    ...
    0090 - 14 c9 bd 6a d7 ca 01 6b-09 40 6a eb 5d e0 4e f5   ...j...k.@j.].N.

    Start Time: 1485890791
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)

The important part is the Verify return code: 0 (ok).

If that is successful, you can then check that Java has the correct CA certificates marked as trusted in the truststore. Depending on your version of Java and OS, you may need to update the JRE and your ca-certificates package (on *nix).

EDIT

What I wrote below applies to GetHTTP and InvokeHTTP processors, not GetTwitter.

You can configure a StandardSSLContextService in Controller Services which points the truststore file to $JRE_HOME/lib/security/cacerts (for example, on my Mac, it is /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/security/cacerts), and set the truststore type to JKS and the truststore password to changeit.

There is an existing Jira discussing adding this by default, but there are pros and cons to that decision.

Apache Nifi Twitter Connection Issue

1 Answers