Short version: I'm seeing an SSL certificate chain that's different based on how I access the https server. What's going on, and how do I verify the certificate under these circumstances?
Slightly longer version:
I'm trying to use libcurl to verify the certificate of an SSL connection. The server I'm connecting to is Amazon S3.
When I access Amazon S3 in Firefox, I get this certificate chain:
- VeriSign Class 3 Public Primary Certification Authority - G5
Serial Number: 18:DA:D1:9E:26:7D:E8:BB:4A:21:58:CD:CC:6B:3B:4A- VeriSign Class 3 Secure Server CA - G3
Serial Number: 6E:CC:7A:A5:A7:03:20:09:B8:CE:BC:F4:E9:52:D4:91- *.s3.amazonaws.com
Serial Number: 43:FB:BA:C2:66:27:E0:97:1E:1C:11:E0:30:C3:6B:66
- *.s3.amazonaws.com
- VeriSign Class 3 Secure Server CA - G3
When I access the same URL via the OpenSSL command line tool, I get this certificate chain:
- *.s3.amazonaws.com
Serial number: 43fbbac26627e0971e1c11e030c36b66 - VeriSign Class 3 Secure Server CA - G3
Serial number: 6ecc7aa5a7032009b8cebcf4e952d491 - VeriSign Class 3 Public Primary Certification Authority - G5
Serial number: 35973187f3873a07327ece580c9b7eda
The "*.s3.amazonaws.com" and "VeriSign Class 3 Secure Server CA - G3" certificates appear to be the same, but the one after that is different! It's named "VeriSign Class 3 Public Primary Certification Authority - G5" in both chains, but the certificate serial number is different. (Other information is different as well; let me know if you want a longer dump.)
I believe that this difference is the reason that I can't get libcurl to verify the SSL certificate. The certificate with serial number "18:DA:D1..." is in my CACERT.PEM file, but the certificate with serial number "35:97:31..." is not.
Obviously, the simple fix would be to add certificate "35:97:31..." to my CACERT.PEM file, but I want to make the right change here, not just a quick fix.
- What does this difference in certificate chains signify?
- Is it even possible for a SSL server to return different certificate chains based on the client (Firefox vs. OpenSSL/libcurl)?
- How should I get libcurl to verify this SSL certificate?
Really long version and background information:
I'm using libcurl with OpenSSL to download from Amazon S3. Libcurl is returning "SSL certificate problem: unable to get local issuer certificate", which I know means that the root certificate isn't listed in my CACERT.PEM file. (I'm using the one downloaded from curl's website, which is converted from Mozilla's certificate store.) I am able to verify certificates on other SSL connections, so I know my libcurl setup is correct.
To see what was going on, and why the certificate didn't verify, I pulled up the same URL in Firefox. Firefox didn't show any SSL certificate warnings. Here's the certification path Firefox shows. The root certificate, "VeriSign Class 3 Public Primary Certification Authority - G5", is listed in my CACERT.PEM file, and the serial number of the certificate matches what's shown in the screenshot.
Here's the serial numbers of all three certificates in the chain:
- VeriSign Class 3 Public Primary Certification Authority - G5
Serial Number: 18:DA:D1:9E:26:7D:E8:BB:4A:21:58:CD:CC:6B:3B:4A- VeriSign Class 3 Secure Server CA - G3
Serial Number: 6E:CC:7A:A5:A7:03:20:09:B8:CE:BC:F4:E9:52:D4:91- *.s3.amazonaws.com
Serial Number: 43:FB:BA:C2:66:27:E0:97:1E:1C:11:E0:30:C3:6B:66
- *.s3.amazonaws.com
- VeriSign Class 3 Secure Server CA - G3
On a different platform (different OS, different version of OpenSSL, etc.), I tried accessing the same URL using the OpenSSL command line tool, and I got a different certification path!
$ openssl s_client -showcerts -connect stackoverflowtest.s3.amazonaws.com:443
CONNECTED(00000003)
depth=3 C = US, O = "VeriSign, Inc.", OU = Class 3 Public Primary Certification Authority
verify return:1
depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5
verify return:1
depth=1 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA - G3
verify return:1
depth=0 C = US, ST = Washington, L = Seattle, O = Amazon.com Inc., OU = S3-A, CN = *.s3.amazonaws.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./OU=S3-A/CN=*.s3.amazonaws.com
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
-----BEGIN CERTIFICATE-----
MIIFOTCCBCGgAwIBAgIQQ/u6wmYn4JceHBHgMMNrZjANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTQwNjI1
MDAwMDAwWhcNMTUwNjA1MjM1OTU5WjB6MQswCQYDVQQGEwJVUzETMBEGA1UECBMK
V2FzaGluZ3RvbjEQMA4GA1UEBxQHU2VhdHRsZTEYMBYGA1UEChQPQW1hem9uLmNv
bSBJbmMuMQ0wCwYDVQQLFARTMy1BMRswGQYDVQQDFBIqLnMzLmFtYXpvbmF3cy5j
b20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC+mjGH4PrN0TDNmTsF
my2fHpHpTuySRwBMe/nuGIkL3cvKIxFKLHvkK9kx1UpO0skdQTdCY55LywhubLNO
fD19IzJdoRGdlqgkDAYC8vz3LRYj8WWsGnROfS/YFtgj25YaHPnsNp6lWrff4/qi
ctbojJpMxm+9Q0A4nTzrZymHEUkRbx6AVVUBVKH3uZi/w0aV+i4cp2bs+CYIK3DL
Qp584DJ9bOImgUhDfz19+Wtv64zIezE0Uz9eOkqgQ1X//XumyZWyD6N6+h/XqTnc
YTvIer/s83T/IngGMbfPRqjpQCay6ySXCNbJ5izMgo+gwN84t7JhaI+EYcxf1dDN
w0mXAgMBAAGjggF9MIIBeTAJBgNVHRMEAjAAMA4GA1UdDwEB/wQEAwIFoDAdBgNV
HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwZQYDVR0gBF4wXDBaBgpghkgBhvhF
AQc2MEwwIwYIKwYBBQUHAgEWF2h0dHBzOi8vZC5zeW1jYi5jb20vY3BzMCUGCCsG
AQUFBwICMBkaF2h0dHBzOi8vZC5zeW1jYi5jb20vcnBhMB8GA1UdIwQYMBaAFA1E
XBZTRMGCfh0gqyX0AWPYvnmlMCsGA1UdHwQkMCIwIKAeoByGGmh0dHA6Ly9zZC5z
eW1jYi5jb20vc2QuY3JsMFcGCCsGAQUFBwEBBEswSTAfBggrBgEFBQcwAYYTaHR0
cDovL3NkLnN5bWNkLmNvbTAmBggrBgEFBQcwAoYaaHR0cDovL3NkLnN5bWNiLmNv
bS9zZC5jcnQwLwYDVR0RBCgwJoISKi5zMy5hbWF6b25hd3MuY29tghBzMy5hbWF6
b25hd3MuY29tMA0GCSqGSIb3DQEBBQUAA4IBAQBlXrn1FTPjVIFbcuQNbBesrAMI
NV4L7jS1mobEwFrb7UrqZ7kHvuvoR/BpDygATyqLvPihs7nUc2TUHsw/41EAHKoq
QBVfRTOH0yWaTC6SYSx7fiElL+k55Pvrz4+7gLRy5zUlVX3iUw93zr95ka/LPuCL
7PQOFPeQDOgveDjcSNVtLcTfQfvog/rMSu/4XPFHu7zaZwUEurt9CzLeVbdB6O25
bHuHTaZLP0wmjCIbwgXu8bFWqOTnAjG70EtYrbIiQhl/ISJU6HioFzLiy5Ibp07r
RbV4ir5EI2EPKxIy30YDvpCQ0WQvYLWFV0qQOuOXkMC2M2IsBmVn2/9GQ7eP
-----END CERTIFICATE-----
1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
-----BEGIN CERTIFICATE-----
MIIF7DCCBNSgAwIBAgIQbsx6pacDIAm4zrz06VLUkTANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMTAwMjA4MDAwMDAwWhcNMjAwMjA3MjM1OTU5WjCBtTEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQg
aHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMmVmVy
aVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCxh4QfwgxF9byrJZenraI+nLr2wTm4i8rCrFbG
5btljkRPTc5v7QlK1K9OEJxoiy6Ve4mbE8riNDTB81vzSXtig0iBdNGIeGwCU/m8
f0MmV1gzgzszChew0E6RJK2GfWQS3HRKNKEdCuqWHQsV/KNLO85jiND4LQyUhhDK
tpo9yus3nABINYYpUHjoRWPNGUFP9ZXse5jUxHGzUL4os4+guVOc9cosI6n9FAbo
GLSa6Dxugf3kzTU2s1HTaewSulZub5tXxYsU5w7HnO1KVGrJTcW/EbGuHGeBy0RV
M5l/JJs/U0V/hhrzPPptf4H1uErT9YU3HLWm0AnkGHs4TvoPAgMBAAGjggHfMIIB
2zA0BggrBgEFBQcBAQQoMCYwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLnZlcmlz
aWduLmNvbTASBgNVHRMBAf8ECDAGAQH/AgEAMHAGA1UdIARpMGcwZQYLYIZIAYb4
RQEHFwMwVjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL2Nw
czAqBggrBgEFBQcCAjAeGhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMDQG
A1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMtZzUu
Y3JsMA4GA1UdDwEB/wQEAwIBBjBtBggrBgEFBQcBDARhMF+hXaBbMFkwVzBVFglp
bWFnZS9naWYwITAfMAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq1EgYLHsZLjAlFiNo
dHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAoBgNVHREEITAfpB0w
GzEZMBcGA1UEAxMQVmVyaVNpZ25NUEtJLTItNjAdBgNVHQ4EFgQUDURcFlNEwYJ+
HSCrJfQBY9i+eaUwHwYDVR0jBBgwFoAUf9Nlp8Ld7LvwMAnzQzn6Aq8zMTMwDQYJ
KoZIhvcNAQEFBQADggEBAAyDJO/dwwzZWJz+NrbrioBL0aP3nfPMU++CnqOh5pfB
WJ11bOAdG0z60cEtBcDqbrIicFXZIDNAMwfCZYP6j0M3m+oOmmxw7vacgDvZN/R6
bezQGH1JSsqZxxkoor7YdyT3hSaGbYcFQEFn0Sc67dxIHSLNCwuLvPSxe/20majp
dirhGi2HbnTTiN0eIsbfFrYrghQKlFzyUOyvzv9iNw2tZdMGQVPtAhTItVgooazg
W+yzf5VK+wPIrSbb5mZ4EkrZn0L74ZjmQoObj49nJOhhGbXdzbULJgWOw27EyHW4
Rs/iGAZeqa6ogZpHFt4MKGwlJ7net4RYxh84HqTEy2Y=
-----END CERTIFICATE-----
2 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
-----BEGIN CERTIFICATE-----
MIIExjCCBC+gAwIBAgIQNZcxh/OHOgcyfs5YDJt+2jANBgkqhkiG9w0BAQUFADBf
MQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xNzA1BgNVBAsT
LkNsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkw
HhcNMDYxMTA4MDAwMDAwWhcNMjExMTA3MjM1OTU5WjCByjELMAkGA1UEBhMCVVMx
FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVz
dCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2lnbiwgSW5jLiAtIEZv
ciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAz
IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzUwggEi
MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1nmAMqudLO07cfLw8
RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbext0uz/o9+B1fs70Pb
ZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIzSdhDY2pSS9KP6HBR
TdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQGBO+QueQA5N06tRn/
Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+rCpSx4/VBEnkjWNH
iDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/NIeWiu5T6CUVAgMB
AAGjggGRMIIBjTAPBgNVHRMBAf8EBTADAQH/MDEGA1UdHwQqMCgwJqAkoCKGIGh0
dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMuY3JsMA4GA1UdDwEB/wQEAwIBBjA9
BgNVHSAENjA0MDIGBFUdIAAwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVy
aXNpZ24uY29tL2NwczAdBgNVHQ4EFgQUf9Nlp8Ld7LvwMAnzQzn6Aq8zMTMwNAYD
VR0lBC0wKwYJYIZIAYb4QgQBBgpghkgBhvhFAQgBBggrBgEFBQcDAQYIKwYBBQUH
AwIwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAHBgUr
DgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVyaXNp
Z24uY29tL3ZzbG9nby5naWYwNAYIKwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhho
dHRwOi8vb2NzcC52ZXJpc2lnbi5jb20wDQYJKoZIhvcNAQEFBQADgYEADyWuSO0b
M4VMDLXC1/5N1oMoTEFlYAALd0hxgv5/21oOIMzS6ke8ZEJhRDR0MIGBJopK90Rd
fjSAqLiD4gnXbSPdie0oCL1jWhFXCMSe2uJoKK/dUDzsgiHYAMJVRFBwQa2DF3m6
CPMr3u00HUSe0gST9MsFFy0JLS1j7/YmC3s=
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./OU=S3-A/CN=*.s3.amazonaws.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
---
No client certificate CA names sent
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 4624 bytes and written 399 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-SHA
Session-ID: B642ED1E6FE32F7A374B5A62847DEC63C2F37DCA7A18FD669B8F0FCDC98C49BF
Session-ID-ctx:
Master-Key: EE2D31F43D341A0895B36E0BCCE7557B221F1469AC1B7B0BA22D843C75F25F949822B0D0E22E967A1F373F034E9624E4
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1430871883
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
closed
When I decode the certificates it lists, I get a different certificate chain than Firefox gave me.
- *.s3.amazonaws.com
Serial number: 43fbbac26627e0971e1c11e030c36b66 - VeriSign Class 3 Secure Server CA - G3
Serial number: 6ecc7aa5a7032009b8cebcf4e952d491 - VeriSign Class 3 Public Primary Certification Authority - G5
Serial number: 35973187f3873a07327ece580c9b7eda
The cert in use, and its immediate parent cert are the same, but the next one up has the same name, but a different serial number.
Here's my version information:
> curld --version curl 7.40.0 (i386-pc-win32ce) libcurl/7.40.0 OpenSSL/1.0.1e Protocols: dict ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps tftp Features: NTLM SSL
> curld --version curl 7.40.0 (i386-pc-win32) libcurl/7.40.0 OpenSSL/1.0.1c Protocols: dict ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps tftp Features: AsynchDNS Largefile NTLM SSL
$ cat /etc/redhat-release CentOS release 6.6 (Final) $ openssl version OpenSSL 1.0.1e-fips 11 Feb 2013 $ yum list openssl Installed Packages openssl.x86_64 1.0.1e-30.el6.8 @updates
Firefox: 37.0.2, running on Windows 7 x64.
I did review this question: SSL Certificate - Certification Path in browser different from Certificate Chain File, but my problem appears to be different: In that case, the certificate chain in the OpenSSL command line tool went 1 - 2 - 3 - 4, and in IE it was 1 - 2 - 3. Because IE considered "3" to be a root certificate, the chain stopped early. In my case, Firefox is reporting 1 - 2 - 3, and OpenSSL is reporting 1 - 2 - 4; The chain is different.
