It's worth shedding a bit more "hands-on" light about what happens here, adding upon @Steffen Ullrich's answer here and elsewhere:
Notes:
- I'll use another website than the OP, because the OP's website currently has no issues.
- I used Ubunto to run the following commands (
curl
and openssl
). I tried running curl
on my Windows 10, but got different, unhelpful output.
The error experienced by the OP can be "reproduced" by using the following curl
command:
curl -vvI https://www.vimmi.net
Which outputs (note the last line):
* TCP_NODELAY set
* Connected to www.vimmi.net (82.80.192.7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, Server hello (2):
* SSL certificate problem: unable to get local issuer certificate
* stopped the pause stream!
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
Now let's run it with the --insecure
flag, which will display the problematic certificate:
curl --insecure -vvI https://www.vimmi.net
Outputs (note the last two lines):
* Rebuilt URL to: https://www.vimmi.net/
* Trying 82.80.192.7...
* TCP_NODELAY set
* Connected to www.vimmi.net (82.80.192.7) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* [...]
* Server certificate:
* subject: OU=Domain Control Validated; CN=vimmi.net
* start date: Aug 5 15:43:45 2019 GMT
* expire date: Oct 4 16:16:12 2020 GMT
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
The same result can be seen using openssl
, which is worth mentioning because it's used internally by python:
echo | openssl s_client -connect vimmi.net:443
Outputs:
CONNECTED(00000005)
depth=0 OU = Domain Control Validated, CN = vimmi.net
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 OU = Domain Control Validated, CN = vimmi.net
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:OU = Domain Control Validated, CN = vimmi.net
i:C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2
---
Server certificate
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
[...]
---
DONE
So why both curl
and openssl
can't verify the certificate Go Daddy issued for that website?
Well, to "verify a certificate" (to use openssl's error message terminology) means to verify that the certificate contains a trusted source signature (put differently: the certificate was signed by a trusted source), thus verifying vimmi.net
identity ("identity" here strictly means that "the public key contained in the certificate belongs to the person, organization, server or other entity noted in the certificate").
A source is "trusted" if we can establish its "chain of trust", with the following properties:
- The Issuer of each certificate (except the last one) matches the Subject of the next certificate in the list
- Each certificate (except the last one) is signed by the secret key corresponding to the next certificate in the chain (i.e. the signature
of one certificate can be verified using the public key contained in
the following certificate)
- The last certificate in the list is a trust anchor: a certificate that you trust because it was delivered to you by some trustworthy
procedure
In our case, the issuer is "Go Daddy Secure Certificate Authority - G2". That is, the entity named "Go Daddy Secure Certificate Authority - G2" signed the certificate, so it's supposed to be a trusted source.
To establish this entity's trustworthiness, we have 2 options:
Assume that "Go Daddy Secure Certificate Authority - G2" is a "trust anchor" (see listing 3 above). Well, it turns out that curl
and openssl
try to act upon this assumption: they searched that entity's certificate on their default paths (called CA paths), which are:
- for
curl
, it's /etc/ssl/certs
.
- for
openssl
, it's /use/lib/ssl
(run openssl version -a
to see that).
But that certificate wasn't found, leaving us with a second option:
- Follow steps 1 and 2 listed above; in order to do that, we need to get the certificate issued for that entity.
This can be achieved by downloading it from its source, or using the browser.
- for example, go to
vimmi.net
using Chrome, click the padlock > "Certificate" > "Certification Path" tab, select the entity > "View Certificate", then in the opened window go to "Details" tab > "Copy to File" > Base-64 encoded > save the file)
Great! Now that we have that certificate (which can be in whatever file format: cer
, pem
, etc.; you can even save it as a txt
file), let's tell curl
to use it:
curl --cacert test.cer https://vimmi.net
Going back to Python
Once we have:
- "Go Daddy Secure Certificate Authority - G2" certificate
- "Go Daddy Root Certificate Authority - G2" certificate (wasn't mentioned above, but can be achieved in a similar way).
We need to copy their contents into a single file, let's call it combined.cer
, and let's put it in the current directory. Then, simply:
import requests
res = requests.get("https://vimmi.net", verify="./combined.cer")
print (res.status_code) # 200
- BTW, "Go Daddy Root Certificate Authority - G2" is listed as a trusted authority by browsers and various tools; that's why we didn't have to specify it for
curl
.
Further reading:
https://www.thenewboston.com/tops.php?type=text&period=this-month&page=1
. Can you verify the page exists? – NuclearPeonhttps://www.thenewboston.com/forum/category.php?id=15&orderby=recent&page=
instead which should bring you to the Python section of the website and I'm getting the exact same error with it. – Bill Jenkins