How do browsers decrypt a self-signed certificate?

Question

I have done some research about how do browsers work with CA-signed certificates. The browser comes with a list of trusted CA's public keys. When a CA-signed certificate was received by a browser, the browser decrypts the certificate using the trusted CA's public key to get the server's public key.

I am confused about how does the browser work with self-signed certificates. A self-signed certificate is signed by the server's private key. When a browser receives a self-signed certificate, how does the browser decrypt the cert to get the public key without knowing the public key?

Answer 1

When a CA-signed certificate was received by a browser, the browser decrypts the certificate using the trusted CA's public key to get the server's public key.

The browser does not decrypt the certificate to get the public key. The browser just uses the public key of the CA to verify the issuers signature of the certificate. See SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate?.

When a browser receives a self-signed certificate, how does the browser decrypt the cert to get the public key without knowing the public key?

Again, there is no decryption. But apart from that there is no way for the browser to check if the self-signed certificate is valid. A self-signed certificate basically is somebody claiming to be themselves. Contrary to a CA signed certificate there is no trusted third party which says this but just the (untrusted) owner of the certificate itself.

Thus the only way such a certificate can be trusted in the browser is either because it was explicitly added as trusted or if the user of the browser explicitly added an exception after they got a warning about the certificate.