Can One Public Key be Used to Encrypt and Decrypt Data during the SSL Handshake?

Question

When a server sends a Certificate message to a client, the public key in the server's certificate will be used to verify server’s identity(decryption with the public key).

The server follows its Certificate message with a ServerKeyExchange message, the session key information is signed using the same public key contained in the server’s certificate (encryption with the public key).

So I feel a Public key can be used to encrypt and decrypt data as well, am I right? If yes, I wonder why text book just states one key(e.g. public key) is used to encrypt , and the other one(private key) is used to decrypt, rather than mention that a key can be used to both encrypt and decrypt?

[UPDATE2]
Really thanks for Bruno's help.

After reading Bruno's replies and RFC4346(section7.4.2 and 7.4.3) again and again, I suddenly felt I grasp the main points. :)
However, I am not sure I am right, and hope someone can confirm my following understanding. Thanks.

1.Server Certificate
SSL and TLS Essential section 3.6.1:
(SSL and TLS Essential: Securing the Web Author: Stephen A. Thomas)

...the public key in the server's certificate will only be used to verify its(server) identity.

Bruno wrote,

The public key in the server certificate isn't used to verify the server's identity itself.

I agree Bruno's viewpoint now, because a certificate is only a private-key-signed(encrypted) message that also contains someone else's(e.g., a client ) public key, so a client should use its trusted copy of the server's public key (usually, web browsers include dozens of these certificates in advance), instead of the public key in the server certificate, to verify the server's identity.

Is it right?

2.Server Key Exchange
SSL and TLS Essential section 3.6.2:

...the key information is signed using the public key contained in the server's certificate.

Bruno wrote,

Similarly, you don't really sign something with a public key. You only need one of the keys to sign, and that's the private key. You verify the signature with the matching public key. ... "signing with a public key" is an unusual and misleading expression.

I think Bruno is right. The reasons are as follows,

RFC4346 section 7.4.2. Server Certificate

It MUST contain a key that matches the key exchange method, as follows.

Key Exchange Algorithm           Certificate Key Type

 RSA                     RSA public key; the certificate MUST
                         allow the key to be used for encryption.

 DHE_DSS                 DSS public key.

 DHE_RSA                 RSA public key that can be used for signing.

 DH_DSS                  Diffie-Hellman key. The algorithm used
                         to sign the certificate MUST be DSS.

 DH_RSA                  Diffie-Hellman key. The algorithm used
                         to sign the certificate MUST be RSA.

So the server sends one of 6 public key types in a certificate first.

RFC4346 section 7.4.3 Server Key Exchange Message

The server key exchange message is sent by the server.
...
This is true for the following key exchange methods:

DHE_DSS
DHE_RSA
DH_anon
...
This message conveys cryptographic information to allow the client to communicate the premaster secret.

The server chooses one of 3 key exchange methods, and uses its private key to sign(encrypt) cryptographic information.

When the client receives the encrypted cryptographic information, it will use the public key in ServerCertificate message to verify(decrypt) and get the plain-text cryptographic information.

Is it right?

Answer 1

In public key cryptography:

• The private key is used for signing and deciphering/decrypting.
• The public key is used for verifying signatures and enciphering/encrypting.

See the glossary of the TLS specification:

public key cryptography: A class of cryptographic techniques employing two-key ciphers. Messages encrypted with the public key can only be decrypted with the associated private key. Conversely, messages signed with the private key can be verified with the public key.

You cannot encrypt with a private key or decrypt with a public key, not for mathematical reasons, but because it doesn't make sense w.r.t. the definition of encrypt:

To convert ordinary language or other data into code; to hide the meaning of a message by converting it into a form that cannot be interpreted without knowing the secret method for interpretation, called the key; to encode.

In a situation where you "encrypt with the private key", you effectively "scramble" the data indeed, but what's required to turn back the message into its original form is not a secret. Hence, it doesn't make sense to talk about encrypting in this context. Whether the mathematical operations behind it may work one way or the other doesn't matter at this stage.

Similarly, you don't really sign something with a public key. You only need one of the keys to sign, and that's the private key. You verify the signature with the matching public key.

It's quite common (even in the TLS specification) to say "signing with a certificate", when what's really implied is computing the signature with the private key matching the certificate. In many cases, not specifically TLS, the certificate itself is communicated along with the signature (whether-or-not one chooses to trust that certificate is another matter).

Expressions such as "using your certificate to authenticate" or "using your certificate to sign" are generally acceptable, so long as you understand that "certificate" is used there to shorten "certificate and its private key", and that it's in fact the private key that's necessary for those operations.

I don't have the book you're quoting, but this quote sounds misleading or incorrect (perhaps taken out of context here):

...the public key in the server's certificate will only be used to verify its(server) identity.

The public key in the server certificate isn't used to verify the server's identity itself. What it does is ensuring that only someone/something with the corresponding private key will be able to decipher what you've encrypted with this public key: in this case (authenticated key exchange), the pre-master-secret which the server will prove it knows to the client by producing the correct Finished message, based on the pre-master-master it has managed to decipher.

The identity binding is done by the certificate itself, with is the signed combination of the public key, some identifiers (e.g. Subject DN and Subject Alternative Names) and possibly various other attributes (e.g. key usage, ...). This side of the identity verification (i.e. checking who this certificate belongs to) is established by verifying the integrity of the certificate and that you trust what it says (usually PKI), and by verifying that the identity it belongs to is indeed the one you wanted to connect to (host name verification). This is done by verifying the certificate signature itself using a Certification Authority (CA) or by an outside mechanism, for example if you have explicitly granted an exception for a given certificate (possibly self-signed) using knowledge you have outside the scope of the PKI to which the certificate belongs. This step is rather independent of the TLS specification, although you'll need all these pieces together to make the communication secure.

There's a similar problem with this quote (again, possibly taken out of context):

...the key information is signed using the public key contained in the server's certificate.

Although saying "signed with a certificate" is a common expression (as explained above), I'd say "signing using the public key" is definitely confusing, since "public key" is normally used in contrast to "private key", and it's really the private key that's used for signing. While even the TLS specification (Section F.1.1.2) talks about "signing with a certificate" in a few places, "signing with a public key" is an unusual and misleading expression.

I'm not sure whether "(decrypt)" and "(encrypt)" are in the book or your additions:

the public key in the server's certificate can be used to verify(decrypt) the server's identity and sign(encrypt) the key information(,then the client will use the key information to encrypt pre_master_secret)

You actually verify you're talking to the actual server identified by that certificate because it's the only one capable of deciphering what you've encrypted with its public key (in the client key exchange message).

As it's put in the TLS specification Section F.1.1.2:

After verifying the server's certificate, the client encrypts a
pre_master_secret with the server's public key. By successfully
decoding the pre_master_secret and producing a correct finished
message, the server demonstrates that it knows the private key
corresponding to the server certificate.

What you're asking at the end doesn't completely make sense:

I know the public key can be used to verify server's identity(Certificate message), but I can't understand the public key why can be used to sign the key information, because the client doesn't have the corresponding private key, how does the client verify the key information?

The public key isn't used to verify the server's identity. You verify that you're talking to the server who has the private key matching the certificate it presented earlier by the fact it was able to decipher the pre-master-key and produce the correct finished message.

EDIT 2:

Following your edit, it seems that you're still using "sign(encrypt)" and "verify(decrypt)" as if encrypting was the same as signing and verifying was the same as decrypting. I would suggest once again that you stop making these associations: these are 4 distinct operations. While the maths may be the same when using RSA, this doesn't work for DSA, which is only a signature algorithm (so signing/verifying only).

When the client receives the encrypted cryptographic information, it will use the public key in ServerCertificate message to verify(decrypt) and get the plain-text cryptographic information.

The client doesn't receive any encrypted data during the handshake (only signed data).

For a better general understanding, you should start by trying to understand how Diffie-Hellman and its ephemeral variant (for DHE cipher suites) work. (In practice, I wouldn't focus too much on non-ephemeral DH_RSA/DH_DSS cipher suites. To be honest, I'm not sure whether they're much used. I haven't seen any example of certificate with the necessary DH attributes, and these cipher suites are not in supported lists by OpenSSL or Java 7. DHE/EDH are much more common, and don't require special attributes in the certificate.)

If you use an RSA key exchange algorithm, the client will encrypt the pre-master-key in the client key exchange message; if it's using one of the DH key exchange algorithms, it will send its DH parameters so that client and server can agree on a pre-master-key (in this case, the client will have checked that the server's DH parameters come from the right server by verifying the signature of the server key exchange message sent beforehand). See description of the Client Key Exchange Message:

With this message, the premaster secret is set, either though direct transmission of the RSA-encrypted secret or by the transmission of Diffie-Hellman parameters that will allow each side to agree upon the same premaster secret.

Regarding the other points:

a certificate is only a private-key-signed(encrypted) message that also contains someone else's(e.g., a client ) public key, so a client should use its trusted copy of the server's public key (usually, web browsers include dozens of these certificates in advance), instead of the public key in the server certificate, to verify the server's identity.

Three things happen to verify you're talking to the right server:

1. The handshake itself, if successful, guarantees that you're talking to the server that has the private key for the certificate it has presented in the server certificate message. If using RSA key exchange, this is guaranteed by the fact it's the only one that can decipher what the client sent in the client key exchange message (since it's encrypted with the public key); if using an EDH key exchange, this is guaranteed because the server signed its DH parameters in the server key exchange message, verifiable with this public key.
2. The fact that you can verify the certificate itself. This is rather independent of how TLS works, but it's usually done using a PKI: the client has a pre-set list of trusted CA certificates, the public keys of which can be used to verify the signature in new certificates it doesn't already know about (such as the server certificate). Verifying that signature allows the client to bind that public key to an identifier (Subject DN and/or alt. name). This gives you the identity of the server to which the client is talking.
3. The host name verification: it's not good enough to know that you're talking to someone who's presented you a genuine ID that's valid for them, you also need to check that the name matches the server you intended to connect to.

When I said "The public key in the server certificate isn't used to verify the server's identity itself", I meant that the public key wasn't used to verify points 2 and 3. Point 1 guarantees you that you're talking to the server that has the private key matching the certificate it presented, but it doesn't tell you who this is. Verifying the identity of the server is is up to point 2, so as to be able to bind an identifier to that key/cert.

Answer 2

In a PPK algorithm like RSA, you have two different communication channels. Information encrypted using the public key is only readable to the possessor of the private key, and information encrypted using the private key is only readable to the possessor of the public key.

In fact, the selection of which half of the pair is "public" is completely arbitrary.

Now, in practice this doesn't much matter; the whole world has access to the public key, so encrypting something with the private part wouldn't do anything to secure it. But you can use this for authentication: since only one holder has the private key, if a message is validly encrypted using it, then the private-key holder must have been the author.

That's why your book doesn't say that the private key is used for encryption: because it's used for integrity, not for confidentiality, as any message sealed using it would be readable to anyone possessing the non-secret public half. While the integrity validation mechanism is technically encryption (it is encipherment using modular exponentiation), it would be confusing to mention this in a foundations-of-cryptography context as it's not what people think when they hear "encryption" - they think "privacy".

Answer 3

I had the same restlessness, since its difficult to ask without falling in technical holes. Quick, easy answer: regardless of the method details, yes, you could reverse engineer and decrypt from one key (is just math at the end) but because of the complexity of the operation, using a long value (you may use ie. a 4096-bit value) would make the reverse operation timeless, so you had to use either a machine so big that still doesnt exists or wait indefinitely, making it inviable... until someone comes up with a quick method to do it.

may want to take a look: http://www.usna.edu/CS/si110arch/si110AY12F/lec/l29/lec.html