SSL renegotiation with full duplex socket communication

Question

I have a very simple client-server with one blocking socket doing full-duplex communication. I've enabled SSL/TLS to the application. The model is that of a typical producer-consumer. The client produces the data, sends it to the server and the server processes them. The only catch is that, once in a while the server sends data back to the client which the client handles accordingly. Below is a very simple pseudo code of the application:

  1 Client:
  2 -------
  3 while (true)
  4 {
  5         if (poll(pollin, timeout=0) || 0 < SSL_pending(ssl))
  6         {
  7                 SSL_read();
  8                 // Handle WANT_READ or WANT_WRITE appropriately.
  9                 // If no error, handle the received control message.
 10         }
 11         // produce data.
 12         while (!poll(pollout))
 13                 ; // Wait until the pipe is ready for a send().
 14         SSL_write();
 15         // Handle WANT_READ or WANT_WRITE appropriately.
 16         if (time to renegotiate)
 17                 SSL_renegotiate(ssl);
 18 }
 19
 20 Server:
 21 -------
 22 while (true)
 23 {
 24         if (poll(pollin, timeout=1s) || 0 < SSL_pending(ssl))
 25         {
 26                 SSL_read();
 27                 // Handle WANT_READ or WANT_WRITE appropriately.
 28                 // If no error, consume data.
 29         }
 30         if (control message needs to be sent)
 31         {
 32                 while (!poll(pollout))
 33                         ; // Wait until the pipe is ready for a send().
 34                 SSL_write();
 35                 // Handle WANT_READ or WANT_WRITE appropriately.
 36         }
 37 }

The trouble happens when, for testing purposes, I force SSL renegotiation (lines 16-17). The session starts nice and easy, but after a while, I get the following errors:

Client:
-------
error:140940F5:SSL routines:SSL3_READ_BYTES:unexpected record

Server:
-------
error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpected message

Turns out, around the same time that the client initiates a renegotiation (line 14), the server ends up sending application data to the client (line 34). The client as part of the renegotiation process receives this application data and bombs with a "unexpected record" error. Similarly, when the server does the subsequent receive (line 26), it ends up receiving a renegotiation data when it was expecting application data.

What am I doing wrong? How should I handle/test SSL renegotiations with a full-duplex channel. Note that, there are no threads involved. It's a simple single threaded model with reads/writes happening on either end of the socket.

UPDATE : To verify that there is nothing wrong with the application that I have written, I could even reproduce this quite comfortably with OpenSSL's s_client and s_server implementations. I started a s_server and once the s_client got connected to the server, I programmatically send a bunch of application data from the server to the client and a bunch of 'R' (renegotiation requests) from the client to the server. Eventually, they both fail in exactly the same manner as described above.

s_client:

RENEGOTIATING
4840:error:140940F5:SSL routines:SSL3_READ_BYTES:unexpected record:s3_pkt.c:1258:

s_server:

Read BLOCK
ERROR
4838:error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpected message:s3_pkt.c:1108:SSL alert number 10
4838:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:1185:

UPDATE 2: Ok. As suggested by David, I reworked the test application to use non-blocking sockets and always do SSL_read and SSL_write first and do the select based on what they return and I still get the same errors during renegotiations (SSL_write ends up getting application data from the other side in the midst of renegotiation). The question is, at any point in time, if SSL_read returns WANT_READ, can I assume it is because there is nothing in the pipe and go ahead with SSL_write since I have something to write? If not, that's probably why I end up with errors. Either that, or I am doing the renegotiation all wrong. Note, if SSL_read returns WANT_WRITE, I always do a select and call SSL_read again.

David Schwartz David Schwartz · Accepted Answer · 2013-09-10T20:53:32

You're trying to "look through" the SSL black box. This is a huge mistake.

     if (poll(pollin, timeout=0) || 0 < SSL_pending(ssl))
     {
             SSL_read();

You're making the assumption that in order for SSL_read to make forward progress, it needs to read data from the socket. This is an assumption that can be false. For example, if a renegotiation is in progress, the SSL engine may need to send data next, not read data.

     while (!poll(pollout))
             ; // Wait until the pipe is ready for a send().
     SSL_write();

How do you know the SSL engine wants to write data to the pipe? Did it give you a WANT_WRITE indication? If not, maybe it needs to read renegotiation data in order to send.

To use SSL in non-blocking mode, just attempt the operation you want to do. If you want to read decrypted data, call SSL_read. If you want to send encrypted data, call SSL_write. Only call poll if the SSL engine tells you to, with a WANT_READ or WANT_WRITE indication.

Update:: You have a "half of each" hybrid between blocking and non-blocking approaches. This cannot possibly work. The problem is simple: Until you call SSL_read, you don't know whether or not it needs to read from the socket. If you call poll first, you will block even if SSL_read does not need to read from the socket. If you call SSL_read first, it will block if it does need to read from the socket. SSL_pending won't help you. If SSL_read needs to write to the socket to make forward progress, SSL_pending will return zero, but calling poll will block forever.

You have two sane choices:

Blocking. Leave the sockets set blocking. Just call SSL_read when you want to read and SSL_write when you want to write. They will block. Blocking sockets can block, that's how they work.
Non-blocking. Set the sockets non-blocking. Just call SSL_read when you want to read and SSL_write when you want to write. They will not block. If you get a WANT_READ indication, poll in the read direction. If you get a WANT_WRITE indication, poll in the write direction. Note that it is perfectly normal for SSL_read to return WANT_WRITE, and then you poll in the write direction. Similarly, SSL_write can return WANT_READ, and then you poll in the read direction.

Your code would (mostly) work if the implementation of SSL_read was basically, "read some data then decrypt it" and SSL_write was "encrypt some data and send it". The problem is, these functions actually run a sophisticated state machine that reads and writes to the socket as needed and ultimately causes the effect of giving you decrypted data or encrypting your data and sending it.

SSL renegotiation with full duplex socket communication

2 Answers