Does OpenSSL *require* the application to send/read non-application data?

Question

Let's suppose there is a client and a server, and the server sends a rekey request, or any other non-application data that might or might not be important (speaking generally here). I want to know if I am bound to sending and reading data for OpenSSL so that I can make my life and code simpler. To let you understand what I mean, I prepared these examples:

Suppose we succeeded with the TLS handshake and are now ready to exchange data. The client sends a HTTP request to the server, with Connection header set to close, meaning it won't send any more requests. Now OpenSSL would like to do something under the hood, and thus asking for sending. Maybe we did shutdown(SHUT_WR) and can't do that, or the kernel buffer is permanently full.

Suppose a game which exchanges lots of data - server sends information about the game to the clients, and clients send information about mouse and keyboard to the server. Normally, during the data exchanges, OpenSSL will have more than enough opportunities to do anything it would like, so actually there shouldn't be any need to explicitly listen to it's WANTS_something requests. Now, what if a client becomes AFK, so that it doesn't send any keyboard and mouse information, but OpenSSL wanted to do something that requires us to send something (if there could be such an event).

Both of the above scenarios have a shared idea in mind. The question is: can OpenSSL keep sending/receiving user data without sending/receiving TLS data? Or is it a requirement for the application to fulfill OpenSSL's requests to maintain a healthy connection/encryption?

If it is required to fulfill it's requests, there is another problem I might have with coding that. Suppose it asks for a read to process non-application data, but first in queue is actually application-data. I always want to read application data only when the application asks to do so, otherwise it is quite pointless, because we buffer data that we might not actually use and the peer will keep sending data, because the TCP congestion window will not shrink (since we read the data). Or, OpenSSL will buffer the application data internally, which is the same scenario, because eventually I will need to flush the data out of it to process more non-application data. Is there any flag or a way of processing ONLY non-application data? Is there any other cool solution to this?

And yes, I do understand that it is required for non-application data to be sent during the handshake and shutdown.

Answer 1

Apart from the initial handshake and the shutdown sequence, TLS only sends messages when the application requests it. If you haven't done anything special and you get a WANT_READ or WANT_WRITE error from OpenSSL (or the similar indication from any TLS library), it means that you need to continue reading or writing to either finish sending your own data, or receive data from your peer.

Either side may send a rekey request, but it will only do that because it needs that before it can transmit more data.

Either side may send a heartbeat if the corresponding extension is enabled. If you don't want heartbeats, you don't have to enable them. If you've indicated your acceptance of heartbeats, you don't need to read them immediately, but if you delay reading them for too long, your peer may drop the connection. However, once again, this is an optional feature, and if you don't want it, just don't enable it.

TLS is a strict streaming protocol, so if there's any non-application data, you have to read it before you can read subsequent application data. You can't just skip non-application data.

I always want to read application data only when the application asks to do so

That's not how networked applications usually work. Usually you can't predict when your peer is going to send data, so you would not block indefinitely without reading application data. This works similarly whether the data has gone through TLS or not: you select (or poll or equivalent) on the network socket and other file descriptors, and if there's data available on the network socket, you read from it. The difference with TLS is that you read from the SSL socket instead of reading from the network socket directly; OpenSSL will take care of reading from the network socket.

otherwise it is quite pointless, because we buffer data that we might not actually use and the peer will keep sending data, because the TCP congestion window will not shrink (since we read the data).

If you don't want to buffer data indefinitely while the application isn't ready to process it, just don't keep reading it. Whether the data is directly coming from TCP or going through TLS doesn't make any difference.

Is there any flag or a way of processing ONLY non-application data?

There's no way to read only non-application data since it's necessary to read the data from the TCP socket in order to know whether it's application data or not. But why does it matter?

Is there any other cool solution to this?

I'm unable to determine whether my answer is “cool”, or for that matter whether it's a “solution”, because I don't see the problem. As far as I can tell, you're imagining a poorly defined problem that doesn't actually exist.