How to find if there is more data to read when response is split across packets but SSL_pending is zero and SSL_get_error returns SSL_ERROR_NONE

3
votes

We have a client application coded in C++ that connects to a Java application server via SSL using openssl. We use non-blocking socket connection mode with option to choose 'select' or 'poll' mechanism for identifying the data packets available to be read.

the client has been working fine for quite a while, but recently we noticed an issue where the response is getting split into multiple packets and SSL_read is returning just one character 'H' with SSL_ERROR_NONE and SSL_pending zero, as a result of which our SSL_read loop is ending with incomplete response. On a subsequent reinvocation of SSL_read [as a hack] we get the remaining response starting with 'TTP 1.1'.

Since SSL_pending=0 and SSL_get_error()=SSL_ERROR_NONE, we do not find any other way to know if there is more data to be read and hence ending the 'SSL_read' loop which just reads one character. 

Here is the relevant pseudo code... 

bool done=false;
int pending=1; bool inited=false;
bool want_read=false; bool want_write=false;
timeval tv;
String response; //internal class for string, legacy            
while(!done) {
if( !inited || want_read || want_write ) {
    try {
        /* Wait a few seconds. */
        if(!inited) {
            tv.tv_sec = timeOutValue.tv_sec;
            tv.tv_usec = timeOutValue.tv_usec;
        } else {
            tv.tv_sec = 0;
            tv.tv_usec = 500000; //hack to reduce the waiting period
        } 
                    #ifdef USE_POLL
            poll the socket
        #else
            call 'select' on the socket
        #endif
        inited=true;
        continue;
    } catch(const XSeption&) {
        close_socket(&sock);
        done=true;
        cout << "Error waiting on select/poll call" << endl;
        break;
    }
}

memset(temp, 0, BUFSIZE);
charsRead = SSL_read(ssl,temp, (sizeof(char)*BUFSIZE));
cout << endl << "charsRead = " << charsRead << endl;    

if(charsRead>0) {
    response.append(temp, charsRead);
}
cout << "Response : " << response << endl;

int sslerror = SSL_get_error(ssl,charsRead);
cout << "SSL_error_code = " << sslerror << endl;

pending=SSL_pending(ssl);
cout << "pending characters in the current SSL buffer : " << endl;

/*
if(charsRead==1) {
    memset(temp, 0, BUFSIZE);
    cout << "read only one character which is odd,hence reading again" << endl;
    charsRead = SSL_read(ssl,temp, (sizeof(char)*BUFSIZE));
    cout << endl << "charsRead in second read = " << charsRead << endl;
    if(charsRead>0) {
        response.append(temp, charsRead);
    }
    cout << "Second Response : " << response << endl;

    sslerror = SSL_get_error(ssl,charsRead);
    cout << "SSL_error_code = " << sslerror << endl;

    pending=SSL_pending(ssl);
    cout << "pending characters in the current SSL buffer : " << endl;
}
*/

switch(sslerror){

    case SSL_ERROR_NONE:
        cout << "No SSL Error" << endl;
        done=true; //ideally, we should mark the process completed here; but if mark this completed here, then we are getting only 1 character 'H'
        break;
    case SSL_ERROR_WANT_READ:
        cout << "In SSL_ERROR_WANT_READ" << endl;
        //SSLread Still pending
        want_read=true;
        //continue;
        break;

    case SSL_ERROR_WANT_WRITE:
        cout << "In SSL_ERROR_WANT_WRITE" << endl;
        //SSLread Still pending
        want_write=true;
        //continue;
        break;

    case SSL_ERROR_SSL:
        done=true;
        cout << "encountered SSL INTERNAL ERROR" << endl;
        close_socket(&sock);
        break;

    case SSL_ERROR_SYSCALL:
        done=true;
        cout << "encountered ERROR SYSCALL" << endl;
        close_socket(&sock);
        break;

    case SSL_ERROR_ZERO_RETURN:
        done=true;
        cout << "encountered SSL ZERO RETURN" << endl;
        close_socket(&sock);
        break;

    default:
        done=true;
        cout << "encountered default error" << endl;
        close_socket(&sock);
        break;
    } //end of switch  
} //end of while  

cout << "Final Response : " << response << endl;

So, how can I identify that the response is complete or is pending when SSL_pending returns zero and the SSL_get_error is SSL_ERROR_NONE and i do not know how long [number of bytes/characters the response can be?

Are my expectations wrong? Why is the SSL_read returning a single character the first time even though we provide a larger buffer?

Any help in this regard is highly appreciated...

UPDATE:

while(!done) {
currentTime = getTime();
tval.tv_sec = timeOutValue.tv_sec - (currentTime - beginTime);
tval.tv_usec = timeOutValue.tv_usec;
if ( tval.tv_sec <= 0 ) //the allotted time for processing this request has elapsed
{
    //do not close the socket or SSL session since this is just a timeout issue
    throw Exception::TIMEOUT);
}

#ifdef USE_POLL
    fds.fd = sock;
    fds.events = POLLIN;
#else
    FD_ZERO(&rset);
    FD_SET(sock, &rset);
#endif

if(!inited || want_read || want_write) {
    timeval tv;
    /*
    When we first enter this method or for NON-SSL requests, we would wait till the SELECT call returns a ready file-descriptor but in the case of a SSL requests processing the response message , we just issue SELECT with 0(zero) or very little timeout since SSL_read is giving us a common error code for actual need to check at the socket (SELECT/POLL) and the need to call SSL_read again, if we can find a way to differentiate the need to call SELECT/POLL Vs invoke SSL_read, then this if-else construct needs to be removed and we can then use the complete remaining time for timeout parameter in SELECT/POLL call even for SSL requests
    */
    if(!inited ) {
        tv.tv_sec=tval.tv_sec;
        tv.tv_usec=tval.tv_usec;
    } else {
        tv.tv_sec=0;
        tv.tv_usec=1000;
    }
    try {
        #ifdef USE_POLL
            poll_call(&fds, 1, &tv);
        #else
            select_call(sock+1, &rset, NULL, NULL, &tv);
        #endif
    } catch(const Exception&) {
        /*
        do not close the socket or throw exception; the socket will be closed only if an error occurs or if the server itself the closed the connection
        */
    }
    inited=true;
    want_read=false;
    want_write=false;
}

memset(temp, 0, BUFSIZE);

charsRead = openSSL->SSLread(temp, (sizeof(char)*BUFSIZE));
if(charsRead>0) {
    response.append(temp, charsRead);
    done=is_response_complete(response);
} else {
    int sslerror=openSSL->SSLgetErrorCode(charsRead);
    switch(sslerror){

        case SSL_ERROR_NONE:
            break;

        case SSL_ERROR_WANT_READ:
            want_read=true;
            break;

        case SSL_ERROR_WANT_WRITE:
            want_write=true;
            break;

        case SSL_ERROR_SSL:
            close(openSSL, Exception::SSL_CONNECTION_PROBLEM, 
                    ErrorDescription(sslerror));
            break;

        case SSL_ERROR_SYSCALL:
            close(openSSL,Exception::SERVER_CONNECTION_PROBLEM,
                    ErrorDescription(sslerror));
            break;

        case SSL_ERROR_ZERO_RETURN:
        default:
            close(openSSL,Exception::SSL_CONNECTION_PROBLEM,
                        ErrorDescription(sslerror));
            break;
    } //end of switch
} //end off ssl_error check
}
3
c++copenssl
Is the BIO blocking or non-blocking? – alk

3 Answers

3
votes

I have see this behavior as well. The proper way to handle it is your application should expect any amount of data at any time... it's a "stream" of data, not packetized request/responses. The only way to identify a "complete request/response" is to parse it at the application level. If you don't have a complete one, buffer it and wait for more data. The transport mechanism can't and doesn't tell you that... it's just saying "hey I have data".

3
votes

With OpenSSL, the normal process is inverted;

  • with normal sockets, you select() until it says the sockets are readable or writable.

  • With SSL sockets, you read and write until they return SSL_ERROR_WANT_READ/WRITE. Only then do you call select().

"Since SSL_pending=0 and SSL_get_error()=SSL_ERROR_NONE, we do not find any other way to know if there is more data to be read"

So you should assume there is always more data to be read until you get SSL_ERROR_WANT_READ. Instead of telling you there is more data to be read, OpenSSL uses WANT_READ/WANT_WRITE to tell you there is no more data to be read.

I am seeing the same behaviour in my app; When I do an SSL_read, I get 1 byte:

G

As I did not get a WANT_READ/WRITE, I do SSL_read again and I get the rest of the data

ET /index.html HTTP/1.1
Host: etc etc

I continue to do this until I get WANT_READ/WRITE, and then I wait on select() because OpenSSL said to do so.

1
votes

As you do read via a non-blocking BIO, it is not guaranteed that you'll receive all data expected+ with only one call to SSL_read().

So it might be necessary to call SSL_read() again several times up until all data you are expecting+ had been read, or the connection had been closed, or an error occurred.

To do so you could place the call to SSL_read() inside the while-loop doing the select().

(+) How much data actually is expected to be transfered is unknown by the transport layer. This either shall be known to both (sender and receiver) or negotiating needs be part of the application level protocol.