Winsock using send() and FD_WRITE

Question

I'm new to Winsock. I am writing an HTTP server and I am aiming to send a large file by reading and sending in chunks. To optimize this process, I attempt to use a nonblocking socket where I read the next chunk from the disk while I send the current one.

My problem now is that I get an FD_WRITE message even when it seems the buffer should be full and my function deletes the associated data from memory prematurely. I believe this causes my responses contain less data than they should send() stops sending prematurely and the client (which is a well-known one) receives about 70% of the data. When I use blocking sockets it works fine, it's just longer.

I tried using Wget, a simple HTTP client, to get a better idea of what's going on. From what I can see, the datastream thins out when I detect WSAEWOULDBLOCK errors when checking for errors after using send(). It looks like during those sends, not all the data gets sent.

When I set the sleep time to over 2000ms after checking for the FD_WRITE message, everything works as it basically comes down to using a blocking socket. I also tried setting times around 100-200ms, but those fail as well. As it is, the condition checking for FD_WRITE always returns valid before entering the loop.

WSAEVENT event = WSACreateEvent();
const int sendBufferSize = 1024 * 64;
int connectionSpeed = 5; //estimated, in MBytes/s
const int sleepTime = sendBufferSize / (connectionSpeed * 1024 * 1024);
size = 0;
const int bufSize = 1024 * 1024 * 35;
int lowerByteIndex = 0;
int upperByteIndex = bufSize;
size = bufSize;
int totalSIZE = 0;
unsigned char* p;
unsigned char* pt;
clock_t t = std::clock();

p = getFileBytesC(resolveLocalPath(path), size, lowerByteIndex, upperByteIndex);
lowerByteIndex += bufSize;
upperByteIndex += bufSize;
totalSIZE += size;
while (upperByteIndex <= fileSize + bufSize)
{
    int ret = send(socket, (char*)p, size, 0);
    pt = getFileBytesC(resolveLocalPath(path), size, lowerByteIndex, upperByteIndex);
    totalSIZE += size;
    lowerByteIndex += bufSize;
    upperByteIndex += bufSize;
    if (ret == SOCKET_ERROR && WSAGetLastError() == WSAEWOULDBLOCK)
    {
        while (SOCKET_ERROR == WSAEventSelect(socket, event, FD_WRITE))
        {
            Sleep(50);
        }
    }
    Sleep(sleepTime); //should be around 30-50ms. Wait for the buffer to be empty
    delete[] p;
    p = pt;
    std::cout << std::endl << (std::clock() - t) / (double)CLOCKS_PER_SEC;
}
send(socket, (char*)p, size, 0);
delete[] p;
std::cout << std::endl << (std::clock() - t) / (double)CLOCKS_PER_SEC ;
if (totalSIZE == fileSize) std::cout << std::endl << "finished transfer. UBI=" << upperByteIndex;
else
{
    std::cout << std::endl << "ERROR: finished transfer\r\nBytes read=" << totalSIZE;
}
Sleep(2000);
closeSocket(socket);

user207421 user207421 · Accepted Answer · 2015-10-02T04:57:54

You can't write correct non-blocking send() code without storing the value returned in a variable. It is the number of bytes actually sent. You can't assume the entire buffer was sent in non-blocking mode.
If send() returns -1 with WSAGetLastError() == WSAEWOULDBLOCK or whatever it is, then is the time to call WSASelect(), or WSAEVENTSelect() if you must, but with a timeout. Otherwise, i.e. if it returns a positive count, you should just advance your offset and decrement your length by the amount sent and repeat until there is nothing left to send.
Your sleeps are just literally a waste of time.

But I would question the whole approach. Sending on a blocking-mode socket is asynchronous anyway. There is little to be gained by your present approach. Just read the file in chunks and send the chunks in blocking mode.

Winsock using send() and FD_WRITE

2 Answers