Sending Large Base64 String over TCP Socket

Question

I am trying to send a Base64 encoded image from TCP client using GO and TCP server in C++.

Here is the code snippet for C++ Receiver

std::string recieve(int bufferSize=1024,const char *eom_flag = "<EOF>"){
char buffer[bufferSize];
std::string output;
int iResult;
char *eom;
do{
    iResult = recv(client, buffer, sizeof(buffer), 0);
    //If End OF MESSAGE flag is found.
    eom = strstr(buffer,eom_flag);
    //If socket is waiting , do dot append the json, keep on waiting.
    if(iResult == 0){
        continue;
    }
    output+=buffer;
    //Erase null character, if exist.
    output.erase(std::find(output.begin(), output.end(), '\0'), output.end());
        //is socket connection is broken or end of message is reached.
}while(iResult > -1 and eom == NULL);
//Trim <EOF>
std::size_t eom_pos = output.rfind(eom_flag);
return output.substr(0,eom_pos);}

Idea is to receive the message until End of Message is found, thereafter continue to listen for another message on the same TCP connection.

Golang TCP client code snippet.

//Making connection
connection, _ := net.Dial("tcp", "localhost"+":"+PortNumber)
if _, err := fmt.Fprintf(connection, B64img+"<EOF>"); err != nil {
            log.Println(err)
            panic(err)
        }

Tried approaches:

Increasing the buffer size in the C++ receiver.
Removing the null character from the end of the string in the C++ receiver.

Observations:

Length of string sent by the client is fixed, while the length of the string after receive function is larger and random. Example: Go client string length is 25243. For the same string, length after receive when i run send and receive in the loop is 25243, 26743, 53092, 41389, 42849.
On Saving the received string in a file, I see <0x7f> <0x02> character in the string.

I am using winsock2.h for c++ socket.

recv does not necessarily give back as many bytes as will fit in the buffer - the function returns the number of bytes actually returned as its result. You should account for that where you currently do output+=buffer;. — 500 - Internal Server Error
char buffer[bufferSize]; -- This is not valid C++. Use std::vector<char> buffer(bufferSize); — PaulMcKenzie
see stackoverflow.com/questions/4840535/c-recv-problem you may have the same problem because you write the information on the same buffer. (you are never sure to have 100% of the message into the buffer when the function returns, so you have to add it to a "full message buffer") — Philippe Thomassigny
You may also have another problem: since the message is never totally into the buffer, you may have a truncated "<EOF>" into the buffer so you will never find it (C++ is faster that network so you may hay a function return every 2-3 characters ;) — Philippe Thomassigny
Another issue: following the recv call, if (iResult == 0) { continue; } is wrong. The only time you will get a return value of zero is if/when your peer has disconnected. By using continue, you'll then just repeat the recv, which will simply return 0 again, resulting in an infinite loop. — Gil Hamilton

user253751 user253751 · Accepted Answer · 2019-10-21T15:38:38

You are treating the received data as a C string - a sequence of bytes ending with a 0 byte - which is not correct.

recv receives some bytes and puts them in buffer. Let's say it received 200 bytes.

You then do strstr(buffer,eom_flag);. strstr doesn't know that 200 bytes were received. strstr starts from the beginning of the buffer, and keeps looking until it finds either , or a 0 byte. There is a chance that it might find a in the other 824 bytes of the buffer, even though you didn't receive one.

Then you do output += buffer;. This also treats the buffer as if it ends with a 0 byte. This will look through the whole buffer (not just the first 200 bytes) to find a 0 byte. It will then add everything up to that point into output. Again, it might find a 0 byte in the last 824 bytes of the buffer, and add too much data. It might not find a 0 byte in the buffer at all, and then it will keep on adding extra data from other variables that are stored next to buffer in memory. Or it might find a 0 byte in the first 200 bytes, and stop there (but only if you sent a 0 byte).

What you should do is pay attention to the number of bytes received (which is iResult) and add that many bytes to the output. You could use:

output.insert(output.end(), buffer, buffer+iResult);

Also (as Phillipe Thomassigny has pointed out in a comment), the "" might not be received all at once. You might receive "" separately. You should check whether output has an "" instead of checking whether buffer has an "". (The performance implications of this are left as an exercise to the reader)

By the way, this line doesn't do anything at the moment:

output.erase(std::find(output.begin(), output.end(), '\0'), output.end());

because '\0' never gets added to output, because with output += buffer;, a '\0' tells it where to stop adding.

Sending Large Base64 String over TCP Socket

1 Answers