Data Transfer Protocol Design

3
votes

I am writing a protocol to transfer gigabytes of data over a network using TCP, to try to teach myself a little bit about programming on protocols. I am unsure of how to design this transfer protocol, in order to transfer the data in the fastest and most efficient way.

I am using Qt on windows.

At the moment, my design of my application protocol (the part to transfer the data) is as follows:

  1. First shoot the login details.
  2. Write the first data packet (into the socket) of 4 kilobytes, and then wait for the server to confirm it has got the packet.
  3. When the server confirms receiving the data packet (by writing int "1"), write the next 4 kilobytes.
  4. When all data has been transferred, send the md5sum of the data transferred to the server.
  5. If the server confirms again with an int 8, data transfer completes.

At the moment, I am not able to get speeds higher than 166KB/sec on the same computer when transferring over 127.0.0.1. I have been trying to read other protocol designs, but there is hardly any documentation on data transfer protocols that one can write for their application.

Is the protocol design that I've posted wrong or suffering from some serious issues? Should the protocol wait for each packet to be confirmed by the server or should I write it continuously?

1
c++qtsocketstcpprotocols
TCP already handles acknowledge (ACK), checksums, and usually, adaptive algorithms for payload sizes. So everything you're doing seems redundant. Still, 166KB/s seems extremely slow on localhost. – Brett Hale
So I should constantly write into the SOCKET without waiting for the server to send a confirmation ? – user1066991
Absolutely. The TCP/IP stack will take care of sending, acknowledging, retrying, ordering, etc. You need only worry about return values from the socket read/write calls. – Brett Hale
Will the order also be the same as the order of the write ? e.g. I write packet 1-100 in ascending order, will the packets be received in the same ascending order ? Does TCP guarantee that ? – user1066991
yes, TCP is reliable stream protocol. It will guarantee in order delivery of your data. The data is received in the same order as you send it. Good understanding of TCP will help you all your life: en.wikipedia.org/wiki/Transmission_Control_Protocol – Ahmad Mushtaq

1 Answers

0
votes

First, I would recommend spending some time on reading about TCP, and about Sliding Window Protocol.

I think there are 2 reasons why your implementation is so slow: first, you wait for acknowledgement of each packet - very slow, you should use sliding window. Second, you use MD5 checksumming. There is nothing wrong with that, but TCP already implements some basic checksumming, and MD5 implementation you use can be very slow.

And finally, typical way to find out why something works very slow is to use profiling.