Sending TCP packets via Netty, Netty is dividing the data into different packets?

Question

I'm currently using Netty 4.0.7.Final to write a server that receives images (size: ~10k) over TCP. I modified Netty's sample echo client handler to just read a file into bytes, and send it over to my Netty server.

public EchoClientHandler(int firstMessageSize) throws IOException {
    File image = new File("google.jpeg");
    byte[] imageBytes = FileUtils.readFileToByteArray(image);
    byte[] bytes = Base64.encodeBase64(imageBytes);
    String base64 = new String(bytes);
    //System.out.println("base64="+ base64);
    firstMessage = Unpooled.copiedBuffer(base64, CharsetUtil.UTF_8);
}

My test image is 9k, and I can see the whole image is being sent via Netty logging

io.netty.handler.logging.LoggingHandler logMessage
INFO: [id: 0x132baef0, /127.0.0.1:49710 => localhost/127.0.0.1:2112] WRITE(11964B)

However, when the Netty server receives the message, it seems to be dividing the message into two packets, the first packet is 1024 bytes, and the second one is 10940 bytes, which adds up to 1024+10940 = 11964 bytes (total size of the image)

2013-08-24 22:56:33,700 [nioEventLoopGroup-3-1] INFO  MessageDecoder - capacity = 1024
2013-08-24 22:56:33,700 [nioEventLoopGroup-3-1] INFO  MessageDecoder - readable bytes = 1024
2013-08-24 22:56:33,709 [nioEventLoopGroup-3-1] INFO  MessageDecoder - capacity = 16384
2013-08-24 22:56:33,710 [nioEventLoopGroup-3-1] INFO  MessageDecoder - readable bytes = 10940

Here's what my decoder looks like (although I doubt the decoder has anything to do with it, it looks like Netty is handling this before it even reaches the decoder)

public class MessageDecoder extends ByteToMessageDecoder {

private static final Logger LOGGER = LoggerFactory.getLogger(MessageDecoder.class);

@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
    // Convert to String first
    LOGGER.info("capacity = " + in.capacity());
    LOGGER.info("readable bytes = " + in.readableBytes());
    String rawString = in.readBytes(in.readableBytes()).toString(CharsetUtil.UTF_8);    
    LOGGER.info("Received base64 String={}", rawString);
}

I also tried large number of files, it looks Netty is always dividing a message into packets of 1024 bytes + whatever size for the rest of the file??

I'm wondering why Netty is doing this ? And is there a way to just get the complete packet in one-go?

Thanks so much.

I do not think Netty is doing this. This has to do with how tcp/ip works. It depends on what routers your message is going through. For example MTU(maximum transmission unit) is about 1400 bytes on any WLAN. — Sobvan
thanks a lot, I was able to resolve using a DelimiterBasedFrameDecoder (comes with Netty) and specified the maximum frame size — littlejedi

Dev Dev · Accepted Answer · 2013-08-24T15:50:03

If you'd like to abstract fragmentation from your handler you need to frame your messages. This can be done easily by making use of the LengthFieldPrepender when sending and the LengthFieldBasedFrameDecoder when receiving. This ensures that your message decoder only sees a byte buffer representing a full message.

Note that your frame handlers should be first in your ChannelPipeline except when you are also using an SSL handler and/or compression handler in which case SSL should be first, followed by compression, followed by the frame handlers. Being first in the pipeline means that a handler will be first to handle and inbound event and last to handle an outbound event.

Sending TCP packets via Netty, Netty is dividing the data into different packets?

1 Answers