Correct usage of Netty LengthFieldPrepender and LengthFieldBasedFrameDecoder

Question

I have an application using Netty 3.6.6. I use netty to send random packet data from a client to a server.

The sender uses this pipeline:

    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
            @Override
            public ChannelPipeline getPipeline() {
                return Channels.pipeline(
                        new LengthFieldPrepender(4) );
            }
    });

I wrap my packet data with a ChannelBuffer and I want the handler to prepend 4 byte length so that receiver can know where a packet begins and ends.

And the receiver uses:

        bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
            public ChannelPipeline getPipeline() throws Exception {
                    return Channels.pipeline(
                            new NettyReceiveHandler(listener));
            }
        });


    public class NettyReceiveHandler extends LengthFieldBasedFrameDecoder {
        @Override
        protected Object decode(ChannelHandlerContext ctx, Channel channel,
                ChannelBuffer buffer) throws Exception {
            ChannelBuffer decodedBuffer = (ChannelBuffer) super.decode(ctx, channel, buffer);
            if(decodedBuffer == null)
            {
                return null; // not ready yet
            }
            listener.handleObject(decodedBuffer);   
            return null;    // no upstream
        }

        public NettyReceiveHandler(NettyRecvListener listener) {
            super(THREEMiB, 0, 4, 0, 4); 
            this.listener = listener;       
        }

        private static final Logger logger = Logger.getLogger(
                NettyReceiveHandler.class.getName());

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) {
            logger.severe("Unexpected exception from downstream." + e.getCause());
            e.getChannel().close();
        }

        private final NettyRecvListener listener;
        private static final int THREEMiB = 3*1024*1024;
    }

Everything seems to work as expected. Sender sends randomly sized ChannelBuffers one after another and the receiver receives the same ChannelBuffers in the same order

Now my questions are:

1.When I write a large ChannelBuffer to the channel, it is broken down into several writes For example, I see this in the logs for my tests

WARNING: The pipeline contains no upstream handlers; discarding: [id: 0xa1b55c95, /127.0.0.1:52359 => localhost/127.0.0.1:9991] WRITTEN_AMOUNT: 131071

So lets assume I am sending two 1MiB buffers(bufferA and bufferB) and each write is roughly broken down into 8 writes - bufferA1, bufferA2, ... and bufferB1, bufferB2, ... If bufferA4 fails to write( due to too much load or receiver incorrectly spending too much time in the callback ), is the framing going to break? That is bufferA is incorrectly comprised of bufferA1, bufferA2, bufferA3, bufferA5, bufferA6, bufferA7, bufferA8, bufferB1 on the receiver side.

Are there any guarantees about not breaking framing.

2.Is always returning null in NettyReceiveHandler::decode the correct approach as I dont have any upstream handlers?

Thanks for any help.

johnstlr johnstlr · Accepted Answer · 2013-07-08T10:28:07

Assuming a TCP connection then your large writes won't be dropped. Netty will queue the data until it can be written. Netty may signal that it's write queue is full by raising an INTEREST_OPS event but it won't stop you queuing more data.

Personally I would handle this a bit differently. I wouldn't extend LengthFieldBasedFrameDecoder but instead create a pipeline with 2 handlers - a standard LengthFieldBasedFrameDecoder and a NettyReceiveHandler. Then all you have to do is override messageReceived and call your listener with e.getMessage() (or whatever you've called the Message event parameter). There's no messing around calling LengthFieldBasedFrameDecoder's decode method, or worrying about whether you need to return null. Just handle the message.

Correct usage of Netty LengthFieldPrepender and LengthFieldBasedFrameDecoder

1 Answers