EmptyByteBuf appears in Decoder chain in Netty server

Question

Netty version: 5.0.0.Alpha2

Here's how I add handlers/decoders to my pipeline. (childHandler)

childHandler(ch ->
    ch.pipeline().addLast(
      new LoggingHandler(LogLevel.TRACE),
      new LengthFieldBasedFrameDecoder(2048, 4, 2, -2, 0),
      new MyDecoder2(),
      new Encoder(),
      handler
    )
)

The next thing I do that I send some garbage to the input of this server. LengthFieldBasedFrameDecoder throws TooLongFrameException, which is expected in this case.

But what happens futher is MyDecoder2.decode() is invoked with EmptyByteBuf as input buffer AFTER LengthFieldBasedFrameDecoder has thrown an exception. Where does this EmptyByteBuf come from if the previous Decoder should return nothing?

I expect MyDecoder2 either to receive ByteBuf with correct framing, processed by LengthFieldBasedFrameDecoder either receive nothing.

 Caused by: java.lang.IndexOutOfBoundsException: null
        at io.netty.buffer.EmptyByteBuf.checkIndex(EmptyByteBuf.java:866) ~[na:0.0]
        at io.netty.buffer.EmptyByteBuf.getBytes(EmptyByteBuf.java:317) ~[na:0.0]
        at spire.broker.proto.MyDecoder2.decode(MyDecoder2.java:27) ~[na:0.0]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeLast(ByteToMessageDecoder.java:371) ~[na:0.0]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:275) ~[na:0.0]

Maybe I do not understand correctly how Netty works with chain of decoders in pipeline.

Moh-Aw Moh-Aw · Accepted Answer · 2015-05-18T08:03:11

Take a look at the code of the channelInactive() method of ByteToMessageDecoder which a LengthFieldBasedFrameDecoder extends:

@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
    RecyclableArrayList out = RecyclableArrayList.newInstance();
    try {
        if (cumulation != null) {
            callDecode(ctx, cumulation, out);
            decodeLast(ctx, cumulation, out);
        } else {
            decodeLast(ctx, Unpooled.EMPTY_BUFFER, out);
        }
    } catch (DecoderException e) {
        throw e;
    } catch (Exception e) {
        throw new DecoderException(e);
    } finally {
        ...
    }
}

Once the channel goes inactive it will call the decode method one last time with an Unpooled.EMPTY_BUFFER. This gives you the chance to clean up if you have made a custom decoder. If you are not interested in that, then you need to override the decodeLast() method to do nothing.

EmptyByteBuf appears in Decoder chain in Netty server

1 Answers