How to decode buffer in Netty with unknown length?

Question

I've searched for many examples in Netty project and they seem simple and great. However, decoders have one thing in common: they read some header first and then they know in advance how many bytes to read.

I have to implement more complex protocol. It's a proprietary binary protocol construced like this:

+--------+-----------+------------+-------+------------+-------+-----------+
| header | msg_count | msg_type_1 | msg_1 | msg_type_n | msg_n | msg_count |
+--------+-----------+------------+-------+------------+-------+-----------+
 11 bytes    1 byte      1 byte     bytes     1 byte     bytes     1 byte

Header contains serial number. Then there is message count and n messages. These messages varies in length. At the end of transmission there is message count reprised again and it's a sign of a completed transmission.

Messages can have different types. Some types are fixed width, but some types varies, e. g.:

msg type 0x01: fixed width of 4 bytes
msg type 0x02: first byte - message length of m, then m bytes

I don't have a clue how to implement Netty decoder for such protocol. Main problem here is that there is no clear sign of an end until I actually parse the whole buffer - messages are parsed fine and message count is reprised at the end.

Is there any mechanism in Netty that can help me solve this case?

how do you know when a message ends and a new message starts? — jtahlborn
@jtahlborn after reprised msg_count at the end, there can be another header incoming right away. There is no delimiter. — Tomasz Kalkosiński
no, how do you know the boundary between msg_1 and msg_type_n? — jtahlborn
@jtahlborn I've improved my post with message types and lengths — Tomasz Kalkosiński

jtahlborn jtahlborn · Accepted Answer · 2016-03-21T23:36:49

At the very least, you should be able to implement a ReplayingDecoder. Maybe not the most efficient implementation, but probably the simplest.

How to decode buffer in Netty with unknown length?

1 Answers