In the previous article, we introduced sticky package and half package and their general solutions. Today, we will focus on how Netty implements the framing scheme.
<!-- more -->
Decoding core process
Three kinds of decoders, FixedLengthFrameDecoder, DelimiterBasedFrameDecoder, LengthFieldBasedFrameDecoder, have been introduced before. They all inherit from ByteToMessageDecoder, while ByteToMessageDecoder inherits from channelinboundhandler adapter. The core method is channelRead. So let's look at the channelRead method of ByteToMessageDecoder:
@Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { if (msg instanceof ByteBuf) { CodecOutputList out = CodecOutputList.newInstance(); try { // Convert incoming messages to data ByteBuf data = (ByteBuf) msg; // The ultimate goal is to put all the data into the calculation first = cumulation == null; // Put the first data directly if (first) { cumulation = data; } else { // Not the first data to be added cumulation = cumulator.cumulate(ctx.alloc(), cumulation, data); } // Decode callDecode(ctx, cumulation, out); } // The following code is omitted because it is not part of the decoding process }
Let's look at the callDecode method:
protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) { try { while (in.isReadable()) { int outSize = out.size(); if (outSize > 0) { // The following code is omitted, because in the initial state, outSize can only be 0 and cannot enter here } int oldInputLength = in.readableBytes(); // In decode, the remove operation of handler is not performed. // Clean up the data only after the decode is executed. decodeRemovalReentryProtection(ctx, in, out); // Omit the following code because the following is not the decoding process either
Take a look at the decodeRemovalReentryProtection method:
final void decodeRemovalReentryProtection(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception { // Set current state to decoding decodeState = STATE_CALLING_CHILD_DECODE; try { // Decode decode(ctx, in, out); } finally { // Perform the remove operation of the handler boolean removePending = decodeState == STATE_HANDLER_REMOVED_PENDING; decodeState = STATE_INIT; if (removePending) { handlerRemoved(ctx); } } } // Each implementation has its own special decoding method. protected abstract void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception;
From the above process, it can be concluded that before decoding, the data needs to be written into the calculation first, and after decoding, it needs to be removed through the handler.
Specific decoding process
Just mentioned that the decode method is implemented in the subclass, so for the three decoding methods we said, see their implementation one by one.
FixedLengthFrameDecoder
Its source code is:
@Override protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception { Object decoded = decode(ctx, in); if (decoded != null) { out.add(decoded); } } protected Object decode( @SuppressWarnings("UnusedParameters") ChannelHandlerContext ctx, ByteBuf in) throws Exception { // Whether the collected data is less than the fixed length means that it cannot be parsed. if (in.readableBytes() < frameLength) { return null; } else { return in.readRetainedSlice(frameLength); } }
Just like the name of this class, it is decoded with fixed length. Therefore, when setting the decoder, you need to pass in frameLength in the construction mode.
DelimiterBasedFrameDecoder
Its source code is:
@Override protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception { Object decoded = decode(ctx, in); if (decoded != null) { out.add(decoded); } } protected Object decode(ChannelHandlerContext ctx, ByteBuf buffer) throws Exception { // Whether the current separator is a line break (\ nor \ r\n) if (lineBasedDecoder != null) { return lineBasedDecoder.decode(ctx, buffer); } // Try all delimiters and choose the delimiter which yields the shortest frame. int minFrameLength = Integer.MAX_VALUE; ByteBuf minDelim = null; // One segmentation for other separators for (ByteBuf delim: delimiters) { int frameLength = indexOf(buffer, delim); if (frameLength >= 0 && frameLength < minFrameLength) { minFrameLength = frameLength; minDelim = delim; } } // The following code is omitted
According to its name, the separator is the core. It divides the delimiters into two categories, only the newline delimiter (\ nor \ r\n) and others. Therefore, it should be noted that you can define multiple separators, which are supported.
LengthFieldBasedFrameDecoder
This class is more complex. If you look at the method directly, you can easily see the confusion. Therefore, I'm going to take a look at its private variables in combination with the explanation on the class.
2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field, the length field represents the length of the whole message
Let's give another twist to the previous example. The only difference from the previous example is that the length field represents the length of the whole message instead of the message body, just like the third example. We have to count the length of HDR1 and Length into lengthAdjustment. Please note that we don't need to take the length of HDR2 into account because the length field already includes the whole header length.
* BEFORE DECODE (16 bytes) AFTER DECODE (13 bytes) * +------+--------+------+----------------+ +------+----------------+ * | HDR1 | Length | HDR2 | Actual Content |----->| HDR2 | Actual Content | * | 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" | | 0xFE | "HELLO, WORLD" | * +------+--------+------+----------------+ +------+----------------+
lengthFieldOffset: this field indicates that the Length field starts from the first few bytes. In the above example, the Length field starts from the first byte (HDR1 is the 0 byte), so the value is 0.
lengthFieldLength: this field represents the number of bytes occupied by the Length field. In the example above, the Length field takes two bytes, so the value is 2.
Length adjustment: this field represents the distance from the end of the length field to the beginning of the Actual Content. In the above example, because the meaning of the length field is the whole message (including HDR1, length, HDR2, Actual Content, generally length refers to only Actual Content), from the end of length to the beginning of the real content (the beginning of HDR1), it is equivalent to reducing 3 bytes, so it is - 3.
Initial bytes to strip: you need to skip a few bytes from the end of the Length field when presenting. In the above example, because the real content starts from HDR1 and the final content displayed starts from HDR2, there are three bytes left in the middle, so the value is 3.
This kind of decoding method is more complex. Interested students can try to analyze it by themselves.
summary
This article mainly explains the three ways of framing in Netty with the source code in Netty. I'm sure you have a different understanding.
If you are interested, you can visit my blog or pay attention to my public number and headline number. Maybe there will be unexpected surprises.
