Netty comes with multiple sticky package decoders. Today we introduce LineBasedFrameDecoder, line break decoder.

Line unpacking device

Next, let's take a concrete example to see how the unpacker comes with netty.

This class is called LineBasedFrameDecoder. Based on line delimiter, TA can handle both n and r n types of line delimiters. The core method is inherited decode method.

protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
    Object decoded = decode(ctx, in);
    if (decoded != null) {
        out.add(decoded);
    }
}

netty's own unpackers are like the template above. Let's take a look at decode(ctx, in);

protected Object decode(ChannelHandlerContext ctx, ByteBuf buffer) throws Exception {
    int eol = findEndOfLine(buffer);
    int length;
    int length;
    if (!this.discarding) {
        if (eol >= 0) {
            length = eol - buffer.readerIndex();
            int delimLength = buffer.getByte(eol) == '\r' ? 2 : 1;
            if (length > this.maxLength) {
                buffer.readerIndex(eol + delimLength);
                this.fail(ctx, length);
                return null;
            } else {
                ByteBuf frame;
                if (this.stripDelimiter) {
                    frame = buffer.readRetainedSlice(length);
                    buffer.skipBytes(delimLength);
                } else {
                    frame = buffer.readRetainedSlice(length + delimLength);
                }

                return frame;
            }
        } else {
            length = buffer.readableBytes();
            if (length > this.maxLength) {
                this.discardedBytes = length;
                buffer.readerIndex(buffer.writerIndex());
                this.discarding = true;
                if (this.failFast) {
                    this.fail(ctx, "over " + this.discardedBytes);
                }
            }

            return null;
        }
    } else {
        if (eol >= 0) {
            length = this.discardedBytes + eol - buffer.readerIndex();
            length = buffer.getByte(eol) == '\r' ? 2 : 1;
            buffer.readerIndex(eol + length);
            this.discardedBytes = 0;
            this.discarding = false;
            if (!this.failFast) {
                this.fail(ctx, length);
            }
        } else {
            this.discardedBytes += buffer.readableBytes();
            buffer.readerIndex(buffer.writerIndex());
        }

        return null;
    }
}

ByteProcessor FIND_LF = new IndexOfProcessor((byte) '\n');

private static int findEndOfLine(ByteBuf buffer) {
    int i = buffer.forEachByte(ByteProcessor.FIND_LF);
    if (i > 0 && buffer.getByte(i - 1) == '\r') {
        --i;
    }

    return i;
}

Find the newline position

final int eol = findEndOfLine(buffer);

private static int findEndOfLine(final ByteBuf buffer) {
    int i = buffer.forEachByte(ByteProcessor.FIND_LF);
    if (i > 0 && buffer.getByte(i - 1) == '\r') {
        i--;
    }
    return i;
}

ByteProcessor FIND_LF = new IndexOfProcessor((byte) '\n');

for loop traversal, find the position of the firstn, if the character beforen isr, then return to the position ofr.

Processing of non-discarding mode

Next, netty will determine whether the current unpacking belongs to the discard mode and identify it with a member variable.

private boolean discarding;

The first unpacking is not in discarding mode

Processing of Finding Line Separators in Non-discarding Mode

// 1.Calculate the delimiter and package length
final ByteBuf frame;
final int length = eol - buffer.readerIndex();
final int delimLength = buffer.getByte(eol) == '\r'? 2 : 1;

// Abnormal data discarded
if (length > maxLength) {
    buffer.readerIndex(eol + delimLength);
    fail(ctx, length);
    return null;
}

// Does it include delimiters when fetching packages?
if (stripDelimiter) {
    frame = buffer.readRetainedSlice(length);
    buffer.skipBytes(delimLength);
} else {
    frame = buffer.readRetainedSlice(length + delimLength);
}
return frame;

1. First, create a new frame and calculate the length of the current packet and the length of the separator (because there are two separators)
2. Then determine whether the length of the unpacking is longer than the max Length allowed by the unpacker. This parameter is passed in the constructor. If the maximum length is exceeded, the data is discarded and returned to null.
3. Finally, a complete packet is taken out. If stripDelimiter is specified as false when constructing this unpacker, the parsed package contains a delimiter, which is not included by default.

Processing of no delimiter found in non-discarding mode

No corresponding line separator was found, indicating that the byte container did not have enough data to be spliced into a complete business data package, and went into the following process.

final int length = buffer.readableBytes();
if (length > maxLength) {
    discardedBytes = length;
    buffer.readerIndex(buffer.writerIndex());
    discarding = true;
    if (failFast) {
        fail(ctx, "over " + discardedBytes);
    }
}
return null;

First, get the number of readable bytes in the current byte container. Then, judge whether it has exceeded the maximum allowable length. If not, return null directly. The data in the byte container does not change. Otherwise, you need to enter the discarding mode.

A member variable discardedBytes is used to indicate how much data has been discarded, and then the read pointer of the byte container is moved to the write pointer, which means discarding this part of the data. Setting the member variable discarding to true indicates that the current discarding mode is in discarding mode. If failFast is set, an exception is thrown directly. By default, failFast is false, which means that data is discarded quietly.

discarding mode

If you're in discarding mode when unpacking, there are also two scenarios

Find the line separator in discarding mode

In discarding mode, if a delimiter is found, you can discard everything that precedes the delimiter.

final int length = discardedBytes + eol - buffer.readerIndex();
final int delimLength = buffer.getByte(eol) == '\r'? 2 : 1;
buffer.readerIndex(eol + delimLength);
discardedBytes = 0;
discarding = false;
if (!failFast) {
    fail(ctx, length);
}

After calculating the length of the separator, the data before the separator is discarded directly. Of course, the discarded characters also include the separator. After such a discarding, the normal data packet may follow. The next unpacking will enter the normal unpacking process.

Row separator not found in discarding mode

This situation is relatively simple, because the current discarding mode is still in the discarding mode, and the failure to find the line separator means that the current complete data package has not been discarded, and the data currently read is part of the discarding, so it is discarded directly.

discardedBytes += buffer.readableBytes();
buffer.readerIndex(buffer.writerIndex());

Specific delimiter unpacking

This class is called DelimiterBasedFrameDecoder, which can be passed to TA as a list of delimiters. Packets are split according to the list of delimiters. Readers can analyze the DelimiterBasedFrameDecoder completely according to the idea of line wrapper.