2021SC@SDUSC
preface
Starting with this blog, we will introduce several encoders and decoders that have been implemented in netty. In this blog, we will introduce netty's LineBasedFrameDecoder class, which divides messages into different parts based on line breaks.
1, LineBasedFrameDecoder
package io.netty.handler.codec;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.util.ByteProcessor;
import java.util.List;
public class LineBasedFrameDecoder extends ByteToMessageDecoder {
private final int maxLength;
private final boolean failFast;
private final boolean stripDelimiter;
private boolean discarding;
private int discardedBytes;
private int offset;
public LineBasedFrameDecoder(final int maxLength) {
this(maxLength, true, false);
}
public LineBasedFrameDecoder(final int maxLength, final boolean stripDelimiter, final boolean failFast) {
this.maxLength = maxLength;
this.failFast = failFast;
this.stripDelimiter = stripDelimiter;
}
@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
Object decoded = decode(ctx, in);
if (decoded != null) {
out.add(decoded);
}
}
protected Object decode(ChannelHandlerContext ctx, ByteBuf buffer) throws Exception {
final int eol = findEndOfLine(buffer);
if (!discarding) {
if (eol >= 0) {
final ByteBuf frame;
final int length = eol - buffer.readerIndex();
final int delimLength = buffer.getByte(eol) == '\r'? 2 : 1;
if (length > maxLength) {
buffer.readerIndex(eol + delimLength);
fail(ctx, length);
return null;
}
if (stripDelimiter) {
frame = buffer.readRetainedSlice(length);
buffer.skipBytes(delimLength);
} else {
frame = buffer.readRetainedSlice(length + delimLength);
}
return frame;
} else {
final int length = buffer.readableBytes();
if (length > maxLength) {
discardedBytes = length;
buffer.readerIndex(buffer.writerIndex());
discarding = true;
offset = 0;
if (failFast) {
fail(ctx, "over " + discardedBytes);
}
}
return null;
}
} else {
if (eol >= 0) {
final int length = discardedBytes + eol - buffer.readerIndex();
final int delimLength = buffer.getByte(eol) == '\r'? 2 : 1;
buffer.readerIndex(eol + delimLength);
discardedBytes = 0;
discarding = false;
if (!failFast) {
fail(ctx, length);
}
} else {
discardedBytes += buffer.readableBytes();
buffer.readerIndex(buffer.writerIndex());
// We skip everything in the buffer, we need to set the offset to 0 again.
offset = 0;
}
return null;
}
}
private void fail(final ChannelHandlerContext ctx, int length) {
fail(ctx, String.valueOf(length));
}
private void fail(final ChannelHandlerContext ctx, String length) {
ctx.fireExceptionCaught(
new TooLongFrameException(
"frame length (" + length + ") exceeds the allowed maximum (" + maxLength + ')'));
}
private int findEndOfLine(final ByteBuf buffer) {
int totalLength = buffer.readableBytes();
int i = buffer.forEachByte(buffer.readerIndex() + offset, totalLength - offset, ByteProcessor.FIND_LF);
if (i >= 0) {
offset = 0;
if (i > 0 && buffer.getByte(i - 1) == '\r') {
i--;
}
} else {
offset = totalLength;
}
return i;
}
}
2, Analysis
First, the LindeBasedFrameDecoder class inherits from ByteToMessageDecoder. The message accepted by this class is ByteBuf and converts the message to other types of messages. In addition, all decoders that inherit the ByteToMessageDecoder class are not allowed to have the @ Sharable annotation.
In LineBasedFrameDecoder, there are two constructors, namely
public LineBasedFrameDecoder(final int maxLength) public LineBasedFrameDecoder(final int maxLength, final boolean stripDelimiter, final boolean failFast)
After reading the source code, I know that the first constructor calls the second constructor. The meaning of the three attributes will be explained here. maxLength is the maximum message length supported by the decoder. stripDelimiter indicates whether the newline character needs to be removed after parsing the message. failFast is true. When the parsed message length exceeds maxLength, a TooLongFrameException exception will be thrown. Otherwise, an exception will be thrown after all data are read.
As mentioned earlier, the LineBasedFrameDecoder divides the message into different parts according to the newline character. The first is to find the position of the newline character in the message by calling the findEndOfLine() method.
int i = buffer.forEachByte(buffer.readerIndex() + offset, totalLength - offset, ByteProcessor.FIND_LF);
In this method, the newline character is found by traversing the contents of bytebuf. Offset is the offset, which indicates the last scanned position. Because of the call of the decode method, a message is not divided into multiple parts at one time. Therefore, some information needs to be saved. It is precisely because the information needs to be saved that these decoder s cannot be shared.
ByteProcessor.FIND_LF is \ n, so i here is the position in the message. Then, judge whether the previous position of the position is \ r. if yes, i minus 1 returns. Therefore, if this method finds only \ n, it directly returns the position of \ n. if it finds \ r\n, it returns the position of \ r. if not, it returns - 1.
Next, we will analyze the decode() method. In the LineBasedFrameDecoder class, there are two decode methods.
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception protected Object decode(ChannelHandlerContext ctx, ByteBuf buffer) throws Exception
The first decode method overrides the parent class's decode method, while the specific implementation calls the second decode method. Therefore, we will mainly analyze the second decode method.
In this method, the first is to find the position of the newline character. Then, enter different branches according to whether to discard or not. discarding is true, which means that if the data length exceeds maxLength, it will be discarded.
First analyze the non discarded mode, and first judge whether the eol is greater than or equal to 0, that is, whether there is a newline character. Assuming that there is a newline character, first obtain the length of the data intercepted this time. If the length exceeds maxLength, skip the data this time. Otherwise, judge whether to bring a newline character, and return the data this time according to the result. If there is no line break, the read data is all of the data. If the length of the read data exceeds the maximum length, set it to discard mode and skip all data.
Next, analyze the case where discarding is true. Still, first judge whether there is a newline character. Assuming there is a newline character, the length of the data to be read is the length of the previously discarded data plus the length of the part cut from the data, and then discard the data. Here, my understanding is that if a piece of data is divided into two parts due to network reasons during transmission, the total data length should be the sum of the two data lengths. Moreover, since the length of the first data has exceeded the maximum length, both data should be discarded. If only the first data is discarded and the second data is retained, If the data is incorrect and incomplete, it is still wrong. Then set discarding to false.
Finally, if there is no newline in the data, skip all the data.
3, Summary
In this blog, the code of LineBasedFrameDecoder, one of the common decoders in netty, is analyzed. This class is to solve the problem of unpacking and pasting packets. The same data in different packets is reassembled through line feed, or different data in a packet is separated. It is said that in the ftp protocol, the data is separated by adding line breaks at the end.