summary
When communicating between our client and server, we must abide by certain protocols. These protocols may have been designed in advance, such as Http protocol, or we can customize them.
Last We explained some codecs provided by Netty, so that we can design some protocols ourselves.
This article mainly explains the Http protocol in Netty and our custom protocol.
1.HttpServerCodec
The codec of the server complies with the HTTP protocol. First look at the following classes
First, let's take a look at the class structure of HttpServerCodec. The inherited CombinedChannelDuplexHandler actually combines its two generic tasks. I won't say more
The two key decoding and encoding classes are inherited as follows:
Generally, those ending in Codec can be decoded or encoded. Decoder means decoding and encoder means encoding. Decoding is a subclass of ChannelInboundHandlerAdapter (i.e. inbound handler), which is specially used to listen to the data sent by the client and decode it first. Encoder is a subclass of channeloutbooundhandleradapter (i.e. outbound handler), that is, the data to be sent shall be encoded first and then sent to the corresponding client.
We only need to add the following code to the server (the complete code is omitted).
//Codec for HTTP protocol ch.pipeline().addLast(new HttpServerCodec());
Demo decoding:
You may wonder, what kind of information will Codec decode from the client? ByteBuf, String? The specific details are analyzed below
Add the following code to the server, start the server, open the browser, enter localhost: XXXX (bound port), and view the output
The results are as follows (there are too many request headers and lines sent by some browsers to be displayed)
You can see that I only sent a request once in the browser, but printed it twice. In fact, HttpServerCodec parses our request into two parts. The first part is HttpRequest, which contains the request header and request line. The second part is HttpContent, which represents the request body (even for get requests, there will be a request body, at most there is no content).
According to this logic, we may need to distinguish between the request header and the request body in the decoded channelRead in the future:
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//Print out msg class information
// System.out.println(msg.getClass());
if(msg instanceof HttpRequest){ //The parent interface of DefaultHttpRequest is HttpRequest
//Execute the code logic of the request header
}else if (msg instanceof HttpContent){//Similarly, the parent class can be compared, which is more general
//Execute the code logic of the request body
}
}But this seems too troublesome. If I only care about one of them and don't want to make a lot of if...else judgments, we can simplify it in another way, that is, simplechannelinboundhandler < T >:
The generic type of SimpleChannelInboundHandler is the message type he cares about. If it is not the specified message type, the handler will be skipped. You can see that the msg type in the rewritten channelRead0 is the specified generic type
ch.pipeline().addLast(new SimpleChannelInboundHandler<HttpRequest>() {//We only care about the request head here
@Override
protected void channelRead0(ChannelHandlerContext channelHandlerContext, HttpRequest msg) throws Exception {
//Business logic
}
}Response client:
Of course, our server needs to reply to the client after receiving the client's request. According to the above instructions, we know that we need to respond to a response object that also conforms to the Http protocol to the client. When writing back the same data, it will be encoded by HttpServerCodec. Please refer to the following classes first
DefaultFullHttpResponse: the classes provided by Netty to respond to the client are as follows
The HTTP protocol version class reference is as follows:
Status code class reference (very many):
demonstration:
bootstrap.childHandler(new ChannelInitializer<NioSocketChannel>() {
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler());
//Codec for HTTP protocol,
ch.pipeline().addLast(new HttpServerCodec());
//The generic type of SimpleChannelInboundHandler is the message format he cares about. If it is not the specified format, the handler will be skipped
ch.pipeline().addLast(new SimpleChannelInboundHandler<HttpRequest>() {//We only care about the request head here
@Override
protected void channelRead0(ChannelHandlerContext channelHandlerContext, HttpRequest msg) throws Exception {
log.debug("The request method is{}",msg.method());//The request method is GET
log.debug("Request path is{}",msg.uri());//The requested resource path is/
log.debug("Request header is{}",msg.headers());//The request header is defaulthttpheaders [host: localhost: 8081, connection: keep alive...]
//Returns a response to the client
//The response object provided by netty. The response version specified here is the same as the version at the time of request, and the response code is 200
DefaultFullHttpResponse response=new DefaultFullHttpResponse(msg.protocolVersion(),HttpResponseStatus.OK);
//The content method writes the return content for the above response. The return value is ByteBuf. See the source code for details
//You can also put the ByteBuf of the message body in the construction method when constructing the object
response.content().writeBytes("<h1>I am Netty</h1>".getBytes());
//Writeback response
channelHandlerContext.writeAndFlush(response);
}
});
}
});As a result, netty has encoded the response and written it back to the corresponding channel, and the client has also received the response
However, another problem is that the browser has been loading in circles. This is because the server did not tell it that I have responded. The browser thought there was still data, so it will be loading all the time and waiting for reception.
Solution: add a field content length in the response header For the length of the response we want to send, change the code to
Field name information in various response headers:
The browser correctly received the response information.
In addition, the browser may automatically send icon requests to the server
In the future, we need to return different resources of the client according to different request URIs.
2. Custom agreement
Previously, we explained the common HTTP protocols. If you want to design a set of protocols suitable for your business to enhance efficiency and reduce waste.
2.1 user defined agreement elements
To customize the protocol, you need to consider the following:
- Magic Number: it is used to determine whether it is an invalid packet at the - time. The little partner who knows about the jvm may know that the magic number of the class file is CAFE BABY. If you use the java command to execute the file, if the first four bytes (magic number) of the class file are not the agreed CAFE BABY, the jvm will not execute the calss file.
- Version number: it can support protocol upgrade. For example, after your protocol is upgraded, several fields are added. If you want to use the old protocol, you must distinguish and identify it according to the version number of the protocol.
- Serialization algorithm: which serialization and deserialization methods are used for the message body, such as json, jdk, etc
- Instruction type: login, registration, single chat, group chat? Business related
- Request sequence number: for duplex communication, it provides asynchronous capability and marks each different request message
- Text length
- Message body: the message body is like all kinds of complex data transmitted from the background to the front end. We need to use the serialization algorithm to parse the data, otherwise the data will be disordered when receiving. Therefore, the front and rear end interaction generally adopts json format for encoding and decoding
Interested partners can check the HttpMessage class to see how Netty designs the Http protocol (too complex).
We should compare HttpServerCodec to learn the custom protocol, so that even if we forget the code one day, we can find the answer in the official solution and learn more about HttpServerCodec.
Text start
1. First, establish a Message abstract class and its various implementation classes. It is the bearer of our custom protocol. We send messages in the unit of Message. What is written here is relatively simple, and many things will be expanded in the future.
public abstract class Message implements Serializable {//implements Serializable
private int sequenceId;
private int messageType;
//Possible instruction types, expressed in numbers
public static final int LoginRequestMessage=0;//Login request message
public static final int LoginResponseMessage=1;//Login reply message
public static final int ChatRequestMessage=2;//Chat request message
public static final int ChatResponseMessage=3;//Chat reply message
//It is implemented by the specific subclass Message, and each subclass corresponds to the above code
public abstract int getMessageType();
//Omit methods such as get set
}The implementation class only lists one of them, which will be expanded slowly in the future:
public class LoginRequestMessage extends Message{
private String username;
private String password;
private String nickname;
@Override
public int getMessageType() {
return LoginRequestMessage;
}
public LoginRequestMessage(String username, String password, String nickname) {
this.username = username;
this.password = password;
this.nickname = nickname;
}
//toString...
}
2. The implementation of user-defined encoding and decoding function inherits ByteToMessageCodec. Like the CombinedChannelDuplexHandler of HttpServerCodec, this class integrates HttpRequestDecoder and HttpResponseEncoder for encoding and decoding functions
/**
* Implementation of custom codec function
* ByteToMessageCodec Its function is to convert ByteBuf and custom messages, so generics write our custom message class,
* HttpServerCodec It uses its own HttpMessage class
*/
public class MessageCodec extends ByteToMessageCodec<Message> {
//Rewrite encoding and decoding functions
@Override
//Code. Look at the method parameters. Out is the ByteBuf that needs to be passed out,
//We need to put msg into out and send it out according to our specified protocol format
protected void encode(ChannelHandlerContext ctx, Message msg, ByteBuf out) throws Exception {
//1) Magic number, let's write a BBQ here! It's magic number. It occupies 4 bytes in total
out.writeBytes(new byte[]{'B', 'B', 'Q', '!'});
//2) . version number, one byte is enough. Write it here as 1
out.writeByte(1);
//3) . serialization algorithm, which uses one byte to represent the serialization method, such as JDK - > 0; json->1
out.writeByte(0);
//4) . one byte instruction type. Each subclass of Message encapsulates the type of this Message, which can be retrieved directly
out.writeByte(msg.getMessageType());
//5) . request sequence number, 4 bytes
out.writeInt(msg.getSequenceId());
//6) . body, which converts the object into a binary byte array. Here, JDK serialization is adopted
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(msg);
byte[] bytes = baos.toByteArray();
//7) . write in the length, accounting for 4 bytes
out.writeInt(bytes.length);
//Fill 1 byte so that the header information is 16 bytes, that is, the integer power of 2
out.writeByte(0xff);
//8) . write content
out.writeBytes(bytes);
}
@Override
//Decoding is how to decode the code. out is the parsed data set, and multiple pieces of data may be parsed
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
int magicNum = in.readInt();
byte version = in.readByte();
byte serialType = in.readByte();
byte messageType = in.readByte();
int sequenceId = in.readInt();
int length = in.readInt();
in.readByte();//Fill field
byte[] bytes = new byte[length];
in.readBytes(bytes, 0, length);
Message message=null;
//Deserialize data
if (serialType == 0) {
ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
message= (Message) ois.readObject();
}else if (serialType==1){
//Other anti sequence methods
}
System.out.println("Data decoded as:\n magic number:"+magicNum+" Version number:"+version+" Serialization algorithm:"+serialType+" Instruction type:"+
messageType+" Request serial number:"+sequenceId+" length:"+length);
System.out.println("text:"+message);
//Save out and send the message to the next Handler
out.add(message);
}
}3. Test work
Coding test:
Add the log and our newly created codec to the initialization Handler of our server, and then send a message casually (don't ask me why I send a login request in the server, because I only have this message subclass for the time being)
bootstrap.childHandler(new ChannelInitializer<NioSocketChannel>() {
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler());
//Add our custom codec
ch.pipeline().addLast(new MessageCodec());
//
ch.writeAndFlush(new LoginRequestMessage("java","netty","jvm"));
}
});When a client connects, the server executes the code logic inside and sends a Message. The log is as follows:
We can see that this data is encoded into the desired protocol format and sent to the client. The client only needs to decode it with the decoder we have written.
Decoding test:
Add our custom codec to the client, listen for read events, print the Message after receiving the Message, and print its class information to see whether it is List type or Message type
handler(new ChannelInitializer<NioSocketChannel>() {
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler())
.addLast(new MessageCodec())
.addLast(new ChannelInboundHandlerAdapter(){
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println("I got the message"+msg);
System.out.println("The message type is"+msg.getClass());
}
});
}
});The results are as follows:
The client receives the message perfectly and decodes it successfully. It is verified that the parsed type is LoginRequestMessage, not List. Instead, the parsed data in the List set should be sent one by one iteratively.
Problem analysis:
But there is still a hidden danger in the code Last We learned the sticky package and semi package problem. The custom protocol analysis here can only solve the sticky package. Half a package can't solve it, because the code is dead, how many bytes to read. If the number of bytes is not enough, an error will occur when converting to pojo or retrieving data. Therefore, first ensure that a complete message is obtained through lenghtfieldbased.
The following is a demonstration of the half packet problem. Set a limit on the cache size of the received data at the receiving end
Retest results:
For the first time, we only got 60 bytes. The following error is reported, that is, when we read ButeBuf, we crossed the boundary and didn't have so many bytes to read to us
That is, my server sends a long Message now. After all the messages are serialized, they are sent to the client. The sliding window of the client may not be enough, and only half of them may be received. At this time, the decoder of the client will parse them, and an error will occur.
We use the LengthFieldBasedFrameDecoder decoder in the previous article to solve the problem. When the decoder finds that the length of the actually received message is less than the length of the message in the frame, it will not immediately give it to the next handler. It will continue to receive it. After the received message reaches the specified length, it will receive a complete message and send it to the next handler, The complete message will also be sent to our custom decoder, and there will be no error at this time
Its construction method is repeated (in order): the maximum frame length (according to our requirements and the maximum value that the length field can represent), the offset of the length field in our frame (the protocol length field we designed earlier starts from bit 11), the number of bytes occupied by the length field (4 bytes), and the data of 1 byte after adjusting the length field is the data content (because we have one byte to fill in later), intercept the first 0 bytes (because we don't need to intercept messages, and the complete data needs to be sent to our custom parser).
The results are as follows. It can be seen that the log is received twice when printing, but the LengthFieldBasedFrameDecoder combines the two data and then decodes it without problems
Another small problem analysis:
When we create a Handler, a channel is used to establish a connection, which is a new series of codecs. Can we share the Handler?
In fact, there are some problems with sharing. For example, the LengthFieldBasedFrameDecoder is used to solve the problem of sticky package and half package. If its instance object is placed outside and used by multiple channel s together, one thread has put data in it. Before the data is put together, the next thread has stored data in it. There must be a multi-threaded concurrency problem. However A Handler like LoggingHandler will not have a problem, because it only does a print job without recording status and other operations. It is thread safe.
If you see a Handler with this annotation, it has no thread safety problem and can be shared with an object.
Now I want to add this annotation to my custom codec and share this object, so I need to consider whether I have thread safety problems. Logically, I think my codec has no thread safety problems, because this class does not have some shared data, but only some local data.
I'll add a @ Sharable to play, and change the place where MessageCodec is used to the following shared object
Start and find that an error is reported successfully. It says that Shared annotation is not allowed in this class.
Take a closer look at the above error. It occurs in the < init > method of the parent class, that is, the constructor. The parent class is constructed as follows
In short, if we add Sharable annotation to this subclass, we will report an error. Netty's design is quite exquisite
Now if we want to add this annotation, we can only bypass the ByteToMessageCodec parent class and change the father.
public class MessageCodec2 extends MessageToMessageCodec<ByteBuf,Message> {
@Override
protected void encode(ChannelHandlerContext ctx, Message msg, List<Object> out) throws Exception {
}
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
}
}MessageToMessageCodec means that I got a complete Message from the beginning. There is no sticky packet or semi packet problem, that is, corresponding to the situation where we used the LengthFieldBasedFrameDecoder to solve this problem, so it can be used safely without state problems.
Its two generics are the type of ButeBuf passed from the LengthFieldBasedFrameDecoder to be decoded as Message, so these are the two types. The other difference is the small difference of parameters. We need to create ButeBuf ourselves and put it into the List set in the parameters. The principle is the same.
We can use @ Sharable annotation happily
Be sure to use it with LengthFieldBasedFrameDecoder.
The result is no problem. It can be shared and used, saving a little memory ^ ^ ^.