Complete custom message protocol design with Netty

Keywords: Java

Message Protocol

The concept of a message protocol sounds very tall, but what does a message protocol really mean?

Message protocol refers to how the data (messages) transmitted by both sides of the communication express their description.

For example, HTTP protocol, when a browser opens a web page, it first establishes a connection with the server, then sends a request (request mainly includes some request headers, request types, request URL s, request messages, etc.). After the server receives the request, it first parses the current request through the established rules, and then responds to the data flow that returns the response to the client. These established rules are also known as message protocols.

Custom Message Protocol

So what does a custom message typically include? For example:

  • Version number,
  • Message type, request/response, GET, POST, DELETE
  • Message Length
  • The body of the message
  • Serialization algorithm
  • ...

How do I customize a message protocol???

Protocol Definition

statusCode | sessionId | reqType | contentLength | content

We customized a protocol above where statusCode represents the status code, sessionId, reqType, contentLength is the request header information, and content is the message content.
The following is the definition of a custom messaging protocol through the Netty framework, as well as the process of data exchange between client and server using the current protocol.

Implement custom messaging protocol through Netty

1. Project catalogue


First, create a Maven project where netty-msg-agreement represents a custom messaging protocol, netty-msg-client represents a client, netty-msg-server represents a server, and client, server modules depend on and agree modules.

netty-msg-protocol pom.xml dependency: add netty-all dependency and lombox dependency.

<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-all</artifactId>
    <version>4.1.69.Final</version>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.18.14</version>
    <scope>provided</scope>
</dependency>

2. netty-msg-agreement


MessageRecord represents the message record through which the client server delivers messages. During message delivery, the client side encodes and decodes messages through MessageRecordDecoder and MessageRecordEncoder.

The main code is as follows:

2.1 MessageRecord

/**
 * Message Logging
 * Status Code|Request Header (Session id | Request mode | Request body length)|Message content
 */
@Data
@AllArgsConstructor
@NoArgsConstructor
public class MessageRecord {
    // Status code: 4 bytes
    private int statusCode;
    // Message Request Header
    private Header header;
    // Message Content
    private Object body;
}

2.1 Header

/**
 * Request Header: Custom Design
 */
@Data
@AllArgsConstructor
@NoArgsConstructor
public class Header {
    // Session id: 8 bytes
    private long sessionId;
    // Request method: 1 byte
    private byte reqType;
    // Request body length: 4 bytes
    private int contentLength;
}

2.3 MessageRecordEncoder-Encoder

/**
 * Encoder
 */
public class MessageRecordEncoder extends MessageToByteEncoder<MessageRecord> {
    @Override
    protected void encode(ChannelHandlerContext channelHandlerContext, MessageRecord messageRecord, ByteBuf byteBuf) throws Exception {
        System.out.println(">>>>>>>>>>>Message Encoding start>>>>>>>>>>>");
        // Status line
        byteBuf.writeInt(messageRecord.getStatusCode());

        // Request Header
        Header header = messageRecord.getHeader();
        byteBuf.writeLong(header.getSessionId());
        byteBuf.writeByte(header.getReqType());

        Object body = messageRecord.getBody();
        if (body != null){// Message content is not empty
            ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
            ObjectOutputStream out = new ObjectOutputStream(outputStream);
            out.writeObject(body);
            byte[] bytes = outputStream.toByteArray();
            // Message Length
            byteBuf.writeInt(bytes.length);
            // Message Content
            byteBuf.writeBytes(bytes);
        }else {// Message content is empty
            byteBuf.writeInt(0);
        }
        // Write and refresh
        channelHandlerContext.writeAndFlush(messageRecord);
        System.out.println(">>>>>>>>>>>Message Encoding end>>>>>>>>>>>");
    }
}

2.4 MessageRecordDecoder-Decoder

/**
 * Decoder
 */
public class MessageRecordDecoder extends ByteToMessageDecoder {
    @Override
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> out) throws Exception {
        System.out.println(">>>>>>>>>>>Message decoding start>>>>>>>>>>>");
        // Get data from byteBuf
        int statusCode = byteBuf.readInt();// Get 4 bytes
        Header header = new Header();
        header.setSessionId(byteBuf.readLong());// Get 8 bytes
        header.setReqType(byteBuf.readByte());
        header.setContentLength(byteBuf.readInt());// Get 4 bytes
        if (header.getContentLength() > 0){// Message length greater than 0
            MessageRecord messageRecord = new MessageRecord();
            messageRecord.setStatusCode(statusCode);
            messageRecord.setHeader(header);
            // Get message body
            byte[] bytes = new byte[header.getContentLength()];
            byteBuf.readBytes(bytes);// Read message body contents to bytes
            ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);// java Native deserialization tool
            ObjectInputStream objectInputStream = new ObjectInputStream(inputStream);
            messageRecord.setBody(objectInputStream.readObject());
            System.out.println("Messages received are:" + messageRecord);

            // Note: Message transfer objects need to be added to `List<Object> out', if not added, the message content will not be processed by the service side receive
            out.add(messageRecord);
        }else {
            System.out.println("Message content is empty, not parsed");
        }
        System.out.println(">>>>>>>>>>>Message decoding end>>>>>>>>>>>");
    }
}

2.5 Enumerations

/**
 * Request Mode Enumeration
 */
public enum RequestTypeEnums {
    GET((byte) 1),
    POST((byte) 2),
    DELETE((byte) 3),
    ;

    private byte reqType;

    RequestTypeEnums(byte reqType) {
        this.reqType = reqType;
    }

    public byte getReqType() {
        return this.reqType;
    }
}

/**
 * Status row enumeration
 */
public enum StatusCodeEnums {
    SUCCESS(0, "Success"),
    FAIL(-1, "fail"),
    EXCEPTION(-2, "abnormal"),
    ;

    private int statusCode;
    private String desc;

    StatusCodeEnums(int statusCode, String desc) {
        this.statusCode = statusCode;
        this.desc = desc;
    }

    public int getStatusCode() {
        return statusCode;
    }

    public String getDesc() {
        return this.desc;
    }
}

3. netty-msg-server


ProtocolServer is the service startup class and waits for the client to connect. ServerFinalHeaders is a service-side message processing class.

Add a netty-msg-agreement dependency to netty-msg-server pom.xml.

<dependency>
   <groupId>org.example</groupId>
    <artifactId>netty-msg-agreement</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

3.1 ProtocolServer

/**
 * Server
 */
public class ProtocolServer {

    public static void main(String[] args) {
        EventLoopGroup boss = new NioEventLoopGroup();
        EventLoopGroup work = new NioEventLoopGroup(Runtime.getRuntime().availableProcessors() * 2);
        ServerBootstrap bootstrap = new ServerBootstrap();

        bootstrap.group(boss, work).channel(NioServerSocketChannel.class)
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel socketChannel) throws Exception {
                        socketChannel.pipeline()
                                .addLast(new LengthFieldBasedFrameDecoder(1024 * 1024,
                                        13, // statusCode + sessionId + reqType
                                        4, // Request Body Length
                                        0,
                                        0))
                                .addLast(new MessageRecordDecoder())
                                .addLast(new MessageRecordEncoder())
                                .addLast(new ServerFinalHeaders());
                    }
                });

        try {
            int port = 8080;
            ChannelFuture future = bootstrap.bind(port).sync();
            System.out.println(">>>>>>>>>>ProtocolServer start success>>>>>>>>>>" + port);
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            boss.shutdownGracefully();
            work.shutdownGracefully();
        }
    }
}

3.2 ServerFinalHeaders

public class ServerFinalHeaders extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        MessageRecord messageRecord = (MessageRecord) msg;
        System.out.println("Server Messages received are:" + messageRecord);

        // Write the message back to the client
        messageRecord.setBody("server data: " + messageRecord.getBody());
        ctx.channel().writeAndFlush(messageRecord);
        super.channelRead(ctx, msg);
    }
}

4. netty-msg-client


The ProtocolClient starts the class for the client, connects to the service-side ProtocolServer through the class, and delivers messages. ClientFinalHeaders is a client-side message processing class.

Add a netty-msg-agreement dependency to netty-msg-server pom.xml.

<dependency>
   <groupId>org.example</groupId>
    <artifactId>netty-msg-agreement</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

4.1 ProtocolClient

/**
 * Client
 */
public class ProtocolClient {

    public static void main(String[] args) {
        EventLoopGroup work = new NioEventLoopGroup(Runtime.getRuntime().availableProcessors() * 2);
        Bootstrap bootstrap = new Bootstrap();

        bootstrap.group(work).channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel socketChannel) throws Exception {
                        socketChannel.pipeline()
                                .addLast(new LengthFieldBasedFrameDecoder(1024 * 1024,
                                        13, // statusCode + sessionId + reqType
                                        4, // Request Body Length
                                        0,
                                        0))
                                .addLast(new MessageRecordDecoder())
                                .addLast(new MessageRecordEncoder())
                                .addLast(new ClientFinalHeaders());
                    }
                });

        try {
            ChannelFuture future = bootstrap.connect(new InetSocketAddress("localhost", 8080)).sync();
            Channel channel = future.channel();
            for (int i = 0; i < 5; i++) {
                MessageRecord msg = new MessageRecord();
                msg.setStatusCode(StatusCodeEnums.SUCCESS.getStatusCode());

                Header header = new Header();
                header.setSessionId(System.currentTimeMillis());
                header.setReqType(RequestTypeEnums.POST.getReqType());
                msg.setHeader(header);

                String body = String.format("No.%s Bar Request Data:%s", i + 1, UUID.randomUUID().toString());
                msg.setBody(body);

                channel.writeAndFlush(msg);
            }
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            work.shutdownGracefully();
        }
    }
}

4.2 ClientFinalHeaders

public class ClientFinalHeaders extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        MessageRecord messageRecord = (MessageRecord) msg;
        System.out.println("Client Messages received are:" + messageRecord);
        super.channelRead(ctx, msg);
    }
}

Posted by explore on Tue, 09 Nov 2021 10:49:14 -0800