Preface
Now there are many examples of using Netty to build RPC framework on the Internet. Why do I write an article here to discuss it? I know very well that I may not have written them so well. There are two reasons why we should write about it:
- First, because after learning Netty, we need to practice constantly to better grasp the use of Netty. Obviously, it is a good practice to implement RPC framework based on Netty.
- Second, because there are many RPC frameworks on the market, such as Dubbo, which are Netty at the bottom of the framework communication, so through this example, we can also better experience the design of RPC.
Next, I will explain how to implement a simple RPC framework based on Netty from the following points:
- What is RPC?
- What aspects should we pay attention to in implementing RPC framework?
- How to use Netty?
<!--more-->
What is RPC?
RPC, Remote Procedure Call, can call remote services like local calls. It's a way of communication between processes. The concept must be clear to everyone. You can see it written by Shen Jian, 58. RPC article Later, I realized that we could actually think about RPC in a different way, that is, starting from local calls, and then deriving RPC calls.
1. Local function call
Local functions are common to us, such as the following examples:
public String sayHello(String name) { return "hello, " + name; }
We just need to pass in a parameter and call the sayHello method to get an output, that is, input parameter - > method body - > output. The input parameter, the output parameter and the method body are all in the same process space. This is the local function call.
2. Socket communication
Is there any way to communicate between different processes? The caller is in process A and needs to call method A, but method A is in process B.
The easiest way to think of is to use Socket communication, using Socket can complete cross-process calls, we need to agree on a process communication protocol to carry out parameters, call functions, parameters. Writing Socket should know that Socket is a relatively primitive way, we need to pay more attention to some details, such as parameters and functions need to be converted into byte streams for network transmission, that is, serialization operation, and then deserialization; using socket for underlying communication, code programming is more error-prone.
If a caller needs to pay attention to so many questions, it is undoubtedly a disaster. So is there any simple way that our callers do not need to pay attention to the details, so that the caller can call the local function as well as just call in the parameters, call the method, and wait for the return result?
3. RPC framework
RPC framework is used to solve the above problems. It enables callers to call remote services like calling local functions. The underlying communication details are transparent to callers. It shields all kinds of complexity and gives callers the ultimate experience.
What aspects of RPC calls need to be addressed
As mentioned earlier, the RPC framework allows callers to call remote services as if they were calling local functions. So how do you do that?
When in use, the caller calls the local function directly and passes in the corresponding parameters. It does not need to deal with other details. As for the communication details, it is handed over to the RPC framework for implementation. Actually, RPC framework adopts the way of proxy class, specifically dynamic proxy. It creates new class dynamically at runtime, that is, proxy class, and realizes communication details in this class, such as parameter serialization.
Of course, not only is serialization, but we also need to agree on a protocol format for communication between the two parties, specifying the protocol format, such as the data type of the request parameters, the parameters of the request, the method name of the request, etc., so that the network transmission can be carried out after serialization according to the format, and then the server receives the request object and decodes it in the specified format, so that the server knows the specific tone. By which method, what parameters will be passed?
Just mentioned network transmission, an important part of RPC framework is network transmission. Services are deployed on different hosts. How to efficiently carry out network transmission, as far as possible without losing packets, and ensure the complete and accurate fast transmission of data? In fact, it's using Netty, our protagonist today, as a high-performance network communication framework, which is adequate for our task.
So much has been said before, which points should be focused on in the next RPC framework?
- Agent (dynamic agent)
- Communication protocol
- serialize
- network transmission
Of course, a good RPC framework needs to pay attention to more than the above points, but this article aims to make a simple RPC framework, understanding the key points above is enough.
Implementation of RPC Framework Based on Netty
Finally, the main point of this paper is to use Netty to implement RPC one by one according to several key points (proxy, serialization, protocol, encoding and decoding) that need to be paid attention to.
1. Protocol
Firstly, we need to determine the protocol format, request object and response object of both sides of communication.
Request object:
@Data @ToString public class RpcRequest { /** * ID of Request Object */ private String requestId; /** * Class name */ private String className; /** * Method name */ private String methodName; /** * Parameter type */ private Class<?>[] parameterTypes; /** * Participation */ private Object[] parameters; }
- The ID of the request object is used by the client to verify that the server request and response match.
Response object:
@Data public class RpcResponse { /** * Response ID */ private String requestId; /** * error message */ private String error; /** * Return results */ private Object result; }
2. serialization
There are many serialization protocols on the market, such as jdk, protobuf, kyro, Hessian of Google, etc. As long as the serialization method of JDK is not selected (because its performance is too poor, the stream generated by serialization is too large), other methods can be used. For convenience, JSON is chosen as the serialization protocol, and fast JSON is used as the JSON framework.
To facilitate subsequent expansion, the serialization interface is defined:
public interface Serializer { /** * java Object to binary * * @param object * @return */ byte[] serialize(Object object) throws IOException; /** * Binary conversion to java objects * * @param clazz * @param bytes * @param <T> * @return */ <T> T deserialize(Class<T> clazz, byte[] bytes) throws IOException; }
Because we adopt the JSON approach, we define the implementation class of JSONSerializer:
public class JSONSerializer implements Serializer{ @Override public byte[] serialize(Object object) { return JSON.toJSONBytes(object); } @Override public <T> T deserialize(Class<T> clazz, byte[] bytes) { return JSON.parseObject(bytes, clazz); } }
If other serialization methods are to be used later, the serialization interface can be implemented by itself.
3. Codec
After the protocol format and serialization method are agreed, we also need codec. The encoder converts the request object to a format suitable for transmission (usually byte stream), and the corresponding decoder is to convert the network byte stream to the message format of the application.
Encoder implementation:
public class RpcEncoder extends MessageToByteEncoder { private Class<?> clazz; private Serializer serializer; public RpcEncoder(Class<?> clazz, Serializer serializer) { this.clazz = clazz; this.serializer = serializer; } @Override protected void encode(ChannelHandlerContext channelHandlerContext, Object msg, ByteBuf byteBuf) throws Exception { if (clazz != null && clazz.isInstance(msg)) { byte[] bytes = serializer.serialize(msg); byteBuf.writeInt(bytes.length); byteBuf.writeBytes(bytes); } } }
Decoder implementation:
public class RpcDecoder extends ByteToMessageDecoder { private Class<?> clazz; private Serializer serializer; public RpcDecoder(Class<?> clazz, Serializer serializer) { this.clazz = clazz; this.serializer = serializer; } @Override protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception { //Because before encoding, write a Int type, 4 bytes to indicate the length. if (byteBuf.readableBytes() < 4) { return; } //Mark the current reading position byteBuf.markReaderIndex(); int dataLength = byteBuf.readInt(); if (byteBuf.readableBytes() < dataLength) { byteBuf.resetReaderIndex(); return; } byte[] data = new byte[dataLength]; //Read the data in byteBuf into the data byte array byteBuf.readBytes(data); Object obj = serializer.deserialize(clazz, data); list.add(obj); } }
4. Netty client
Let's look at how Netty client is implemented, that is, how to use Netty to open the client.
In fact, friends familiar with Netty should know that we need to pay attention to the following points:
- Write a startup method to specify that the transmission uses Channel
- Specify ChannelHandler to read and write data in network transmission
- Add codec
- Adding Failure Retry Mechanism
- Adding methods to send request messages
Let's look at the specific implementation code:
@Slf4j public class NettyClient { private EventLoopGroup eventLoopGroup; private Channel channel; private ClientHandler clientHandler; private String host; private Integer port; private static final int MAX_RETRY = 5; public NettyClient(String host, Integer port) { this.host = host; this.port = port; } public void connect() { clientHandler = new ClientHandler(); eventLoopGroup = new NioEventLoopGroup(); //Startup class Bootstrap bootstrap = new Bootstrap(); bootstrap.group(eventLoopGroup) //Channel for specified transmission .channel(NioSocketChannel.class) .option(ChannelOption.SO_KEEPALIVE, true) .option(ChannelOption.TCP_NODELAY, true) .option(ChannelOption.CONNECT_TIMEOUT_MILLIS,5000) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ChannelPipeline pipeline = ch.pipeline(); //Add encoder pipeline.addLast(new RpcEncoder(RpcRequest.class, new JSONSerializer())); //Add Decoder pipeline.addLast(new RpcDecoder(RpcResponse.class, new JSONSerializer())); //Request Processing Class pipeline.addLast(clientHandler); } }); connect(bootstrap, host, port, MAX_RETRY); } /** * Failure reconnection mechanism, refer to Netty's entry actual gold digging Brochure * * @param bootstrap * @param host * @param port * @param retry */ private void connect(Bootstrap bootstrap, String host, int port, int retry) { ChannelFuture channelFuture = bootstrap.connect(host, port).addListener(future -> { if (future.isSuccess()) { log.info("Successful connection to server"); } else if (retry == 0) { log.error("The number of retries has been exhausted and the connection has been abandoned"); } else { //The number of reconnections: int order = (MAX_RETRY - retry) + 1; //The interval between reconnections int delay = 1 << order; log.error("{} : Connection failed, para. {} Reconnect....", new Date(), order); bootstrap.config().group().schedule(() -> connect(bootstrap, host, port, retry - 1), delay, TimeUnit.SECONDS); } }); channel = channelFuture.channel(); } /** * send message * * @param request * @return */ public RpcResponse send(final RpcRequest request) { try { channel.writeAndFlush(request).await(); } catch (InterruptedException e) { e.printStackTrace(); } return clientHandler.getRpcResponse(request.getRequestId()); } @PreDestroy public void close() { eventLoopGroup.shutdownGracefully(); channel.closeFuture().syncUninterruptibly(); } }
Our focus on data processing is on the ClientHandler class, which inherits the ChannelDuplexHandler class and can process outbound and inbound data.
public class ClientHandler extends ChannelDuplexHandler { /** * Maintaining mapping relationship between request object ID and response result Future using Map */ private final Map<String, DefaultFuture> futureMap = new ConcurrentHashMap<>(); @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { if (msg instanceof RpcResponse) { //Get the response object RpcResponse response = (RpcResponse) msg; DefaultFuture defaultFuture = futureMap.get(response.getRequestId()); //Write the result to DefaultFuture defaultFuture.setResponse(response); } super.channelRead(ctx,msg); } @Override public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception { if (msg instanceof RpcRequest) { RpcRequest request = (RpcRequest) msg; //Before sending the request object, save the request ID and construct a mapping relationship with the response Future. futureMap.putIfAbsent(request.getRequestId(), new DefaultFuture()); } super.write(ctx, msg, promise); } /** * Get response results * * @param requsetId * @return */ public RpcResponse getRpcResponse(String requsetId) { try { DefaultFuture future = futureMap.get(requsetId); return future.getRpcResponse(5000); } finally { //After success, remove from map futureMap.remove(requsetId); } } }
Reference article: https://xilidou.com/2018/09/2...
As can be seen from the above implementation, we define a Map to maintain the mapping relationship between the request ID and the response result. The purpose is to verify whether the server response matches the request. Because Netty channel may be used by multiple threads, when the result returns, you don't know which thread to return from, so a mapping relationship is needed.
Our result is encapsulated in DefaultFuture, because Netty is an asynchronous framework, and all returns are based on Future and Callback mechanisms. Here we customize Future to implement client "asynchronous call"
public class DefaultFuture { private RpcResponse rpcResponse; private volatile boolean isSucceed = false; private final Object object = new Object(); public RpcResponse getRpcResponse(int timeout) { synchronized (object) { while (!isSucceed) { try { object.wait(timeout); } catch (InterruptedException e) { e.printStackTrace(); } } return rpcResponse; } } public void setResponse(RpcResponse response) { if (isSucceed) { return; } synchronized (object) { this.rpcResponse = response; this.isSucceed = true; object.notify(); } } }
- In fact, wait and notify mechanisms are used, assisted by a boolean variable.
5. Netty Server
The implementation of Netty server is similar to that of client, but it should be noted that when the request is decoded, local functions need to be invoked by proxy. The following is the Server side code:
public class NettyServer implements InitializingBean { private EventLoopGroup boss = null; private EventLoopGroup worker = null; @Autowired private ServerHandler serverHandler; @Override public void afterPropertiesSet() throws Exception { //zookeeper is used here as the registry, which is not covered in this article and can be ignored. ServiceRegistry registry = new ZkServiceRegistry("127.0.0.1:2181"); start(registry); } public void start(ServiceRegistry registry) throws Exception { //Thread pool responsible for handling client connections boss = new NioEventLoopGroup(); //Thread pool for read and write operations worker = new NioEventLoopGroup(); ServerBootstrap serverBootstrap = new ServerBootstrap(); serverBootstrap.group(boss, worker) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG, 1024) .childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ChannelPipeline pipeline = ch.pipeline(); //Add Decoder pipeline.addLast(new RpcEncoder(RpcResponse.class, new JSONSerializer())); //Add encoder pipeline.addLast(new RpcDecoder(RpcRequest.class, new JSONSerializer())); //Adding Request Processor pipeline.addLast(serverHandler); } }); bind(serverBootstrap, 8888); } /** * If port binding fails, port number + 1, rebind * * @param serverBootstrap * @param port */ public void bind(final ServerBootstrap serverBootstrap,int port) { serverBootstrap.bind(port).addListener(future -> { if (future.isSuccess()) { log.info("port[ {} ] Binding success",port); } else { log.error("port[ {} ] Binding failed", port); bind(serverBootstrap, port + 1); } }); } @PreDestroy public void destory() throws InterruptedException { boss.shutdownGracefully().sync(); worker.shutdownGracefully().sync(); log.info("Close Netty"); } }
The following is the Handler class that handles read and write operations:
@Component @Slf4j @ChannelHandler.Sharable public class ServerHandler extends SimpleChannelInboundHandler<RpcRequest> implements ApplicationContextAware { private ApplicationContext applicationContext; @Override protected void channelRead0(ChannelHandlerContext ctx, RpcRequest msg) { RpcResponse rpcResponse = new RpcResponse(); rpcResponse.setRequestId(msg.getRequestId()); try { Object handler = handler(msg); log.info("Get the return result: {} ", handler); rpcResponse.setResult(handler); } catch (Throwable throwable) { rpcResponse.setError(throwable.toString()); throwable.printStackTrace(); } ctx.writeAndFlush(rpcResponse); } /** * Server uses proxy to process requests * * @param request * @return */ private Object handler(RpcRequest request) throws ClassNotFoundException, InvocationTargetException { //Use Class.forName to load Class files Class<?> clazz = Class.forName(request.getClassName()); Object serviceBean = applicationContext.getBean(clazz); log.info("serviceBean: {}",serviceBean); Class<?> serviceClass = serviceBean.getClass(); log.info("serverClass:{}",serviceClass); String methodName = request.getMethodName(); Class<?>[] parameterTypes = request.getParameterTypes(); Object[] parameters = request.getParameters(); //Using CGLIB Reflect FastClass fastClass = FastClass.create(serviceClass); FastMethod fastMethod = fastClass.getMethod(methodName, parameterTypes); log.info("Start calling CGLIB Dynamic Proxy Execution Server-side Method..."); return fastMethod.invoke(serviceBean, parameters); } @Override public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { this.applicationContext = applicationContext; } }
6. Client proxy
The client uses Java Dynamic agent to implement the communication details in the agent class. It is well known that Java Dynamic agent needs to implement the InvocationHandler interface.
@Slf4j public class RpcClientDynamicProxy<T> implements InvocationHandler { private Class<T> clazz; public RpcClientDynamicProxy(Class<T> clazz) throws Exception { this.clazz = clazz; } @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { RpcRequest request = new RpcRequest(); String requestId = UUID.randomUUID().toString(); String className = method.getDeclaringClass().getName(); String methodName = method.getName(); Class<?>[] parameterTypes = method.getParameterTypes(); request.setRequestId(requestId); request.setClassName(className); request.setMethodName(methodName); request.setParameterTypes(parameterTypes); request.setParameters(args); log.info("Request content: {}",request); //Open Netty client and connect directly NettyClient nettyClient = new NettyClient("127.0.0.1", 8888); log.info("Start connecting to the server:{}",new Date()); nettyClient.connect(); RpcResponse send = nettyClient.send(request); log.info("The request call returns the result:{}", send.getResult()); return send.getResult(); } }
- Encapsulating request object in invoke method, constructing NettyClient object, opening client and sending request message
Acting factories are as follows:
public class ProxyFactory { public static <T> T create(Class<T> interfaceClass) throws Exception { return (T) Proxy.newProxyInstance(interfaceClass.getClassLoader(),new Class<?>[] {interfaceClass}, new RpcClientDynamicProxy<T>(interfaceClass)); } }
- Create proxy classes for interfaces through Proxy.newProxyInstance
7. RPC remote call test
API:
public interface HelloService { String hello(String name); }
- Prepare a test API interface
Client:
@SpringBootApplication @Slf4j public class ClientApplication { public static void main(String[] args) throws Exception { SpringApplication.run(ClientApplication.class, args); HelloService helloService = ProxyFactory.create(HelloService.class); log.info("Response results“: {}",helloService.hello("pjmike")); } }
- The Method of Client Calling Interface
Server:
//Server-side implementation @Service public class HelloServiceImpl implements HelloService { @Override public String hello(String name) { return "hello, " + name; } }
Operation results:
Summary
Above all, we have implemented a non-very simple RPC framework based on Netty, which is far from a mature RPC framework, even the basic registry has not been implemented. But through this practice, I can say that I have a deeper understanding of RPC and understand what aspects of an RPC framework need to pay attention to. When we use a mature RPC framework in the future, such as Dubbo, To be able to understand the bottom of it is to use Netty as the basic communication framework. In the future, it will be relatively easy to dig deeper into the open source RPC framework source code.
Project address: https://github.com/pjmike/spr...