[RPC] Implementation of RPC framework based on netty+zookeeper step by step

Keywords: Zookeeper JSON Netty github

The last one implements service registration discovery and basic string communication functions. This one implements the functions that we usually invoke using classes of RPC framework.

Implementing consumer end to call remote services through interface classes, the main core is to use dynamic proxy and reflection, which is implemented step by step.

The GitHub code address is posted here, you can download and run it directly if you want to see the code directly: https://github.com/whiteBX/wrpc

Let's start with the consumer side code. The complete RPCConsumer code is as follows:

public class RPCConsumer {

    /**
     * url processor
     */
    private UrlHolder   urlHolder   = new UrlHolder();
    /**
     * netty Client
     */
    private NettyClient nettyClient = new NettyClient();

    /**
     * Remote Call
     *
     * @param appCode
     * @param param
     * @return
     */
    public String call(String appCode, String param) {
        try {
            // Get the service address from zookeeper
            String serverIp = urlHolder.getUrl(appCode);
            if (serverIp == null) {
                System.out.println("Remote Call Error:Currently there is no service provider");
                return "connect error";
            }
            // Connect netty, request and receive response
            RpcClientNettyHandler clientHandler = new RpcClientNettyHandler();
            clientHandler.setParam(param);
            nettyClient.initClient(serverIp, clientHandler);
            String result = clientHandler.process();
            System.out.println(MessageFormat.format("Call the server:{0},Request parameters:{1},Response parameters:{2}", serverIp, param, result));
            return result;
        } catch (Exception e) {
            System.out.println("Remote service invocation failed:" + e);
            return "error";
        }
    }

    /**
     * Get the proxy class
     * @param clazz
     * @param appCode
     * @return
     */
    public <T> T getBean(final Class<T> clazz, final String appCode) {
        return (T) Proxy.newProxyInstance(clazz.getClassLoader(), new Class[] { clazz }, new InvocationHandler() {
            public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
                String param = JSON.toJSONString(args[0]);
                String beanMessage = MessageFormat.format(ConsumerConstant.BEAN_STRING, clazz.getName(),
                    method.getName(), method.getParameterTypes()[0].getName());
                return JSON.parseObject(call(appCode, beanMessage.concat(param)), method.getReturnType());
            }
        });
    }
}

Here, the getBean method mainly obtains the proxy class by dynamic proxy, and what we pass in is our own Service interface class, which binds InvocationHandler to perform the corresponding remote call operation when calling the method of the interface class, including remote call wrapping and return value parsing.

Next, look at the provider related classes, add a new Service registration method in the RPCProvider class, and register our own Service into the system cache:

public class RPCProvider {
    /**
     * netty Client
     */
    private static NettyClient nettyClient = new NettyClient();
    /**
     * zookeeper Client
     */
    private static ZKClient    zkClient    = new ZKClient();

    public void registry(String server, int port) {
        // Open netty listening client connection
        nettyClient.startServer(port);
        // Create zk connections and temporary nodes
        ZooKeeper zooKeeper = zkClient.newConnection(ProviderConstant.ZK_CONNECTION_STRING,
            ProviderConstant.ZK_SESSION_TIME_OUT);
        String serverIp = server + CommonConstant.COMMOA + port;
        zkClient.createEphemeralNode(zooKeeper, ProviderConstant.APP_CODE, serverIp.getBytes());
    }

    /**
     * Registered Service Provider
     * @param clazz
     * @param obj
     */
    public void provide(Class<?> clazz, Object obj) {
        ProviderBeanHolder.regist(clazz.getName(), obj);
    }
}

The ProviderBeanHolder class code is as follows, mainly responsible for caching service registration information:

 public class ProviderBeanHolder {

    /**
     * bean Register Cache
     */
    private static Map<String, Object> providerList = new HashMap<String, Object>();

    /**
     * register
     * @param clazzName
     * @param obj
     */
    public static void regist(String clazzName, Object obj) {
        providerList.put(clazzName, obj);
        System.out.println("register provider: " + clazzName);
    }

    /**
     * Obtain
     * @param clazzName
     * @return
     */
    public static Object getBean(String clazzName) {
        return providerList.get(clazzName);
    }
}

Next, let's look at the RpcServerNettyHandler class, which receives client requests and processes them. Here, the service registration method is invoked by reflection.

public class RpcServerNettyHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) {
        System.out.println("The server receives the request:" + msg);
        try {
            // Resolve the class name + method name + request parameter type (method signature)
            String[] splitParam = msg.toString().split(ProviderConstant.DOLLAR_SPLIT);
            String[] beanMessage = splitParam[0].split(ProviderConstant.SHARP_SPLIT);
            // Access to Registered Services
            Object object = ProviderBeanHolder.getBean(beanMessage[0]);
            if (object == null) {
                System.out.println("Service class not registered:" + beanMessage[0]);
            }
            // Invoking services through reflection
            Class paramType = Class.forName(beanMessage[2]);
            Method method = object.getClass().getDeclaredMethod(beanMessage[1], paramType);
            Object response = method.invoke(object, JSON.parseObject(splitParam[1], paramType));
            // Request response
            ctx.writeAndFlush(JSON.toJSONString(response));
        } catch (Exception e) {
            System.out.println("Service exception");
        }
    }
}

Here is the main use of reflection-related knowledge, the implementation of service invocation and response, the relevant code to achieve here, let's write a service test:

public class HelloRequest {

    private int    seq;

    private String content;

    // Omit getter setter
}
public class HelloResponse {

    private int code;

    private String message;

    // Omit getter setter
}
public interface HelloService {

    HelloResponse hello(HelloRequest request);
}
public class HelloServiceImpl implements HelloService {

    public HelloResponse hello(HelloRequest request) {
        System.out.println("HelloService Receiving requests,serial number:" + request.getSeq());
        HelloResponse response = new HelloResponse();
        response.setCode(200);
        response.setMessage("success:" + request.getSeq());
        return response;
    }
}

The test code is as follows:

    // ProviderTest related code
    public static void main(String[] args) throws InterruptedException {
        RPCProvider provider = new RPCProvider();
        provider.registry("127.0.0.1", 8091);
        provider.registry("127.0.0.1", 8092);
        provider.registry("127.0.0.1", 8093);
        provider.registry("127.0.0.1", 8094);
        provider.registry("127.0.0.1", 8095);
        provider.provide(HelloService.class, new HelloServiceImpl());

        Thread.sleep(Long.MAX_VALUE);
    }
    // ConsumerTest related code
    public static void main(String[] args) {
        RPCConsumer consumer = new RPCConsumer();
        HelloService helloService = consumer.getBean(HelloService.class, ConsumerConstant.APP_CODE);
        int i = 0;
        while (true) {
            HelloRequest request = new HelloRequest();
            request.setSeq(i++);
            HelloResponse helloResponse = helloService.hello(request);
            System.out.println("The client receives a response:" + JSON.toJSONString(helloResponse));
            Thread.sleep(Long.MAX_VALUE);
        }
    }

The results are as follows:

// Server Log
 zookeeper Connection Successful
 Temporary Node Creation Successful: / Regisry / 100000 / 0000000025
 zookeeper Connection Successful
 Temporary Node Creation Successful: / Regisry / 100000 / 0000000026
 zookeeper Connection Successful
 Temporary Node Creation Successful: / Regisry / 100000 / 0000000027
 zookeeper Connection Successful
 Temporary Node Creation Successful: / Regisry / 100000 / 0000000028
 zookeeper Connection Successful
 Temporary Node Creation Successful: / Regisry / 100000 / 0000000029
 Register provider: org.white.wrpc.hello.service.HelloService
 The server receives the request: org. white. wrpc. hello. service. HelloService hello org. white. wrpc. hello. model. request. HelloRequest ${"seq": 0}.
The server receives the request, serial number: 0
 The server receives the request: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":1}.
The server receives the request, serial number: 1
 The server receives the request: org. white. wrpc. hello. service. HelloService hello org. white. wrpc. hello. model. request. HelloRequest ${"seq": 2}.
The server receives the request, serial number: 2

// Client Log
 zookeeper Connection Successful
 Call server: 127.0.0.1,8094, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":0}, response parameter: {"code":200,""message":"success:0"}
The client receives a response: {"code":200,"message":"success:0"}
Call server: 127.0.0.1,8094, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":1}, response parameter: {"code":200,""message":"success:1"}
The client receives a response: {"code":200,"message":"success:1"}
Call server: 127.0.0.1,8092, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":2}, response parameter: {"code":200,""message":"success:2"}
The client receives a response: {"code":200,"message":"success:2"}

Modify the ports in the Provider class and restart the four services. You can see that the client has the following logs:

zookeeper Connection Successful
 zookeeper Connection Successful
 zookeeper Connection Successful
 zookeeper Connection Successful
 Call server: 127.0.0.1,8095, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":29}, response parameter: {"code":200,""message":"success:29"}
The client receives a response: {"code":200,"message":"success:29"}
Call server: 127.0.0.1,8091, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":30}, response parameter: {"code":200,""message":"success:30"}
The client receives a response: {"code":200,"message":"success:30"}
Call server: 127.0.0.1,8097, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":31}, response parameter: {"code":200,""message":"success:31"}
The client receives a response: {"code":200,"message":"success:31"}
Call server: 127.0.0.1,8092, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":32}, response parameter: {"code":200,""message":"success:32"}
The client receives a response: {"code":200,"message":"success:32"}
Call server: 127.0.0.1,8099, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":33}, response parameter: {"code":200,""message":"success:33"}
The client receives a response: {"code":200,"message":"success:33"}
Call server: 127.0.0.1,8096, request parameter: org.white.wrpc.hello.service.HelloService#hello#org.white.wrpc.hello.model.request.HelloRequest${"seq":34}, response parameter: {"code":200,""message":"success:34"}

As you can see, when new services are added, the client can connect to new services without changing anything. When the new services are closed, the client will access the remaining services without any problems.

At this point, the RPC framework's own registered bean s are implemented and invoked through the interface.

Follow-up will step by step achieve load balancing/call link Trace recording/current limiting functions, welcome to continue to pay attention!

Posted by phynias on Tue, 03 Sep 2019 05:59:50 -0700