Preamble Introduction

Netty performs very well, and within some small user volume socket services, deploying only a single machine can meet business needs.However, when you encounter services with medium and large user volumes, you need to consider Netty deployed as a cluster to better meet business needs.However, when Netty deploys a cluster, it will encounter how to communicate across the servers, that is, with Cluster Services X and Y, User A Link Service X, User B Link Service Y. How do they all not communicate within a service?This chapter describes an implementation case to satisfy user communication across services.However, in the actual scenario, some extensibility modifications are needed. The case only implements the core main ideas, which is just a kind of ideas guidance and can not be used directly for business development.

Knowledge Points in this Chapter

• Cases across services use redis for publishing and subscribing to deliver messages, or zookeeper if you are a large service
• When User A sends a message to User B, it needs to pass B's channeId for the server to find out if the channeId belongs to its own service
• Multiple Netty services can also be started on a single machine, and available ports will be automatically found within the program

Environmental preparation

1. jdk1.8 [below jdk1.7 can only partially support netty]
2. Netty4.1.36.Final [netty3.x 4.x 5 changes a lot each time, so does the interface class name]
3. NetAssist Network Debugging Assistant, can download from the Internet or contact me, WeChat Public Number: bugstack bug stack | Focus on replying to your mailbox
4. redis server, case using windows version, from the official website on demand Download it

Code Samples

itstack-demo-rpc-2-09
└── src
    └── main
    │   ├── java
    │   │   └── org.itstack.demo.netty
    │   │        ├── domain
    │   │        │   ├── EasyResult.java
    │   │        │   ├── MsgAgreement.java
    │   │        │   ├── ServerInfo.java
    │   │        │   └── UserChannelInfo.java	
    │   │        ├── redis
    │   │        │   ├── config
    │   │        │   │	 ├── PublisherConfig.java
    │   │        │   │   └── ReceiverConfig.java	
    │   │        │   ├── AbstractReceiver.java
    │   │        │   ├── MsgAgreementReceiver.java
    │   │        │   ├── Publisher.java
    │   │        │   └── RedisUtil.java
    │   │        ├── server
    │   │        │   ├── MyChannelInitializer.java
    │   │        │   ├── MyServerHandler.java
    │   │        │   └── NettyServer.java	
    │   │        ├── service
    │   │        │   └── ExtServerService.java	
    │   │        ├── util
    │   │        │   ├── CacheUtil.java
    │   │        │   ├── MsgUtil.java
    │   │        │   └── NetUtil.java	
    │   │        ├── web
    │   │        │	  └── NettyController.java
    │   │        └── Application.java
    │   ├── resources	
    │   │   └── application.yml
    │   └── webapp	
    │       ├── res		
    │       └── WEB-INF
    │        	└── index.jsp		
    └── test
         └── java
             └── org.itstack.demo.test
                 └── ApiTest.java

Demo Explanation Part Key Code Block, Complete Code Download, Focus on Public Number; bugstack bug stack | Reply: netty case source

/**
 * Message Protocol
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack bug stack  Focus on getting learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on 2019
 */
public class MsgAgreement {

    private String toChannelId; //Send to someone, someone channelId
    private String content;     //Message Content

    public MsgAgreement() {
    }

    public MsgAgreement(String toChannelId, String content) {
        this.toChannelId = toChannelId;
        this.content = content;
    }

    public String getToChannelId() {
        return toChannelId;
    }

    public void setToChannelId(String toChannelId) {
        this.toChannelId = toChannelId;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }
    
}

/**
 * Publisher
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack wormhole stack  Get learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on @2019
 */
@Configuration
public class PublisherConfig {

    @Bean
    public RedisTemplate<String, Object> redisMessageTemplate(RedisConnectionFactory connectionFactory) {
        RedisTemplate<String, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);
        template.setDefaultSerializer(new FastJsonRedisSerializer<>(Object.class));
        return template;
    }

}

/**
 * Subscriber
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack wormhole stack  Get learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on @2019
 */
@Configuration
public class ReceiverConfig {

    @Bean
    public RedisMessageListenerContainer container(RedisConnectionFactory connectionFactory, MessageListenerAdapter msgAgreementListenerAdapter) {
        RedisMessageListenerContainer container = new RedisMessageListenerContainer();
        container.setConnectionFactory(connectionFactory);
        container.addMessageListener(msgAgreementListenerAdapter, new PatternTopic("itstack-demo-netty-push-msgAgreement"));
        return container;
    }

    @Bean
    public MessageListenerAdapter msgAgreementListenerAdapter(MsgAgreementReceiver receiver) {
        return new MessageListenerAdapter(receiver, "receiveMessage");
    }

}

/**
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack wormhole stack  Get learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on @2019
 */
@Service
public class MsgAgreementReceiver extends AbstractReceiver {

    private Logger logger = LoggerFactory.getLogger(MsgAgreementReceiver.class);

    @Override
    public void receiveMessage(Object message) {
        logger.info("Received PUSH Message:{}", message);
        MsgAgreement msgAgreement = JSON.parseObject(message.toString(), MsgAgreement.class);
        String toChannelId = msgAgreement.getToChannelId();
        Channel channel = CacheUtil.cacheChannel.get(toChannelId);
        if (null == channel) return;
        channel.writeAndFlush(MsgUtil.obj2Json(msgAgreement));
    }

}

/**
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack bug stack  Focus on getting learning source 
 * Create by fuzhengwei on 2019
 */
@Service("redisUtil")
public class RedisUtil {

    @Autowired
    private StringRedisTemplate redisTemplate;

    public void pushObj(UserChannelInfo userChannelInfo) {
        redisTemplate.opsForHash().put("itstack-demo-netty-2-09-user", userChannelInfo.getChannelId(), JSON.toJSONString(userChannelInfo));
    }

    public List<UserChannelInfo> popList() {
        List<Object> values = redisTemplate.opsForHash().values("itstack-demo-netty-2-09-user");
        if (null == values) return new ArrayList<>();
        List<UserChannelInfo> userChannelInfoList = new ArrayList<>();
        for (Object strJson : values) {
            userChannelInfoList.add(JSON.parseObject(strJson.toString(), UserChannelInfo.class));
        }
        return userChannelInfoList;
    }

    public void remove(String channelId) {
        redisTemplate.opsForHash().delete("itstack-demo-netty-2-09-user",channelId);
    }

    public void clear(){
        redisTemplate.delete("itstack-demo-netty-2-09-user");
    }

}

/**
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack bug stack  Focus on getting learning source 
 * Create by fuzhengwei on 2019
 */
public class MyServerHandler extends ChannelInboundHandlerAdapter {

    private Logger logger = LoggerFactory.getLogger(MyServerHandler.class);

    private ExtServerService extServerService;

    public MyServerHandler(ExtServerService extServerService) {
        this.extServerService = extServerService;
    }

    /**
     * This channel is active when the client actively links the links on the server side.That is, the client establishes a communication channel with the server and can transfer data.
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        SocketChannel channel = (SocketChannel) ctx.channel();
        System.out.println("Link Report Start");
        System.out.println("Link Report Information: There is a client link to this server. channelId: " + channel.id());
        System.out.println("Link Report IP:" + channel.localAddress().getHostString());
        System.out.println("Link Report Port:" + channel.localAddress().getPort());
        System.out.println("Link report complete");

        //Save user information
        UserChannelInfo userChannelInfo = new UserChannelInfo(channel.localAddress().getHostString(), channel.localAddress().getPort(), channel.id().toString(), new Date());
        extServerService.getRedisUtil().pushObj(userChannelInfo);
        CacheUtil.cacheChannel.put(channel.id().toString(), channel);
        //Notify client that the link was successfully established
        String str = "Notify client that the link was successfully established" + " " + new Date() + " " + channel.localAddress().getHostString() + "\r\n";
        ctx.writeAndFlush(MsgUtil.buildMsg(channel.id().toString(), str));

    }

    /**
     * This channel is inactive when the client actively disconnects the server.That is, the client and the server have closed the communication channel and cannot transfer data.
     */
    @Override
    public void channelInactive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("Client disconnects" + ctx.channel().localAddress().toString());
        extServerService.getRedisUtil().remove(ctx.channel().id().toString());
        CacheUtil.cacheChannel.remove(ctx.channel().id().toString(), ctx.channel());
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object objMsgJsonStr) throws Exception {
        //Receive msg messages {You no longer need to decode yourself here compared to the previous chapter}
        System.out.println(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date()) + " Received message content:" + objMsgJsonStr);

        MsgAgreement msgAgreement = MsgUtil.json2Obj(objMsgJsonStr.toString());

        String toChannelId = msgAgreement.getToChannelId();
        //Determine whether the receiving message user is on this server
        Channel channel = CacheUtil.cacheChannel.get(toChannelId);
        if (null != channel) {
            channel.writeAndFlush(MsgUtil.obj2Json(msgAgreement));
            return;
        }
        //If NULL, the user receiving the message is not on this server and the push message is required to be global
        logger.info("The user receiving the message is not on this server. PUSH!");
        extServerService.push(msgAgreement);
    }

    /**
     * Catch exceptions, and when they occur, you can do something about them, such as printing logs, closing links
     */
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        ctx.close();
        extServerService.getRedisUtil().remove(ctx.channel().id().toString());
        CacheUtil.cacheChannel.remove(ctx.channel().id().toString(), ctx.channel());
        System.out.println("Exception information:\r\n" + cause.getMessage());
    }

}

/**
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack bug stack  Focus on getting learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on 2019
 */
public class CacheUtil {

    // Cache channel
    public static Map<String, Channel> cacheChannel = Collections.synchronizedMap(new HashMap<String, Channel>());

    // Cache service information
    public static Map<Integer, ServerInfo> serverInfoMap = Collections.synchronizedMap(new HashMap<Integer, ServerInfo>());

    // Cache server
    public static Map<Integer, NettyServer> serverMap = Collections.synchronizedMap(new HashMap<Integer, NettyServer>());

}

/**
 * Hole stack: https://bugstack.cn
 * Public Number: bugstack wormhole stack  Get learning source 
 * Cave Group: Group 5398358 Group 5360692
 * Create by fuzhengwei on @2019
 */
@Controller
public class NettyController {

    private Logger logger = LoggerFactory.getLogger(NettyController.class);
    //Default Thread Pool
    private static ExecutorService executorService = Executors.newFixedThreadPool(2);

    @Value("${server.port}")
    private int serverPort;
    @Autowired
    private ExtServerService extServerService;
    @Resource
    private RedisUtil redisUtil;
    //Netty Server
    private NettyServer nettyServer;

    @RequestMapping("/index")
    public String index(Model model) {
        model.addAttribute("serverPort", serverPort);
        return "index";
    }

    @RequestMapping("/openNettyServer")
    @ResponseBody
    public EasyResult openNettyServer() {
        try {
            int port = NetUtil.getPort();
            logger.info("start-up Netty Service, get available ports:{}", port);
            nettyServer = new NettyServer(new InetSocketAddress(port), extServerService);
            Future<Channel> future = executorService.submit(nettyServer);
            Channel channel = future.get();
            if (null == channel) {
                throw new RuntimeException("netty server open error channel is null");
            }
            while (!channel.isActive()) {
                logger.info("start-up Netty Service, loop waiting to start...");
                Thread.sleep(500);
            }
            CacheUtil.serverInfoMap.put(port, new ServerInfo(NetUtil.getHost(), port, new Date()));
            CacheUtil.serverMap.put(port, nettyServer);
            logger.info("start-up Netty Service, complete:{}", channel.localAddress());
            return EasyResult.buildSuccessResult();
        } catch (Exception e) {
            logger.error("start-up Netty Service Failure", e);
            return EasyResult.buildErrResult(e);
        }
    }

    @RequestMapping("/closeNettyServer")
    @ResponseBody
    public EasyResult closeNettyServer(int port) {
        try {
            logger.info("Close Netty Service started, port:{}", port);
            NettyServer nettyServer = CacheUtil.serverMap.get(port);
            if (null == nettyServer) {
                CacheUtil.serverMap.remove(port);
                return EasyResult.buildSuccessResult();
            }
            nettyServer.destroy();
            CacheUtil.serverMap.remove(port);
            CacheUtil.serverInfoMap.remove(port);
            logger.info("Close Netty Service complete, port:{}", port);
            return EasyResult.buildSuccessResult();
        } catch (Exception e) {
            logger.error("Close Netty Service failed, port:{}", port, e);
            return EasyResult.buildErrResult(e);
        }
    }

    @RequestMapping("/queryNettyServerList")
    @ResponseBody
    public Collection<ServerInfo> queryNettyServerList() {
        try {
            Collection<ServerInfo> serverInfos = CacheUtil.serverInfoMap.values();
            logger.info("Query the list of servers.{}", JSON.toJSONString(serverInfos));
            return serverInfos;
        } catch (Exception e) {
            logger.info("Query of server side list failed.", e);
            return null;
        }
    }

    @RequestMapping("/queryUserChannelInfoList")
    @ResponseBody
    public List<UserChannelInfo> queryUserChannelInfoList() {
        try {
            logger.info("Query User List Information Start");
            List<UserChannelInfo> userChannelInfoList = redisUtil.popList();
            logger.info("Query user list information complete. list: {}", JSON.toJSONString(userChannelInfoList));
            return userChannelInfoList;
        } catch (Exception e) {
            logger.error("Failed to query user list information", e);
            return null;
        }
    }

}

server:
  port: 8080

spring:
  mvc:
    view:
      prefix: /WEB-INF/
      suffix: .jsp
  redis:
    host: 127.0.0.1
    port: 6379

<%--
  //Hole stack: https://bugstack.cn
  //Public Number: bugstack wormhole stack  Get learning source 
  Create by fuzhengwei on 2019
--%>
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<title>Focus on Public Number: bugstack Bug Hole Stack | Thematic case development, focusing on source code | bugstack.cn Commissioner for Political Affairs</title>
<meta charset="UTF-8">
<link rel="stylesheet" type="text/css" href="res/js/themes/default/easyui.css">
<link rel="stylesheet" type="text/css" href="res/js/themes/icon.css">
<script type="text/javascript" src="res/js/jquery.min.js"></script>
<script type="text/javascript" src="res/js/jquery.easyui.min.js"></script>

<style>

</style>

<script>
util = {
  formatDate: function (value, row, index) {
      if (null == value) return "";
      var date = new Date();
      date.setTime(value);
      return date.format('yyyy-MM-dd HH:mm:ss');
  }
};
</script>

</head>

<body>
	<div style="margin:20px 0;"></div>
	<table class="easyui-datagrid" title="localhost:${serverPort} | Netty Server" style="width:700px;height:250px"
			data-options="rownumbers:true,singleSelect:true,url:'/queryNettyServerList',method:'get',toolbar:toolbar">
		<thead>
			<tr>
				<th data-options="field:'ip'">IP</th>
				<th data-options="field:'port'">port</th>
				<th data-options="field:'openDate'">Start-up time</th>
			</tr>
		</thead>
	</table>
	<script type="text/javascript">
		var toolbar = [{
			text:'start-up',
			iconCls:'icon-open',
			handler:function(){
			    $.post('/openNettyServer',{}, function (res) {
                               if (res.success) {
                                  $.messager.show({
                                        title: 'Message prompt',
                                        msg: 'Start successfully, please refresh the page later!'
                                  });
                                   $('#easyui-datagrid').datagrid('reload');
                               } else {
                                   $.messager.show({
                                       title: 'Error',
                                       msg: res.msg
                                   });
                               }
                            }, 'json');
			}
		},'-',{
			text:'Close',
			iconCls:'icon-close',
			handler:function(){
			     //You can add your own implementation
			}
		}];
	</script>
    <hr/>
	<!-- server-user -->
    <table class="easyui-datagrid" title="localhost:${serverPort} | User Link Information" style="width:700px;height:250px"
    			data-options="rownumbers:true,singleSelect:true,url:'/queryUserChannelInfoList',method:'get'">
    		<thead>
    			<tr>
    				<th data-options="field:'ip'">IP</th>
    				<th data-options="field:'port'">port</th>
    				<th data-options="field:'channelId'">user ID</th>
    				<th data-options="field:'linkDate'">Link Time</th>
    			</tr>
    		</thead>
    </table>
</body>
</html>

test result

2019-09-01 12:59:29.649  INFO 8952 --- [nio-8081-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2019-09-01 12:59:29.649  INFO 8952 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2019-09-01 12:59:29.681  INFO 8952 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 32 ms
 2019-09-01 12:59:31.350 INFO 8952 --- [nio-8081-exec-2] o.i.demo.netty.web.NettyController: Query the list of servers.[]
2019-09-01 12:59:31.371 INFO 8952 --- [nio-8081-exec-3] o.i.demo.netty.web.NettyController: Start querying user list information
 2019-09-01 12:59:31.380 INFO 8952 --- [nio-8081-exec-3] o.i.demo.netty.web.NettyController: Query user list information complete.List:[]
2019-09-01 13:04:22.864 INFO 8952 --- [nio-8081-exec-6] o.i.demo.netty.web.NettyController: Start the Netty service and get available ports: 7398
 2019-09-01 13:04:22.879 INFO 8952 --- [pool-1-thread-1] o.itstack.demo.netty.server.NettyServer: itstack-demo-netty server start. {Focus on public number: bugstack wormhole stack, get source code}
2019-09-01 13:04:22.880 INFO 8952 --- [nio-8081-exec-6] o.i.demo.netty.web.NettyController: Start Netty service, complete: /0:0:0:0:0:0:7398
 2019-09-01 13:04:23.612 INFO 8952 --- [nio-8081-exec-9] o.i.demo.netty.web.NettyController: Query the list of servers.[{"ip":"10.13.28.13","openDate":1567314262880,"port":7398}]
2019-09-01 13:04:23.634 INFO 8952 --- [i o-8081-exec-10] o.i.demo.netty.web.NettyController: Start querying user list information
 2019-09-01 13:04:23.636 INFO 8952 --- [i o-8081-exec-10] o.i.demo.netty.web.NettyController: Query user list information complete.List:[]
Link Report Start
 Link Report Information: There is a client link to this server.channelId:3a2d5cee
 Link Report IP:10.13.28.13
 Link Report Port:7398
 Link report complete
 2019-09-01 13:04:42.704 INFO 8952 --- [nio-8081-exec-2] o.i.demo.netty.web.NettyController: Query the list of servers.[{"ip":"10.13.28.13","openDate":1567314262880,"port":7398}]
2019-09-01 13:04:42.738 INFO 8952 --- [nio-8081-exec-3] o.i.demo.netty.web.NettyController: Start querying user list information
 2019-09-01 13:04:42.755 INFO 8952 --- [nio-8081-exec-3] o.i.demo.netty.web.NettyController: Query user list information complete.List: [{"channelId": "39d45ff7", "ip": "10.13.28.13", "linkDate": 1567314278944","port": 7397}, {"channelId":"3a2d5cee","ip":"10.13.28.13","linkDate": 1567314280442", "port": 7398}]
2019-09-01 13:05:25.545 INFO 8952 --- [container-2] o.i.d.netty.redis.MsgAgreementReceiver: Received PUSH message: {"content": {"hi!" I'm Weixin Public Number: bugstack bug stack | Welcome & Get the source code.*Users from server A send information to users from server B.[Wrap at end, for half-packed stickers] ","toChannelId":"3a2d5cee"}
2019-09-01 13:05:26.107 INFO 8952 --- [container-3] o.i.d.netty.redis.MsgAgreementReceiver: Received PUSH message: {"content": {"hi!" I'm Weixin Public Number: bugstack bug stack | Welcome & Get the source code.*Users from server A send information to users from server B.[Wrap at end, for half-packed stickers] ","toChannelId":"3a2d5cee"}
2019-09-01 13:05:27.025 INFO 8952 --- [container-4] o.i.d.netty.redis.MsgAgreementReceiver: Received PUSH message: {"content": {"hi!" I'm Weixin Public Number: bugstack bug stack | Welcome & Get the source code.*Users from server A send information to users from server B.[Wrap at end, for half-packed stickers] ","toChannelId":"3a2d5cee"}
2019-09-01 13:05:27.545 INFO 8952 --- [container-5] o.i.d.netty.redis.MsgAgreementReceiver: Received PUSH message: {"content": {"hi!" I'm Weixin Public Number: bugstack bug stack | Welcome & Get the source code.*Users from server A send information to users from server B.[Wrap at end, for half-packed stickers] ","toChannelId":"3a2d5cee"}
2019-09-01 13:05:28.559 INFO 8952 --- [container-6] o.i.d.netty.redis.MsgAgreementReceiver: Received PUSH message: {"content": {"hi!" I'm Weixin Public Number: bugstack bug stack | Welcome & Get the source code.*Users from server A send information to users from server B.[Wrap at end, for half-packed stickers] ","toChannelId":"3a2d5cee"}

{"content":"hi! I'm WeChat Public Number: bugstack Bug Hole Stack | Welcome to your attention&Get the source code.* Come from A Users in the service end are directed to B Users in the server send messages.[Line break at end, for half-packed stickers]","toChannelId":"3a2d5cee"}