First, set sail
Based on the principle of from nothing to have, from having to being excellent, we plan to realize the function through web first, and then optimize it to socket.
1. Technology selection
The web framework uses the gmvc framework written before( Handwritten MVC framework (I) - start again )Message storage adopts the way of existing database, and the framework used is gdao written in the previous period( Handwritten DAO framework (I) - start with "1" ).
2. Project construction
The project is originally in the form of a single project, but considering that it is not very friendly to separate the server and the client, the maven parent-child module is adopted.
The parent pom is configured as follows:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.shuimutong</groupId> <artifactId>gmq</artifactId> <version>${global.version}</version> <packaging>pom</packaging> <url>http://maven.apache.org</url> <modules> <module>gmq-server</module> <module>gmq-client</module> </modules> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <global.version>0.0.1-SNAPSHOT</global.version> <fastjson.version>1.2.60</fastjson.version> <gdao.version>2.0.0-SNAPSHOT</gdao.version> <gmvc.version>1.0.1-SNAPSHOT</gmvc.version> <gutil.version>0.0.2-SNAPSHOT</gutil.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>me.lovegao</groupId> <artifactId>gdao</artifactId> <version>${gdao.version}</version> </dependency> <dependency> <groupId>com.shuimutong</groupId> <artifactId>gmvc</artifactId> <version>${gmvc.version}</version> </dependency> <dependency> <groupId>com.shuimutong</groupId> <artifactId>gutil</artifactId> <version>${gutil.version}</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>${fastjson.version}</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.4</version> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.16</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.6.1</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.6.2</version> </dependency> </dependencies> </dependencyManagement> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.7.0</version> <configuration> <target>1.8</target> <source>1.8</source> </configuration> </plugin> </plugins> </build> </project>
The configuration of the mq server pom is as follows:
<?xml version="1.0"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>com.shuimutong</groupId> <artifactId>gmq</artifactId> <version>${global.version}</version> </parent> <groupId>com.shuimutong</groupId> <artifactId>gmq-server</artifactId> <packaging>war</packaging> <version>${global.version}</version> <name>gmq-server</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.1.0</version> </dependency> <dependency> <groupId>me.lovegao</groupId> <artifactId>gdao</artifactId> </dependency> <dependency> <groupId>com.shuimutong</groupId> <artifactId>gmvc</artifactId> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> </dependency> </dependencies> <build> <finalName>com.shuimutong.gmq_server</finalName> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.7.0</version> <configuration> <target>1.8</target> <source>1.8</source> </configuration> </plugin> </plugins> </build> </project>
GDAO (https://gitee.com/simpleha/gdao.git), GMVC (https://gitee.com/simpleha/gmvc.git), gutil (https://gitee.com/simpleha/gutil.git) that the project depends on need to be clone to local and compiled to maven warehouse.
II. Interface sorting
1,SyncController
Before publishing or subscribing to a message, we need to add a new topic. I've separated the new topic from the later publish and subscribe messages here, mainly considering that the frequency of access of the two classes is different, which is conducive to the optimization in the future.
The specific implementation is as follows:
package com.shuimutong.gmq.server.controller; import java.util.List; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.alibaba.fastjson.JSONObject; import com.shuimutong.gmq.server.bean.enums.RequestParamEnum; import com.shuimutong.gmq.server.bean.enums.ResponseCodeEnum; import com.shuimutong.gmq.server.bean.vo.ResponseDataVo; import com.shuimutong.gmq.server.bean.vo.UriDescVo; import com.shuimutong.gmq.server.exception.ServiceException; import com.shuimutong.gmq.server.service.SyncService; import com.shuimutong.gmq.server.service.TopicService; import com.shuimutong.gmvc.annotation.XAutowired; import com.shuimutong.gmvc.annotation.XController; import com.shuimutong.gmvc.annotation.XRequestMapping; import com.shuimutong.gmvc.util.RequestResolveUtil; /** * Non message information synchronization controller * @ClassName: MessageController * @Description:(Here is a sentence to describe the function of this class.) * @author: Water barrel * @date: 2019 9:45:47 PM, October 20, 2010 * @Copyright: 2019 [All rights reserved */ @XController @XRequestMapping("/sync") public class SyncController { private final static Logger log = LoggerFactory.getLogger(SyncController.class); @XAutowired private TopicService topicService; @XAutowired private SyncService syncService; /** * Get uri description * @param request * @param reponse */ @XRequestMapping("/getPath") public void getPath(HttpServletRequest request, HttpServletResponse reponse) { List<UriDescVo> uriList = syncService.listUriDesc(); ResponseDataVo responseData = new ResponseDataVo(ResponseCodeEnum.OK, uriList); RequestResolveUtil.returnJson(request, reponse, JSONObject.toJSONString(responseData)); } /** * Add topic * @param request * @param reponse */ @XRequestMapping("/addTopic") public void addTopic(HttpServletRequest request, HttpServletResponse reponse) { ResponseDataVo responseData = null; String topic = request.getParameter(RequestParamEnum.TOPIC.getParamName()); if(StringUtils.isBlank(topic)) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "The theme is empty."); } else { try { boolean addState = topicService.addTopic(topic); if(addState) { responseData = new ResponseDataVo(ResponseCodeEnum.OK, "Add success"); } else { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "Subject already exists"); } } catch (ServiceException e) { log.error("addTopicException," + topic, e); responseData = new ResponseDataVo(ResponseCodeEnum.SERVER_ERROR); } } RequestResolveUtil.returnJson(request, reponse, JSONObject.toJSONString(responseData)); } }
getPath() is not used for the time being. Let's discuss it later.
2,MessageController
After the topic of the message is created, the message is published and subscribed.
The specific implementation is as follows:
package com.shuimutong.gmq.server.controller; import java.util.List; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.alibaba.fastjson.JSONObject; import com.shuimutong.gmq.server.bean.SystemConstant; import com.shuimutong.gmq.server.bean.dos.TopicDo; import com.shuimutong.gmq.server.bean.enums.RequestParamEnum; import com.shuimutong.gmq.server.bean.enums.ResponseCodeEnum; import com.shuimutong.gmq.server.bean.vo.ResponseDataVo; import com.shuimutong.gmq.server.exception.ServiceException; import com.shuimutong.gmq.server.service.MessageService; import com.shuimutong.gmq.server.service.TopicService; import com.shuimutong.gmvc.annotation.XAutowired; import com.shuimutong.gmvc.annotation.XController; import com.shuimutong.gmvc.annotation.XRequestMapping; import com.shuimutong.gmvc.util.RequestResolveUtil; import com.shuimutong.guti.bean.TwoTuple; /** * Sending and receiving message controller * @ClassName: MessageController * @Description:(Here is a sentence to describe the function of this class.) * @author: Water barrel * @date: 2019 9:45:47 PM, October 20, 2010 * @Copyright: 2019 [All rights reserved */ @XController @XRequestMapping(SystemConstant.STR_URL_MESSAGE) public class MessageController { private final static Logger log = LoggerFactory.getLogger(MessageController.class); @XAutowired private MessageService messageService; @XAutowired private TopicService topicService; /** * Get message * @param request * @param reponse */ @XRequestMapping(SystemConstant.STR_URL_GET_MESSAGE) public void getMessage(HttpServletRequest request, HttpServletResponse reponse) { ResponseDataVo responseData = null; String topic = request.getParameter(RequestParamEnum.TOPIC.getParamName()); String offsetStr = request.getParameter(RequestParamEnum.OFFSET.getParamName()); String sizeStr = request.getParameter(RequestParamEnum.SIZE.getParamName()); if(StringUtils.isBlank(topic) || !StringUtils.isNumeric(offsetStr) || !StringUtils.isNumeric(sizeStr)) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "The subject is empty or the number is illegal"); } else { int offset = Integer.parseInt(offsetStr); int size = Integer.parseInt(sizeStr); if(offset < 0 || size < 1) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "Digital anomaly"); } else { try { TopicDo topicDo = topicService.findByTopic(topic); if(topicDo == null) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "Subject does not exist"); } else { List<TwoTuple<Long, String>> list = messageService.listMessage(topic, offset, size); responseData = new ResponseDataVo(ResponseCodeEnum.OK, list); } } catch (ServiceException e) { log.error("getMessageException", e); responseData = new ResponseDataVo(ResponseCodeEnum.SERVER_ERROR); } } } RequestResolveUtil.returnJson(request, reponse, JSONObject.toJSONString(responseData)); } /** * Producer sends message to server * @param request * @param reponse */ @XRequestMapping(SystemConstant.STR_URL_SEND_MESSAGE) public void sendMessage(HttpServletRequest request, HttpServletResponse reponse) { ResponseDataVo responseData = null; String topic = request.getParameter(RequestParamEnum.TOPIC.getParamName()); String message = request.getParameter(RequestParamEnum.MESSAGE.getParamName()); if(StringUtils.isBlank(topic) || StringUtils.isBlank(message)) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "Subject or message is empty"); } else { try { TopicDo topicDo = topicService.findByTopic(topic); if(topicDo == null) { responseData = new ResponseDataVo(ResponseCodeEnum.PARAM_ERROR, "Subject does not exist"); } else { messageService.saveMessage(topic, message); responseData = new ResponseDataVo(ResponseCodeEnum.OK); } } catch (ServiceException e) { log.error("sendMessageException", e); responseData = new ResponseDataVo(ResponseCodeEnum.SERVER_ERROR); } } RequestResolveUtil.returnJson(request, reponse, JSONObject.toJSONString(responseData)); } }
3,CacheController
The consumption form of mq designed is the form of pull. In the form of pull, the client needs to count and consume by itself.
So this cache interface is added here to provide kv storage function. The data is stored in the database.
Please check the code by going to the gitee connection below.
With the above interfaces, you can simply publish and subscribe to messages, of course, manually.
III. related tables
Since in order to share, we have to provide all the information. So here are the associated database table structures.
1. Subject table
CREATE TABLE `gmq_topic` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Primary key', `topic` varchar(255) NOT NULL DEFAULT '' COMMENT 'theme', `create_time` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT 'Creation time', PRIMARY KEY (`id`), UNIQUE KEY `uniq_idx_topic` (`topic`) USING BTREE ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='gmq-Thematic table';
2. Message table
CREATE TABLE `gmq_message` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Primary key', `topic` varchar(255) NOT NULL DEFAULT '' COMMENT 'theme', `message_body` text NOT NULL COMMENT 'Message content', `create_time` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT 'Creation time', PRIMARY KEY (`id`), KEY `idx_id_topic` (`id`,`topic`) USING BTREE ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='Messages stored';
3, kv table
CREATE TABLE `gmq_message` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Primary key', `topic` varchar(255) NOT NULL DEFAULT '' COMMENT 'theme', `message_body` text NOT NULL COMMENT 'Message content', `create_time` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp() COMMENT 'Creation time', PRIMARY KEY (`id`), KEY `idx_id_topic` (`id`,`topic`) USING BTREE ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='Messages stored';
Four, summary
The main function is the above. Looking back, it's not a lot, but there are many twists and turns in the specific implementation process, part of which is the reason for the framework.
So from the last blog to this blog, I not only finished this project, but also fixed some problems in the use of the framework. For example, bigint type data is really BigDecimal, not long.
gmq project mainly includes two parts, the server is described above, and the client will be introduced next.
Finally, attach the code address: https://gitee.com/simpleha/gmq.git