Preface
Last document Zookeeper implements centralized management of parameters This paper introduces how to use Zookeeper to monitor and notify nodes to simply realize centralized management of parameters. In fact, JMS publishing and subscribing mechanism can also achieve similar functions. Cluster nodes can update specified parameters by subscribing to designated nodes and using JMS to filter messages. This paper will introduce how to realize simple centralized management of parameters through JMS.
Introduction of Maven
Spring related jar introduction refers to the previous article
<dependency> <groupId>javax.jms</groupId> <artifactId>jms</artifactId> <version>1.1</version> </dependency> <dependency> <groupId>org.apache.activemq</groupId> <artifactId>activemq-all</artifactId> <version>5.10.0</version> </dependency>
target
1. Multiple nodes such as / app1,/app2 can be monitored simultaneously.
2. It is hoped that only configuration such as / app1 can monitor its sub-nodes such as / app1/modual1 and sub-nodes such as / app1/modual1/xxx /... ;
3. Server startup can obtain all the data of child nodes under the current specified parent node.
4. Dynamic notification when adding nodes or updating node data, so that the latest data can be obtained in real time in the code;
5. In spring configuration, parameters can be read from Zookeeper for initialization.
Although there are some differences in the way they are achieved, the final goals are the same. These five goals are listed as well.
Realization
MQWatcher is mainly used to establish connections with JMS, subscribe to designated nodes, establish point-to-point connections, filter out data to be monitored, update data, initialize data, store data, etc.
InitConf Server is mainly used as the server side of point-to-point connection to initialize data.
1. Configuring multiple nodes to listen at the same time
Provides a string array for users to add nodes to listen on:
private String[] keyPatterns;
2. Ability to listen on its children and their children
Unlike Zookeeper, JMS sends all data changes to subscribers, who then filter out the data they need to update.
/** MQ Filter**/ private StringBuffer keyFilter = new StringBuffer(); private final String TOPIC = "dynamicConfTopic"; private void watcherPaths() throws JMSException { Topic topic = session.createTopic(TOPIC); MessageConsumer consumer = session.createConsumer(topic, keyFilter.toString()); consumer.setMessageListener(new MessageListener() { @Override public void onMessage(Message message) { try { String key = message.getStringProperty(IDENTIFIER); TextMessage tm = (TextMessage) message; keyValueMap.put(key, tm.getText()); LOGGER.info("key = " + key + ",value = " + tm.getText()); } catch (JMSException e) { LOGGER.error("onMessage error", e); } } }); }
TOPIC is subscribed to and the filter keyFilter is specified, which is assembled based on keyPatterns.
private final String IDENTIFIER = "confKey"; /** * Generating Receiver Filter */ private void generateKeyFilter() { for (int i = 0; i < keyPatterns.length; i++) { keyFilter.append(IDENTIFIER + " LIKE '" + keyPatterns[i] + "%'"); if (i < keyPatterns.length - 1) { keyFilter.append(" OR "); } } LOGGER.info("keyFilter : " + keyFilter.toString()); }
The specified attribute IDENTIFIER is filtered through LIKE and OR keywords
3. Server startup initialization node data
Initialization data is acquired through request response mode at server startup through point-to-point mode
private final String QUEUE = "dynamicConfQueue"; /** * Initialize key-value values * * @throws JMSException */ private void initKeyValues() throws JMSException { TemporaryQueue responseQueue = null; MessageProducer producer = null; MessageConsumer consumer = null; Queue queue = queueSession.createQueue(QUEUE); TextMessage requestMessage = queueSession.createTextMessage(); requestMessage.setText(generateKeyString()); responseQueue = queueSession.createTemporaryQueue(); producer = queueSession.createProducer(queue); consumer = queueSession.createConsumer(responseQueue); requestMessage.setJMSReplyTo(responseQueue); producer.send(requestMessage); MapMessage receiveMap = (MapMessage) consumer.receive(); @SuppressWarnings("unchecked") Enumeration<String> mapNames = receiveMap.getPropertyNames(); while (mapNames.hasMoreElements()) { String key = mapNames.nextElement(); String value = receiveMap.getStringProperty(key); keyValueMap.put(key, value); LOGGER.info("init key = " + key + ",value = " + value); } }
By specifying the QUEUE request, a temporary response QUEUE is created, and then a MapMessage is accepted to initialize the keyValueMap.
4. Monitor changes in node data
By publishing and subscribing, accepting all data and filtering, Goal 2 has been implemented.
5. In spring configuration, parameters can be read from Zookeeper for initialization
public class MQPropPlaceholderConfigurer extends PropertyPlaceholderConfigurer { private MQWatcher mqwatcher; @Override protected Properties mergeProperties() throws IOException { return loadPropFromMQ(super.mergeProperties()); } /** * Loading configuration constants from MQ * * @param result * @return */ private Properties loadPropFromMQ(Properties result) { mqwatcher.watcherKeys(); mqwatcher.fillProperties(result); return result; } }
With the above processing, the following simple configurations can be used to achieve the goal:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd"> <bean id="person" class="zh.maven.DynamicConf.Person"> <property name="name"> <value>${/a2/m1}</value> </property> <property name="address"> <value>${/a3/m1/v2}</value> </property> <property name="company"> <value>${/a3/m1/v2/t2}</value> </property> </bean> <bean id="mqwatcher" class="zh.maven.DynamicConf.mq.MQWatcher"> <property name="keyPatterns" value="/a2,/a3" /> </bean> <bean id="propertyConfigurer" class="zh.maven.DynamicConf.mq.MQPropPlaceholderConfigurer"> <property name="mqwatcher" ref="mqwatcher"></property> </bean> </beans>
test
1. Start ActiveMQ
activemq.bat
2. InitConf Server startup
Used to listen for initialization requests of cluster nodes, get keyPatterns sent by cluster nodes, and then encapsulate data conforming to their patterns as MapMessage to send to cluster nodes.
@Override public void onMessage(Message message) { try { TextMessage receiveMessage = (TextMessage) message; String keys = receiveMessage.getText(); LOGGER.info("keys = " + keys); MapMessage returnMess = session.createMapMessage(); returnMess.setStringProperty("/a2/m1", "zhaohui"); returnMess.setStringProperty("/a3/m1/v2", "nanjing"); returnMess.setStringProperty("/a3/m1/v2/t2", "zhaohui"); QueueSender sender = session.createSender((Queue) message.getJMSReplyTo()); sender.send(returnMess); } catch (Exception e) { LOGGER.error("onMessage error", e); } }
The above code is just a simple simulation, providing a way of thinking.
3. Start the Main class
public class Main { public static void main(String[] args) throws Exception { ApplicationContext context = new ClassPathXmlApplicationContext(new String[] { "spring-config.xml" }); Person person = (Person) context.getBean("person"); System.out.println(person.toString()); } }
4. Start Topic Publisher
Publish data regularly while viewing the Main class log output of cluster nodes
public class TopicPublisher { private static final String TOPIC = "dynamicConfTopic"; private static final String IDENTIFIER = "confKey"; public static void main(String[] args) throws JMSException { ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616"); Connection connection = factory.createConnection(); connection.start(); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); Topic topic = session.createTopic(TOPIC); MessageProducer producer = session.createProducer(topic); producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT); int i=1; while (true) { TextMessage message = session.createTextMessage(); message.setStringProperty(IDENTIFIER, "/a2/"+i); message.setText("message_" + System.currentTimeMillis()); producer.send(message); System.out.println("Sent message: " + message.getText()); try { Thread.sleep(2000); } catch (InterruptedException e) { e.printStackTrace(); } i++; } } }
The log output is as follows:
2017-08-14 21:52:23 - keyFilter : confKey LIKE '/a2%' OR confKey LIKE '/a3%' 2017-08-14 21:52:24 - init key = /a3/m1/v2/t2,value = zhaohui 2017-08-14 21:52:24 - init key = /a3/m1/v2,value = nanjing 2017-08-14 21:52:24 - init key = /a2/m1,value = zhaohui 2017-08-14 21:52:24 - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@223dd567: defining beans [person,mqwatcher,propertyConfigurer]; root of factory hierarchy name = zhaohui,address = nanjing,company = zhaohui 2017-08-14 21:52:33 - key = /a2/1,value = message_1502718753819 2017-08-14 21:52:35 - key = /a2/2,value = message_1502718755832 2017-08-14 21:52:37 - key = /a2/3,value = message_1502718757846 2017-08-14 21:52:39 - key = /a2/4,value = message_1502718759860 2017-08-14 21:52:41 - key = /a2/5,value = message_1502718761876
Detailed code svn address: http://code.taobao.org/svn/temp-pj/DynamicConf
summary
Through JMS, a simple parametric platform system has been implemented. Of course, there are still many areas to be optimized in production. This paper provides a way of thinking; there is time to prepare for DynamicConf to provide a more perfect solution.