1, Introduction to message oriented middleware

introduce

Message queuing refers to the use of efficient and reliable messaging mechanism for platform independent data exchange, and the integration of distributed systems based on data communication.

Features (functions)

• Application decoupling
• asynchronous communication
• Flow peak clipping
• (massive) log processing
• Message communication
• ......

Application scenario

According to the characteristics of message queue, many scenarios can be derived or used. Here are some examples:

1) Asynchronous communication

SMS and email notification during registration to reduce response time;

2) Application decoupling

The information sender and the message receiver do not need to be coupled, such as calling a third party;

3) Flow peak clipping

For example, second kill system;

2, Comparison of Message Oriented Middleware

1.ActiveMQ

Official website: activemq.apache.org/

Introduction:

ActiveMQ is the most popular and powerful open source message bus produced by Apache. ActiveMQ is a JMS Provider implementation that fully supports the JMS 1.1 and J2EE 1.4 specifications. Although the JMS specification has been issued for a long time, JMS still plays a special role in today's J2EE applications.

characteristic:

1. Support various cross language clients and protocols from Java, C, C + +, C, Ruby, Perl, Python and PHP
2. Fully supports the enterprise integration pattern in JMS client and Message Broker
3. Supports many advanced features such as message groups, virtual targets, wildcards, and composite targets
4. Fully support JMS 1.1 and J2EE 1.4, and support transient, persistent, transaction and XA messages
5. Spring support so that ActiveMQ can be easily embedded into spring applications and configured using spring's XML configuration mechanism
6. Designed for high performance cluster, client server, peer-to-peer communication
7. CXF and Axis support so that ActiveMQ can be easily put into these Web service stacks to provide reliable messaging
8. It can be used as an in memory JMS provider, which is very suitable for unit testing JMS
9. Supports pluggable transport protocols, such as in VM, TCP, SSL, NIO, UDP, multicast, JGroups and JXTA transport
10. Very fast persistence is supported using JDBC and high-performance logging

2.RabbitMQ

Official website: www.rabbitmq.com/

Introduction:

RabbitMQ is an open source implementation of AMQP developed by Erlang language. RabbitMQ is lightweight and easy to deploy in the cloud. It supports a variety of messaging protocols.
RabbitMQ can be deployed in distributed and federated configurations to meet high-scale and high availability requirements. RabbitMQ can run in many operating systems and cloud environments, and provides a wide range of development tools for most popular languages. (translated from official website)

AMQP (Advanced message queue): Advanced message queue protocol. It is an open standard of application layer protocol. It is designed for message oriented middleware. The client and message oriented middleware based on this protocol can transmit messages without being limited by products, development languages and other conditions.

RabbitMQ was initially widely used in the financial industry. According to the description on the official website, it has the following characteristics:

characteristic:

1. Asynchronous message delivery: support multiple message protocols, message queues, transmission confirmation, flexible routing to queues, and multiple exchange types;

2. Support almost all the most popular programming languages: Java, C, C + +, C, Ruby, Perl, Python, PHP and so on;

3. It can be deployed as a cluster with high availability and throughput; Joint across multiple available regions and regions;

4. Pluggable authentication, authorization, support TLS and LDAP.;

5. It provides an easy-to-use user interface, so that users can monitor and manage many aspects of message Broker;

6. It provides many plug-ins to extend from many aspects, and you can also write your own plug-ins.

7. Kafka

Official website: kafka.apache.org/

Introduction:

Kafka is an open source stream processing platform developed by the Apache Software Foundation and written by Scala and Java. Kafka is a high-throughput distributed publish subscribe message system, which can process all action flow data in consumer scale websites.
This action (web browsing, search and other user actions) is a key factor in many social functions on the modern network.
These data are usually solved by processing logs and log aggregation due to throughput requirements.
This is a feasible solution for the log data and offline analysis system like Hadoop, but it requires real-time processing. Kafka aims to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time messages through clusters.

Kafka is mainly used to process active streaming data, so Kafaka is more used in big data system.

characteristic:

1. High throughput for both publish and subscribe. It is understood that Kafka can produce about 250000 messages (50 MB) per second and process 550000 messages (110 MB) per second.
2. Persistent operation is available. Persistent messages to disk, so they can be used for mass consumption, such as ETL, and real-time applications. Prevent data loss by persisting data to the hard disk and replication.
3. Distributed system, easy to expand outward. There will be multiple producer s, broker s and consumer s, all of which are distributed. The machine can be extended without shutdown.
4. The status of message processing is maintained on the consumer side, not on the server side. It can balance automatically when it fails.
5. Support online and offline scenarios.

6. RocketMQ

Official website: rocketmq.apache.org/

Introduction:

RocketMQ is an open-source message middleware from Alibaba. It is currently incubated in Apache and developed in pure Java. It has the characteristics of high throughput, high availability and suitable for large-scale distributed system applications. The idea of RocketMQ originated from Kafka, but it is not a simple copy. It optimizes the reliable transmission and transaction of messages. At present, it is widely used in trading, recharge, stream computing, message push, log streaming, binglog distribution and other scenarios in Alibaba group, supporting many double 11 activities in Alibaba.

characteristic:

1. Support publish / subscribe (Pub/Sub) and peer-to-peer (P2P) message models
2. Reliable first in first out (FIFO) and strict sequential delivery in a queue
3. It supports two message modes: pull and push
4. Accumulation capacity of millions of messages in a single queue
5. Support a variety of message protocols, such as JMS, MQTT, etc
6. The distributed high availability deployment architecture meets at least one message passing semantics
7. Provide docker images for isolation testing and cloud cluster deployment
8. Provide a Dashboard with rich functions such as configuration, indicators and monitoring

3, Installation of ActiveMQ

1. Installation steps

activemq has corresponding installation packages under each system. Use the following to demonstrate the installation of activemq under Linux system.

Enter apache-activemq-5.15.8/bin directory and start activemq./activemq start

The above information is output, indicating that the startup is successful.
2. Problems encountered during installation

During installation, check the running status of activemq,

Display above.

View the running log through the. / bin/activemq console command:

The host name contains illegal characters;

Then the solution is very simple. Change the host name:

1. Method 1: use the hostnamectl command

Hostnamectl set hostname host name

2. Method 2: modify the configuration file / etc/hostname, save and exit

After modification, restart. Here I use method 1:

hostnamectl set-hostname activemq

View running status:

5, ActiveMQ page introduction

After ActiveMQ is installed and started, visit http://ip:8161/admin , the login name and password are admin (modifiable in the configuration file). Enter the ActiveMQ home page:

Here are the functions of each menu:
1.Queue message queue page

Name: the name of the message queue.

Number Of Pending Messages: the number of messages not consumed.

Number Of Consumers: the Number Of Consumers.

Messages Enqueued: messages entering the queue; The total number of messages entering the queue, including those that have been consumed and those that have not been consumed. This number only increases.

Messages Dequeued: messages out of the queue can be understood as the number of messages consumed. In Queues, it is equal to the total number of messages entering the queue (because a message will only be successfully consumed once). If it is temporarily unequal, it is because consumers have not had time to consume.

2.Topic topic page

Name: subject name.

Number Of Pending Messages: the number of messages not consumed.

Number Of Consumers: the Number Of Consumers.

Messages Enqueued: messages entering the queue; The total number of messages entering the queue, including those that have been consumed and those that have not been consumed. This number only increases.

Messages Dequeued: messages out of the queue can be understood as the number of messages consumed. In Topics, because there are many consumers, the number will be higher than the number in the queue.

3.Subscribers view subscriber page

To view subscriber information, this page only has data in Topics message type.
4.Connections page

6, Simple use

Import jar package:

    <dependency>
        <groupId>org.apache.activemq</groupId>
        <artifactId>activemq-core</artifactId>
        <version>5.7.0</version>
    </dependency>

1. Peer to peer (P2P) model

The point-to-point model uses queue as the message carrier. In this mode, a message can only be consumed by one consumer. If it is not consumed, it can only stay in the queue, wait to be consumed, or timeout. For example, if there are 10 messages in the queue and there are two consumers, one consumer consumes 5 messages, one for you and one for me. The following is a code demonstration.

Publisher:

public static void main(String[] args) throws JMSException {
    /*
     * Implementation steps
     * 1.To create a ConnectionFactory factory object, you need to fill in the user name, password and connection address (usually the default, if not modified)
     * 2.Create a Connection through the ConnectionFactory object, and call the start method of the Connection to open the Connection. The Connection method is closed by default
     * 3.A Session session (context object) is created through the Connection object to receive messages. Parameter 1 is whether to enable transactions, and parameter 2 is the sign in mode. It is generally set to automatic sign in
     * 4.The Destination object is created through the Session object, which refers to the object used by a client to formulate the production message target and consumption message source. In PTP mode, Destination is called queue, and in Pub/Sub mode, Destination is called Topic
     * 5.Create sending and receiving objects (producers and consumers) of messages through the Session object
     * 6.Set the persistent or non persistent characteristics of MessageProducer through its setDeliverMode method
     * 7.Use the TextMessage form of JMS specification to create data (through the Session object), and send the data with the send method of MessageProducer. The same applies to clients. Remember to close
     */
    ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnectionFactory.DEFAULT_USER,
            ActiveMQConnectionFactory.DEFAULT_PASSWORD,"tcp://94.191.49.192:61616");
    Connection connection = connectionFactory.createConnection();
    connection.start();
    Session session = connection.createSession(Boolean.FALSE,Session.AUTO_ACKNOWLEDGE);
    Destination destination = session.createQueue("queue");
    MessageProducer producer = session.createProducer(destination);
    producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
    for (int i=0;i<=5;i++) {
        TextMessage textMessage = session.createTextMessage();
        textMessage.setText("I'm the third"+i+"news");
        producer.send(textMessage);
    }
    if(connection!=null){
        connection.close();
    }
}

Message consumer:

public static void main(String[] args) throws JMSException {
        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnectionFactory.DEFAULT_USER,
                ActiveMQConnectionFactory.DEFAULT_PASSWORD,"tcp://94.191.49.192:61616");
        Connection connection = connectionFactory.createConnection();
        connection.start();
        Session session = connection.createSession(Boolean.FALSE,Session.AUTO_ACKNOWLEDGE);
        Destination destination = session.createQueue("queue");
        MessageConsumer consumer = session.createConsumer(destination);
        while (true){
            TextMessage message = (TextMessage) consumer.receive();
            if (message==null){
                break;
            }
            System.out.println(message.getText());
        }
        if(connection!=null){
            connection.close();
        }
    }

Start two consumers first, and then start the publisher:


2. Publish / subscribe (Pub/Sub) model

The publish / subscribe model uses topic as the message communication carrier. This model is similar to the official account of WeChat. The publisher publishes a message and then passes the message to all subscribers. Note: subscribers who want to receive this information must subscribe before the information is published.

Publisher release information:

public static void main(String[] args) throws JMSException, IOException {
        // Create a ConnectionFactory object to connect to the MQ server
        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://94.191.49.192:61616");
        // Create a connection object
        Connection connection;
        connection = connectionFactory.createConnection();
        // Open connection
        connection.start();
        // Create a Session object using the Connection object
        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
        // Create a Destination object. topic object
        Topic topic = session.createTopic("test-topic");
        // Create a consumer object using the Session object.
        MessageConsumer consumer = session.createConsumer(topic);
        // receive messages
        consumer.setMessageListener(new MessageListener() {

            @Override
            public void onMessage(Message message) {
                // Print results
                TextMessage textMessage = (TextMessage) message;
                String text;
                try {
                    text = textMessage.getText();
                    System.out.println("This is the message received:" + text);
                } catch (JMSException e) {
                    e.printStackTrace();
                }

            }
        });
        System.out.println("topic Consumers start....");
        // Waiting to receive message
        System.in.read();
        // close resource
        consumer.close();
        session.close();
        connection.close();
    }

Subscriber subscription information:

 public static void main(String[] args) throws JMSException {
        // 1. To create a connection factory object, you need to specify the ip and port of the service.
        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://94.191.49.192:61616");
        // 2. Create a Connection object using a factory object.
        Connection connection = connectionFactory.createConnection();
        // 3. Open the Connection and call the start method of the Connection object.
        connection.start();
        // 4. Create a Session object.
        // The first parameter: whether to start the transaction. If true starts the transaction, the second parameter is meaningless. Generally, transactions are not enabled. false.
        // Second parameter: reply mode. Automatic answer or manual answer. General automatic response.
        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
        // 5. Use the Session object to create a Destination object. There are two forms: queue and topic. Topic should be used now
        Topic topic = session.createTopic("test-topic");
        // 6. Use the Session object to create a Producer object.
        MessageProducer producer = session.createProducer(topic);
        // 7. Create a Message object, you can use TextMessage.
        for (int i = 0; i < 50; i++) {
            TextMessage textMessage = session.createTextMessage("The first" + i + "One ActiveMQ Message queue destination");
            // 8. Send message
            producer.send(textMessage);
        }
        // 9. Close resource
        producer.close();
        session.close();
        connection.close();
    }

The subscriber needs to subscribe in advance, so run the subscriber first.

3. Comparison of the two modes

1) From the above, we can summarize the implementation steps of ActiveMQ:

• To create a ConnectionFactory object, you need to fill in the user name, password and connection address

• Create a Connection through the ConnectionFactory object

• Create a Session through the Connection object

• Create a Destination object through the Session object; In P2P mode, Destination is called Queue, and in Pub/Sub mode, Destination is called Topic

• Create sending and receiving objects of messages through the Session object

• send message

• close resource

2) It can be seen that the implementation difference between P2P mode and Pub/Sub mode is that the Destination object created through Session is different. In P2P mode, Destination is called Queue, and in Pub/Sub mode, Destination is called Topic