Portal

Getting started with RabbitMQ
RabbitMQ message mode 1

Current restriction at the consumer end

What is the current restriction of the consumer end?

Suppose a scenario, first of all, our RabbitMQ server has tens of thousands of unprocessed messages, and we open a consumer client casually, and the following situation will appear:
Huge amount of messages are pushed all at once, but our single client can't handle so much data at the same time!

The existing business scenario of current restriction at the consumer end

For example, in our case, after the user pays, a message will be sent to MQ Broker. If the 100 elder brother user pays, then there will be 100 messages in MQ Broker (message sending)
The above situation will bring about a problem. When the SMS service has been sent down due to the problem of SMS interface, the 100 messages need to be processed in an instant as soon as the SMS micro service is started. As a result, the SMS micro service will be suspended again

Solutions provided by RabbitMQ for current limiting at the consumer end

RabbitMQ provides a QoS (quality of service) function, that is, on the premise of not automatically confirming messages, if a certain number of messages (by setting the value of QoS based on the Consumer or Channel) are not confirmed, new messages will not be consumed
Void BasicQos(uint prefetchSize, ushort prefetchCount, bool global);
prefetchSize: 0 does not limit message size
 Here's the prefetchSize
 prefetchSize: will tell RabbitMQ not to push more than N messages to a Consumer at the same time, that is, once there are N messages without ack, the Consumer will block until there is ack
 Global: true\false whether to apply the above settings to the Channel; in short, whether the above restrictions are Channel level or Consumer level

Custom consumer

package com.onlyk.api.rabbitmqapi.limit;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;

import java.io.IOException;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 9:46
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class MyConsumer extends DefaultConsumer {
    private Channel channel ;

    public MyConsumer(Channel channel) {
        super(channel);
        this.channel = channel;
    }

    @Override
    public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
        System.err.println("-----------consume message----------");
        System.err.println("consumerTag: " + consumerTag);
        System.err.println("envelope: " + envelope);
        System.err.println("properties: " + properties);
        System.err.println("body: " + new String(body));

        channel.basicAck(envelope.getDeliveryTag(), false);
    }

}

Consumer code

package com.onlyk.api.rabbitmqapi.limit;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 9:47
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Consumer {
    public static void main(String[] args) throws Exception {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();


        String exchangeName = "test_qos_exchange";
        String queueName = "test_qos_queue";
        String routingKey = "qos.#";

        channel.exchangeDeclare(exchangeName, "topic", true, false, null);
        channel.queueDeclare(queueName, true, false, false, null);
        channel.queueBind(queueName, exchangeName, routingKey);

        //1. The first thing about current limiting mode is that autoAck is set to false
        channel.basicQos(0, 1, false);

        channel.basicConsume(queueName, false, new MyConsumer(channel));
    }
}

Production end code

package com.onlyk.api.rabbitmqapi.limit;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 9:48
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Producer {
    public static void main(String[] args) throws Exception {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();

        String exchange = "test_qos_exchange";
        String routingKey = "qos.save";

        String msg = "Hello RabbitMQ QOS Message";

        for(int i =0; i<5; i ++){
            channel.basicPublish(exchange, routingKey, true, null, msg.getBytes());
        }

    }
}

Start the consumer end

Start production side

ACK and return queue of messages

Consumer manual ACK and NACK
When the consumer is consuming, if the business is abnormal, we can record the logs and then make compensation
If there are serious problems such as server downtime, we need to manually perform ACK to ensure the success of consumer consumption!
Consumer's return queue
The purpose of the consumer returning to the queue is to return the message to the Broker for messages that have not been processed successfully!
Generally, we will close the return queue in the actual application, that is to say, it is set to False; because there is a high probability that the return queue message will still fail to be processed!

Custom consumer

package com.onlyk.api.rabbitmqapi.ack;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;

import java.io.IOException;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:04
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class MyConsumer extends DefaultConsumer {
    private Channel channel;
    
    public MyConsumer(Channel channel) {
        super(channel);
        this.channel = channel;
    }
    @Override
    public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
        System.err.println("-----------consume message----------");
        System.err.println("body: " + new String(body));
        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        if ((Integer) properties.getHeaders().get("num") == 0) {
            System.out.println("Simulate the failure of the third-party amount opening and calling method of SMS microservice...");
//            Manually sign in and return to the queue (the third-party interface calls exception handling) and write it in catch
            channel.basicNack(envelope.getDeliveryTag(), false, true);
        } else {
            channel.basicAck(envelope.getDeliveryTag(), false);
        }
    }
}

Consumer code

package com.onlyk.api.rabbitmqapi.ack;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:06
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Consumer {
    public static void main(String[] args) throws Exception {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();


        String exchangeName = "test_ack_exchange";
        String queueName = "test_ack_queue";
        String routingKey = "ack.#";

        channel.exchangeDeclare(exchangeName, "topic", true, false, null);
        channel.queueDeclare(queueName, true, false, false, null);
        channel.queueBind(queueName, exchangeName, routingKey);

        // Manual sign in must close autoAck = false
        channel.basicConsume(queueName, false, new MyConsumer(channel));
    }
}

Production end code

package com.onlyk.api.rabbitmqapi.ack;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.util.HashMap;
import java.util.Map;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:06
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Produces {
    public static void main(String[] args) throws Exception {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();

        String exchange = "test_ack_exchange";
        String routingKey = "ack.save";

        for(int i =0; i<5; i ++){
            Map<String, Object> headers = new HashMap<String, Object>();
            headers.put("num", i);
            AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
                    .deliveryMode(2)
                    .contentEncoding("UTF-8")
                    .headers(headers)
                    .build();
            String msg = "Hello RabbitMQ ACK Message " + i;
            channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
        }
    }
}

Start the consumer end

Start the production side (0 will always resend)

TTL message

TTL is the abbreviation of Time To Live, that is, Time To Live
Effect

RabbitMQ supports message expiration time, which can be specified when sending messages
 RabbitMQ supports the expiration time of the queue. It starts from the time when the message enters the queue. As long as the timeout configuration of the queue is exceeded, the message is automatically cleared

Note: it is mainly for message setting, which has nothing to do with switch, queue and consumer setting
Consumer code

package com.onlyk.api.rabbitmqapi.ttl;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;

import java.util.Map;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:23
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Consumer {
    public static void main(String[] args) throws Exception {

        //1 create a ConnectionFactory and configure it
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        //2 create a connection through a connection factory
        Connection connection = connectionFactory.newConnection();

        //3 create a Channel through connection
        Channel channel = connection.createChannel();

        //4 declare (create) a queue
        String queueName = "test001";
        channel.queueDeclare(queueName, true, false, false, null);

        //5 create consumers
        QueueingConsumer queueingConsumer = new QueueingConsumer(channel);

        //6 set Channel
        channel.basicConsume(queueName, true, queueingConsumer);

        while(true){
            //7 get message
            QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();
            String msg = new String(delivery.getBody());
            System.err.println("Consumer end: " + msg);
            Map<String, Object> headers = delivery.getProperties().getHeaders();
            System.err.println("headers get my1 value: " + headers.get("my1"));
            //Envelope envelope = delivery.getEnvelope();
        }
    }
}

Production end code

package com.onlyk.api.rabbitmqapi.ttl;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.util.HashMap;
import java.util.Map;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:23
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Procuder {
    public static void main(String[] args) throws Exception {
        //1 create a ConnectionFactory and configure it
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        //2 create a connection through a connection factory
        Connection connection = connectionFactory.newConnection();

        //3 create a Channel through connection
        Channel channel = connection.createChannel();

        Map<String, Object> headers = new HashMap<>();
        headers.put("my1", "111");
        headers.put("my2", "222");


        AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
                .deliveryMode(2)
                .contentEncoding("UTF-8")
                .expiration("10000")
                .headers(headers)
                .build();

        //4 send data through Channel
        for(int i=0; i < 5; i++){
            String msg = "Hello RabbitMQ!";
            //1 exchange   2 routingKey
            channel.basicPublish("", "test001", properties, msg.getBytes());
        }
        //5 remember to close the connection
        channel.close();
        connection.close();
    }
}

Start the consumer side and start the production side

It's all right So what's this for? Only start the production end for trial (see the change of console)

Message auto death

Dead letter queue

Dead letter queue: DLX, dead letter exchange
With DLX, when a message becomes dead message in one queue, it can be re publish ed to another Exchange, which is DLX

There are several situations when a message becomes a dead letter

Message rejected (basic.reject/basic.nack) and request = false
 Message TTL expired
 Queue reaches maximum length

Characteristics of dead letter queue

DLX is also a normal Exchange, which is no different from general Exchange. It can be specified on any queue, in fact, it is to set the properties of a queue;
When there is dead message in the queue, RabbitMQ will automatically republish the message to the set Exchange, and then it will be routed to another queue;
You can listen to messages in this queue and process them accordingly. This feature can make up for the function of immediate parameter supported by RabbitMQ3.0;

Dead letter queue settings

First, you need to set Exchange and Queue of dead letter Queue, and then bind:
Exchange: dlx.exchange
Queue: dlx.queue
RoutingKey: #
Then we can declare the switch, queue and binding normally, but we need to add a parameter to the queue:
Arguments.put("x-dead-letter-exchange","dlx.exchange")；
In this way, when the message expires, the request and the queue reach the maximum length, the message can be directly routed to the dead letter queue!

Consumer code

package com.onlyk.api.rabbitmqapi.dlx;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.util.HashMap;
import java.util.Map;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:52
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Consumer {
    public static void main(String[] args) throws Exception {


        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();

        // This is a common switch, queue and route
        String exchangeName = "test_dlx_exchange";
        String routingKey = "dlx.#";
        String queueName = "test_dlx_queue";

        channel.exchangeDeclare(exchangeName, "topic", true, false, null);

        Map<String, Object> agruments = new HashMap<String, Object>();
        agruments.put("x-dead-letter-exchange", "dlx.exchange");
        //This arguments property, to be set on the declaration queue
        channel.queueDeclare(queueName, true, false, false, agruments);
        channel.queueBind(queueName, exchangeName, routingKey);

        //To declare the dead letter queue:
        channel.exchangeDeclare("dlx.exchange", "topic", true, false, null);
        channel.queueDeclare("dlx.queue", true, false, false, null);
        channel.queueBind("dlx.queue", "dlx.exchange", "#");
        channel.basicConsume(queueName, true, new MyConsumer(channel));
    }
}

Production end code

package com.onlyk.api.rabbitmqapi.dlx;

import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

/**
 * @author only Old K I speak for myself
 * @create 2020-03-04 10:52
 * @blogaddress https://blog.csdn.net/weixin_44255950
 */
public class Producer {
    public static void main(String[] args) throws Exception {

        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost("140.143.242.67");
        connectionFactory.setPort(5672);
        connectionFactory.setVirtualHost("/");

        Connection connection = connectionFactory.newConnection();
        Channel channel = connection.createChannel();

        String exchange = "test_dlx_exchange";
        String routingKey = "dlx.save";

        String msg = "Hello RabbitMQ DLX Message";

        for(int i =0; i<1; i ++){

            AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
                    .deliveryMode(2)
                    .contentEncoding("UTF-8")
                    .expiration("10000")
                    .build();
            channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
        }

    }
}

Launch consumer and production

Only start the production side (pay attention to the changes.).. Dlx queue and dlx.queue)