Instance Analysis
Previously, we have read the documentation of AMQP, and have a general understanding of AMQP. This paper will go through the basic operation of AMQP from an example.
Get ready
Environmental Science
RabbitMQ server 3.7.16
RabbitMQ client 5.7.3
The client code uses the RabbitMQ tutorial as follows:
public static void main(String[] args) throws IOException, TimeoutException { ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); try (Connection connection = factory.newConnection(); Channel channel = connection.createChannel();) { boolean durable = true; channel.queueDeclare(QUEUE_NAME, durable, false, false, null); String message = String.join(" ", "dMessage......."); channel.exchangeDeclare("mind", "direct"); channel.basicPublish("mind", "", MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes()); System.out.println(" [x] Sent '" + message + "'"); } }
Here's the result of capturing the package with wireshark
image
We'll look at this later in detail by encoding No
Package Grabbing Analysis
image
1-6 is a three-time handshake step for tcp to create connections, not too much analysis here
image
7-24 is the process by which amqp creates a connection, which we can analyze against the documentation in the previous blog. Each time one end sends a message to the other end, the other end sends an ack to indicate it receives it.
image
1 After a tcp connection is created, the client will send protocol version information to the server. This is version 0.9.1 of amqp. The server will verify whether the version is accepted or not. If it does not meet the requirements, it will return error information. Here is only the correct information. We can implement an error example later when we implement the client ourselves.
image
2 After the service side verifies the protocol, it sends the request to create the connection Connection.Start to the client, and the client returns a Connection.Start-Ok when it is ready. Then the service side sends Connection.Tune to debug the parameters with the client, which has the maximum number of Channel s.Maximum Frame Length etc. Client sends Connection.Tune-OK after debugging. This phase is debugging the parameters of the connection.
image
After debugging the 3 parameters, the client requests the server to open the connection Connection.Open. After the server opens, it returns to Connection.Open-Ok. Connection opens successfully. After the client requests to open the channel Channel.Open, after the server opens, it returns to Channel.Open-Ok. The connection is created successfully.
image
4 After successful connection creation, the client makes a statement about queue and exchange, Queue.Declare -> Queue.Declare-Ok, Exchange.Declare -> Exchange.Declare-Ok.
image
5 With Exchange, clients send messages to Exchange, and we can see what Exchange is sending, and what it is sending
image
image
6 After sending the content, the client closes, closes the channel Channel, and then closes the Connection.
image
7 Finally, tcp closes the connection
code analysis
Here's a code-level analysis of the process, and here's an overall time series for you to refer to
image
Let's still analyze the code in the order we see it in the package
Create a tcp connection
The code is simple
ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); Connection connection = factory.newConnection();
We focused on factory.newConnection(); following this approach we quickly found the init() method for AutorecoveringConnection
public void init() throws IOException, TimeoutException { this.delegate = this.cf.newConnection(); this.addAutomaticRecoveryListener(delegate); }
Focus on this.cf.newConnection()
FrameHandler frameHandler = factory.create(addr, connectionName()); RecoveryAwareAMQConnection conn = createConnection(params, frameHandler, metricsCollector); conn.start(); metricsCollector.newConnection(conn); return conn;
You can see from debug code that factory is an instance of SocketFrameHandlerFactory, so the code in create is as follows:
public FrameHandler create(Address addr, String connectionName) throws IOException { String hostName = addr.getHost(); int portNumber = ConnectionFactory.portOrDefault(addr.getPort(), ssl); Socket socket = null; try { socket = createSocket(connectionName); configurator.configure(socket); socket.connect(new InetSocketAddress(hostName, portNumber), connectionTimeout); return create(socket); } catch (IOException ioe) { quietTrySocketClose(socket); throw ioe; } }
Here we can see the underlying code Socket of the java network.
socket.connect(new InetSocketAddress(hostName, portNumber), connectionTimeout);
This code completes the creation of the tcp connection.
(When you're ready to look at the source code here, you think there must be a place to do this, but you just can't find it, the last bit debug found it...)
Create Connection
At the end of the previous step, we encapsulated the socket object into a FrameHandler instance, from which we can guess that the communication of all subsequent messages cannot be separated from this FrameHandler.
Let's go on and look back
FrameHandler frameHandler = factory.create(addr, connectionName()); RecoveryAwareAMQConnection conn = createConnection(params, frameHandler, metricsCollector); conn.start();
A Connection object is constructed using the FrameHandler instance, and then the start() method is called. The parent AMQConnection method is actually called, which is also the focus of the whole connection process.
The code here is longer, let's pick some important points to look at
initializeConsumerWorkService(); // Initialize worker threads initializeHeartbeatSender(); // Initialize Heartbeat Thread
// Make sure that the first thing we do is to send the header, // which should cause any socket errors to show up for us, rather // than risking them pop out in the MainLoop // Make sure the header we initially sent doesn't appear in MainLoop when an error occurs // This entity is designed to receive the Connection.Start method of the server after it has been sent to the server version AMQChannel.SimpleBlockingRpcContinuation connStartBlocker = new AMQChannel.SimpleBlockingRpcContinuation(); // We enqueue an RPC continuation here without sending an RPC // request, since the protocol specifies that after sending // the version negotiation header, the client (connection // initiator) is to wait for a connection.start method to // arrive. // We don't get a response here by sending a request because the server actively sends the message when it receives the version information _channel0.enqueueRpc(connStartBlocker);
Inside enqueueRpc is waiting in a loop for the server information to receive a successful notification
private void doEnqueueRpc(Supplier<RpcWrapper> rpcWrapperSupplier) { synchronized (_channelMutex) { boolean waitClearedInterruptStatus = false; while (_activeRpc != null) { try { _channelMutex.wait(); // Notify later when the Connection.Start method is received } catch (InterruptedException e) { //NOSONAR waitClearedInterruptStatus = true; // No Sonar: we re-interrupt the thread later } } if (waitClearedInterruptStatus) { Thread.currentThread().interrupt(); } // Update entity information when notification is obtained _activeRpc = rpcWrapperSupplier.get(); } }
_frameHandler.sendHeader(); //Send version information, corresponding to snap package 7
this._frameHandler.initialize(this); //initialize, basically starts a MainLoop thread to get server-side information
Core code for MainLoop thread
Frame frame = _frameHandler.readFrame(); readFrame(frame);
The internal code of _frameHandler.readFrame() is as follows. Here you can see the details of the 2.3.5 Frame Details frame in the translation. Contrast how the client is constructed, the frame structure is as follows
image
public static Frame readFrom(DataInputStream is) throws IOException { int type; int channel; try { type = is.readUnsignedByte(); // Type information for a byte } catch (SocketTimeoutException ste) { // System.err.println("Timed out waiting for a frame."); return null; // failed } if (type == 'A') { // Processing here, if the server does not support the client version, it will send the supported version information, starting with'A' /* * Probably an AMQP.... header indicating a version * mismatch. */ /* * Otherwise meaningless, so try to read the version, * and throw an exception, whether we read the version * okay or not. */ protocolVersionMismatch(is); // This throws an exception } channel = is.readUnsignedShort(); // channel number of two bytes int payloadSize = is.readInt(); // 4 bytes payload size byte[] payload = new byte[payloadSize]; is.readFully(payload); // Read payloadSize size bytes int frameEndMarker = is.readUnsignedByte(); // Tail of a byte if (frameEndMarker != AMQP.FRAME_END) { throw new MalformedFrameException("Bad frame end marker: " + frameEndMarker); } // Construct object and return return new Frame(type, channel, payload); }
The last step is mainly about encapsulating information, and the following is how clients handle encapsulated objects
private void readFrame(Frame frame) throws IOException { if (frame != null) { _missedHeartbeats = 0; if (frame.type == AMQP.FRAME_HEARTBEAT) { // Ignore it: we've already just reset the heartbeat counter. } else { if (frame.channel == 0) { // the special channel 0 channel is used during connection creation _channel0.handleFrame(frame); // This step is to place the Connection.Start content in the entity that channel set up ahead of time } else { if (isOpen()) { // If we're still _running, but not isOpen(), then we // must be quiescing, which means any inbound frames // for non-zero channels (and any inbound commands on // channel zero that aren't Connection.CloseOk) must // be discarded. ChannelManager cm = _channelManager; if (cm != null) { ChannelN channel; try { channel = cm.getChannel(frame.channel); } catch(UnknownChannelException e) { // this can happen if channel has been closed, // but there was e.g. an in-flight delivery. // just ignoring the frame to avoid closing the whole connection LOGGER.info("Received a frame on an unknown channel, ignoring it"); return; } channel.handleFrame(frame); } } } } } else { // Socket timeout waiting for a frame. // Maybe missed heartbeat. handleSocketTimeout(); } }
Let's go back to the start() method, get the Connection.Start method, and set some parameters for the service to be sent individually
connStart = (AMQP.Connection.Start) connStartBlocker.getReply(handshakeTimeout/2).getMethod();
Then follow the Start.Ok, Tune method, corresponding to Snap Pack 9-16
do { Method method = (challenge == null) ? new AMQP.Connection.StartOk.Builder() .clientProperties(_clientProperties) .mechanism(sm.getName()) .response(response) .build() : new AMQP.Connection.SecureOk.Builder().response(response).build(); try { Method serverResponse = _channel0.rpc(method, handshakeTimeout/2).getMethod(); if (serverResponse instanceof AMQP.Connection.Tune) { connTune = (AMQP.Connection.Tune) serverResponse; } else { challenge = ((AMQP.Connection.Secure) serverResponse).getChallenge(); response = sm.handleChallenge(challenge, username, password); } } catch (ShutdownSignalException e) { Method shutdownMethod = e.getReason(); if (shutdownMethod instanceof AMQP.Connection.Close) { AMQP.Connection.Close shutdownClose = (AMQP.Connection.Close) shutdownMethod; if (shutdownClose.getReplyCode() == AMQP.ACCESS_REFUSED) { throw new AuthenticationFailureException(shutdownClose.getReplyText()); } } throw new PossibleAuthenticationFailureException(e); } } while (connTune == null);
Get debugging information and set local parameters
int channelMax = negotiateChannelMax(this.requestedChannelMax, connTune.getChannelMax()); _channelManager = instantiateChannelManager(channelMax, threadFactory); int frameMax = negotiatedMaxValue(this.requestedFrameMax, connTune.getFrameMax()); this._frameMax = frameMax; int heartbeat = negotiatedMaxValue(this.requestedHeartbeat, connTune.getHeartbeat());
SetHeartbeat; starts the heartbeat thread
Send tuned method TuneOk and request to open connection Open
_channel0.transmit(new AMQP.Connection.TuneOk.Builder() .channelMax(channelMax) .frameMax(frameMax) .heartbeat(heartbeat) .build()); _channel0.exnWrappingRpc(new AMQP.Connection.Open.Builder() .virtualHost(_virtualHost) .build());
The connection to this Connection has been created and opened
Create Channel
Next comes the creation of Channel, which is special for creating Connection s in our previous code. The Channel below is created for sending queue messages later.
Channel channel = connection.createChannel()//entry
According to the AMQP document, creating a Channel requires the client to send the Channel.Open method and then receive the Channel.OpenOk from the server, which we can also see from the snapping package.We track the code step by step, at a deeper level, and here we give the call logic, from bottom to top (yes, that's the part of creating the Channel error log intercepted).
com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:295) com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:141) com.rabbitmq.client.impl.ChannelN.open(ChannelN.java:133) com.rabbitmq.client.impl.ChannelManager.createChannel(ChannelManager.java:182) com.rabbitmq.client.impl.AMQConnection.createChannel(AMQConnection.java:555) com.rabbitmq.client.impl.recovery.AutorecoveringConnection.createChannel(AutorecoveringConnection.java:165)
Let's look at the code for privateRpc
private AMQCommand privateRpc(Method m) throws IOException, ShutdownSignalException{ SimpleBlockingRpcContinuation k = new SimpleBlockingRpcContinuation(m); rpc(m, k); // Send Channel.Open Method // At this point, the request method has been sent, and we // should wait for the reply to arrive. // Here we have sent a request and we should wait for a response // Calling getReply() on the continuation puts us to sleep // until the connection's reader-thread throws the reply over // the fence or the RPC times out (if enabled) // Calling the getReply() method will block until the result is obtained or the time-out is reached if(_rpcTimeout == NO_RPC_TIMEOUT) { return k.getReply(); } else { try { return k.getReply(_rpcTimeout); } catch (TimeoutException e) { throw wrapTimeoutException(m, e); } } }
The receive Channel.OpenOk method is done by the MainLoop thread in a similar manner to the previous get Connection.Start method.
message sending
The AMQP connection has been completely created at this point. Here is the message queue correlation. First, the declaration of the queue and Exchange, where the declaration of the queue is actually useless. The code is written to see the declaration process
channel.queueDeclare(QUEUE_NAME, durable, false, false, null); channel.exchangeDeclare("mind", "direct"); channel.basicPublish("mind", "", MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes());
The way this is declared is very simple. It's easy to follow the code. Queue and Exchange are basically the same declaring process, except that queue checks the validity (length) of queues before they are declared. They get the response results in the same way that Channel.OpenOk does.
The process of sending a message is also to send an AMQPCommand, but there are many details, which are ready to be looked at later in the section that implements the client.
Close Connection
At the end of the program execution, execute the try-with-resources section, automatically execute the close() method, executing from bottom to top, that is, close () of Channel first, then close () of Connection; you can also see from the package that Channel.close method is sent first, then Connection.close method is sent. The code details will not be expanded here, but will be put on the later code implementation.
summary
After going through the main process as a whole, we will implement a simple client to understand it better. In addition to understanding the client's operation process, we also learned some knowledge of java.
When try-with-resources is closed, the closing order is the opposite of the declaring order.
Try-with-resources can also have catch and finally blocks, which are executed after the try-with-resources declaration is closed.
java thread state flow
image
Client implementation (to be completed~)
Today our goal is to implement the rabbitmq client and use it to send messages to the specified Exchange.
tcp connection creation
Super Simple
socket = new Socket(); socket.connect(new InetSocketAddress(host, port)); // Save Connected Input and Output Streams inputStream = new DataInputStream(new BufferedInputStream(socket.getInputStream())); outputStream = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
Grab a bag
image
Send Header
We know by picking up the package and the source that the sender sends "AMQP0091"
private int major = 0; private int minor = 9; private int revision = 1;
outputStream.write("AMQP".getBytes()); outputStream.write(0); outputStream.write(major); outputStream.write(minor); outputStream.write(revision); outputStream.flush();
Package Grabbing Result
image
You can see that the server has approved the protocol and sent the Connection.Start method over.
If the protocol servers we send don't know what's going to happen, let's try major for two
Package Grabbing Result
image
Look at the following for yourself
image
We sent 0291, the package is supporting the AMQP protocol, so it should not be known here, so it appears as unknown version, but what I do not understand is that the result returned by the server is also unknown version. According to the instructions in the AMQP document, the server should return to the supported protocol at this time. Let's look at it.
image
It's true that 0091 is a normal protocol, but the package grabbing software doesn't show up, which is strange~