Problems with AMQ under Servlet 3.0

Keywords: Apache Tomcat xml Jetty

Question 1: http status 500

Description:

When the project switched to Tomcat 7, there was a problem with message push implemented by amq. Most of the requests sent by amq.js were 500, only a few were 200.

Analysis:

Under Servlet 2.5 container, the servlet component supports asynchronous communication by default, but it needs to be opened manually by Servlet 3.0. After investigation, it was found that the configuration parameter [async-supported] was used to turn on this asynchronous support.

Solve:

1. At first, it was configurated only in the back-end Servlet component of amq [Ajax Servlet], and 500 errors were found. Originally, as long as the request passes through the Filter, Servlet need to add configuration.
2. Simple and rude point: All Filters and Servlet s in web.xml add [async-supported] support. Or find the Filter chain for the request and add the corresponding Filter chain.
<servlet>
		<servlet-name>AjaxServlet</servlet-name>
		<servlet-class>org.apache.activemq.web.AjaxServlet</servlet-class>
		<load-on-startup>1</load-on-startup>
		<async-supported>true</async-supported>
	</servlet>

Question 2: Amq request timeout

Description:

After resolving the first problem, amq requests often run out of time in actual use, and js errors (Server connection drop.) cause background push messages not to be updated in real time, but occasionally very normal.

Analysis:

The first reaction after seeing the problem is whether Apache's request timeout time is too short, but looking at Apache's timeout parameter and amq's timeout parameter, it is obvious that the former is larger than the latter. It's reasonable that the request won't reach Apache's timeout and be rejected. With this in mind, first look at the implementation of the front-end component of AMQ and find that it was built before the request, as shown in the following code:
var sendPoll = function(reCon) {
		if (reCon) {
			reConnect = reCon;
		}
		// Workaround IE6 bug where it caches the response
		// Generate a unique query string with date and random
		var now = new Date();
		var timeoutArg = sessionInitialized ? timeout : 0.001;
		var data = 'timeout=' + timeoutArg * 1000
				 + '&d=' + now.getTime()
				 + '&r=' + Math.random();
		var successCallback = sessionInitialized ? pollHandler : initHandler;

		var options = { 
			method: 'get',
			data: addClientId( data ),
			success: successCallback,
			error: pollErrorHandler
		};
			
		adapter.ajax(uri, options);
	};
The debugging tool intercepts requests and finds that the value of the parameter timeout is 1, which means that the first connection is a failure, not to mention the subsequent requests.

At this point, we need to look at the implementation of Ajax Servlet. Let's start with a continuation mechanism, which is based on NIO and allows for suspend and rsueme. Before suspend, you usually need to set timeout to set the blocking time, and the "Continuation" object will provide the corresponding listening object to handle whether the event is "timeout" or "completed". In Servlet 3.0 environment, the Continuation in MessageServlet Support uses Servlet 3 Continuation, which only addContinuation Listener method sets timeout. We can see the code:
Code 2.1: doMessages method in MessageServletSupport
if (message == null && client.getListener().getUndeliveredMessages().size() == 0) {
                Continuation continuation = ContinuationSupport.getContinuation(request);

                if (continuation.isExpired()) {
                    response.setStatus(HttpServletResponse.SC_OK);
                    StringWriter swriter = new StringWriter();
                    PrintWriter writer = new PrintWriter(swriter);
                    writer.println("<ajax-response>");
                    writer.print("</ajax-response>");

                    writer.flush();
                    String m = swriter.toString();
                    response.getWriter().println(m);

                    return;
                }

                continuation.setTimeout(timeout);
                continuation.suspend();
                LOG.debug( "Suspending continuation " + continuation );

                // Fetch the listeners
                AjaxListener listener = client.getListener();
                listener.access();

                // register this continuation with our listener.
                listener.setContinuation(continuation);

                return;
            }
When there is no message, the setup timeout method of Servlet3Continuation is not invoked, that is, it cannot be restored from the "suspend" state. This causes the request to remain idle until Apache's request timeout time, and causes the amq client to fail to complete the initialization identification.
It can be seen from this that the initialization of amq client is only a request, setting up a super short time, but the background process is consistent with the normal real-time information acquisition. This can explain a phenomenon: if the queue corresponding to ActiveMq has the corresponding information just at the time of initialization, the initialization flag will be returned correctly and set correctly. But then, if queue data is not available, it waits until apache refuses.
Through the above survey, we find that we have reason to doubt whether the current "Continuation" implementation of ActiveMq component packages is problematic.

Solve:

By looking up the data, we found that this is one of amq's. AMQ-3447 The bug is fixed in 5.9.0, so upgrade the MQ version. In the doMessages method of MessageServletSupport5.9.0, we can see the code:
 if (message == null && client.getListener().getUndeliveredMessages().size() == 0) {
                Continuation continuation = ContinuationSupport.getContinuation(request);

                // Add a listener to the continuation to make sure it actually
                // will expire (seems like a bug in Jetty Servlet 3 continuations,
                // see https://issues.apache.org/jira/browse/AMQ-3447
                continuation.addContinuationListener(new ContinuationListener() {
                    @Override
                    public void onTimeout(Continuation cont) {
                        if (LOG.isDebugEnabled()) {
                            LOG.debug("Continuation " + cont.toString() + " expired.");
                        }
                    }

                    @Override
                    public void onComplete(Continuation cont) {
                        if (LOG.isDebugEnabled()) {
                           LOG.debug("Continuation " + cont.toString() + " completed.");
                        }
                    }
                });

                if (continuation.isExpired()) {
                    response.setStatus(HttpServletResponse.SC_OK);
                    StringWriter swriter = new StringWriter();
                    PrintWriter writer = new PrintWriter(swriter);
                    writer.println("<ajax-response>");
                    writer.print("</ajax-response>");

                    writer.flush();
                    String m = swriter.toString();
                    response.getWriter().println(m);

                    return;
                }

                continuation.setTimeout(timeout);
                continuation.suspend();
                LOG.debug( "Suspending continuation " + continuation );

                // Fetch the listeners
                AjaxListener listener = client.getListener();
                listener.access();

                // register this continuation with our listener.
                listener.setContinuation(continuation);

                return;
            }
Here, the Continuation Listener is set to ensure that an empty Ajax message can be returned to the front desk when the timeout or completion occurs, so that no timeout occurs.


Posted by xtops on Tue, 18 Jun 2019 16:52:40 -0700