Implementation of session server by Nginx reverse proxy + Tomcat+memcached

Keywords: Tomcat Session Nginx JSP

Written in front

The previous article explained the type of session for you. Today, this article will lead you to implement a simple session server step by step, mainly to let you understand the working process of session server. Of course, for small and medium-sized websites, this structure is also fully adequate. The main structure of this section:

  • Introduction to memcached-session-manager

  • Network Topology

  • Nginx reverse proxy configuration

  • Tomcat configuration

  • memcached configuration

  • test

  • error analysis

 

Introduction to memcached-session-manager

memcached-session-manager is a Tomcat session manager that stores sessions in memcached or Redis for highly available, scalable and fault-tolerant Web service scenarios. It supports sticky and non-sticky configurations and is currently using tomcat 6.x, 7.x and 8.x. For sticky sessions, session failover (tomcat crash) is supported, and for non-sticky sessions, this is the default (for different requests, sessions are provided by different tomcats by default). In addition, memcached failover (memcached crash) is achieved through session migration. Nor should there be a single point of failure, so when memcached fails, the session is not lost and can be used in Tomcat or other memcached.

In configuration, you must put spymemcached jar and memcached-session-manager jar in the lib directory corresponding to tomcat. If you want to use Redis instead of memcached, you need to use the. jar package corresponding to the Redis client "jedis". You also need to add some configuration attributes in the Manager class to enable the session to be stored in memcached. Memcached itself does not have configuration functions and must rely on Tomcat or other class servers at the front end.

For a simple implementation, you only need to install a tomcat (6, 7 or 8) and a memcached or Redis (or s.th., which supports the memcached protocol). In your production environment, you may have multiple tomcats, and there should be multiple memcached nodes available on different hardware. Alternatively, you can store session data in Redis instead of memcached. You can use sticky or non-sticky sessions, and memcached-session-manager (msm) supports two modes of operation.

Memcached-session-manager project address, http://code.google.com/p/memcached-session-manager/

 

Network Topology

This is mainly used for a Nginx to realize the front-end scheduling, then connect two Tomcat application servers, add two memcached storage in the back-end of Tomcat to achieve a small architecture, of course, it omits the nearest DNS server from the client, a load balancing cluster near the user, and the cache server cluster after load balancing. Its simple architecture and network address planning are as follows:

All confidants in this article are CentOS 7.3


Nginx reverse proxy configuration

For nginx, this experiment only plays a scheduling role, that is, the role of reverse proxy, and does not achieve static and dynamic separation, path rewriting and other operations, here is mainly to schedule client requests to the back-end application server Tomcat. So it's much easier to configure, just define an upstream group. Then schedule according to the corresponding algorithm. The collective allocation is as follows:

[root@vin ~]# vim /etc/nginx/nginx.conf
...
http {
    upstream vinsent {                      # Define server clusters
            server 192.168.14.66:8080;
            server 192.168.14.77:8080;
    }
    server {
        listen       80 ;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
                proxy_pass      #  Proxy the request to the back-end Tomcat server
        }
        ...
}


Tomcat configuration

Tomcat serves as an application server. Its main function is to process jsp files. Here, we need to provide a file index.jsp for testing and a corresponding version of. jar package. Mainly memcached-session-manager related jar packages, and tools for serializing cookie information of front-end users into a key-value format.

1) Install tomcat and related service packages

[root@vin tools]# cat /etc/redhat-release      # Version checking
CentOS Linux release 7.3.1611 (Core)
[root@vin tools]# iptables -F                  # Close firewall and selinux
[root@vin tools]# setenforce 0

[root@vin tools]# yum install java-1.7.0-openjdk-devel.x86_64   #  Install jdk

[root@vin tools]# yum installtomcat tomcat-lib tomcat-admin-webapps tomcat-webapps tomcat-docs-webapp

2) Provide context containers

Define a context container for testing on a host on two tomcat s, and create a session manager in it.

[root@vin webapps]# vim /etc/tomcat/server.xml 
...
    <Host name="node1.vin.cn" appBase="/data/webapps" autoDeploy="true">
    <Context path="/test" docBase="/usr/local/tomcat/webapps/test" reloadable="true">
        <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
            memcachedNodes="n1:192.168.14.99:11211,n2:192.168.14.88:11211"
            failoverNodes="n1"
            requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
            transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
        />
    </Context>
    </Host>
...

3) Provide test pages for two context s

The structure of the test directory is as follows

[root@vin webapps]# cat /usr/share/tomcat/webapps/test/index.jsp
<%@ page language="java" %>
    <html>
        <head><title>TomcatA</title></head>
        <body>
            <h1><font color="green">TomcatA.vinsent.cn</font></h1>
            <table align="centre" border="1">
            <tr>
                <td>Session ID</td>
                <% session.setAttribute("vinsent.cn","vinsent.cn"); %>
                <td><%= session.getId() %></td>
            </tr>
            <tr>
                <td>Created on</td>
                <td><%= session.getCreationTime() %></td>
            </tr>
            </table>
        </body>
    </html>

# Note: To see this process, you should change it to TomcatB. vinsend. CN on the TomCatB host.

4) Download the jar package

There are two main types of jar packages, one is memcached-session-manager-related packages, and the other is serialization-related packages: serialization packages have four main categories:

  • kryo-serializer:  msm-kryo-serializer, kryo-serializers-0.34+, kryo-3.x, minlog, reflectasm,  asm-5.x, objenesis-2.x

  • javolution-serializer:  msm-javolution-serializer, javolution-5.4.3.1

  • xstream-serializer:  msm-xstream-serializer, xstream, xmlpull, xpp3_min

  • flexjson-serializer: msm-flexjson-serializer, flexjson

Download address: https://github.com/magro/memcached-session-manager/wiki/SetupAndConfiguration


    Here we choose to place them separately. Of course, you can place them all in lib directory as well:

[root@vin webapps]# tree /usr/share/tomcat/webapps/test/
/usr/share/tomcat/webapps/test/
├── index.jsp
└── WEB-INF
    └── lib
        ├── javolution-5.4.3.1.jar
        └── msm-javolution-serializer-2.1.1.jar

2 directories, 3 files

[root@vin webapps]# ls /usr/share/tomcat/lib/
annotations-api.jar      extras                                   tomcat7-websocket.jar  tomcat-jsp-2.2-api.jar
catalina-ant.jar         jasper-el.jar                            tomcat-api.jar         tomcat-juli.jar
catalina-ha.jar          jasper.jar                               tomcat-coyote.jar      tomcat-servlet-3.0-api.jar
catalina.jar             jasper-jdt.jar                           tomcat-el-2.2-api.jar  tomcat-util.jar
catalina-tribes.jar      log4j.jar                                tomcat-i18n-es.jar     websocket-api.jar
commons-collections.jar  memcached-session-manager-2.1.1.jar      tomcat-i18n-fr.jar
commons-dbcp.jar         memcached-session-manager-tc7-2.1.1.jar  tomcat-i18n-ja.jar
commons-pool.jar         spymemcached-2.12.3.jar                  tomcat-jdbc.jar



Memcached configuration

The configuration of memcached server is very simple. Because memcached itself can not actively store cookie information, it only needs to install memcached service on memcached server. As for serialization of data, it is realized by the former server.

[root@vin ~]# yum install memcached             # Install memcached, I have installed
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Package memcached-1.4.15-10.el7.x86_64 already installed and latest version
Nothing to do
[root@vin ~]# systemctl start memcached         # Startup service
[root@vin ~]# ss -tnl | grep 1121               # Check whether the service is started properly
LISTEN     0      128          *:11211                    *:*                  
LISTEN     0      128         :::11211                   :::*


test

We test on the client side. Enter the address of nginx in the browser: refresh and you will see that the content accessed has changed, but the cookie value has not changed.


error analysis

1 Serialization failures, the common first direction is as follows, no visit cookie value will change with the change, explained for storage in memcached database.

2. Hint 404 error pages when accessing

In most cases, context is configured correctly, or the directory structure of the index.jsp file provided is incorrect.

Version 3 does not correspond

     If you can eliminate the above errors and still can't access them, it means that the version of the jar package you provided does not match that of your tomcat version.




Posted by Svoboda on Sun, 23 Dec 2018 15:09:07 -0800