abstract

In the previous article, we have analyzed the connection of relational database and the principle of connection pool. In the Mongo database also exists, we often see some netizens asking Mongo to connect to the database or not, how to close. Whether the built-in database connection pool is single-threaded or multi-threaded, why the Mongo server kills cursors, kills connections and so on, in fact, these problems are basically connection pool problems, and many of them are similar to relational databases, not unique to mongo.
This paper aims to sort out these problems and make a comprehensive analysis.

Client Connection Analysis

Client connections are connected through driver jar. Take java as an example. mongo-java-driver
Connect to mongo, which is the same as relational database. The difference is that relational database has a set of standard blocking database operation implementation written to JDK, namely JDBC. Mongo, on the other hand, is completely provided by driver. Before version 3.0 of mongo-java-driver, only synchronous driver operation was provided. After version 3.x, asynchronous driver operation was provided. There is no diffusion here. There will be related blog posts on asynchronous database operation. This article only introduces synchronous driver operation.

Database operation
A Mongo operation process based on mongo-java-driver-2.14.x

public static void main(final String[] args) {
        try {
            final String host = "localhost";
            // Connection configuration properties
            final MongoClientOptions clientOptions = new MongoClientOptions.Builder()
                    .writeConcern(WriteConcern.ACKNOWLEDGED)
                    .readPreference(ReadPreference.secondaryPreferred())
                    .connectionsPerHost(10).socketTimeout(5000).build();
            final List<MongoCredential> credentialsList = new ArrayList<MongoCredential>();
            final String str = "test";
            final char[] psd = str.toCharArray();
            final MongoCredential credential = MongoCredential.createCredential("test",
                    "test",
                    psd);
            credentialsList.add(credential);
            final ServerAddress address = new ServerAddress(host, 27017);
            //Contains a built-in database connection pool
            final MongoClient client = new MongoClient(address, credentialsList, clientOptions);
            final DB db = client.getDB("test");
            final DBCollection postCollection = db.getCollection("test");
            postCollection.findOne();
            //Connection Closed, Release Resources
            client.close();
        } catch (final UnknownHostException e) {
            e.printStackTrace();
        }
    }

This is just a sample application. MongoClient should only have one instance in a jvm, which manages the connection and operates the database.
The interaction between client and database, mongo protocol is also based on TCP

Several Important Classes

MongoClientOptions: Database Connection Configuration Item

DB: database connection

DBCollection: collection operation

So the configuration core of the mongo connection is the MongolientOptions class. The more important configuration is
Connections PerHost. For online environments, if there are many applications connecting to databases, the number of connections should not be too large.
socketTimeout: Time-out of database operations. Generally, in 5s, for slow operations, the connection should not be occupied all the time, which will damage application performance and block other operations.

private int minConnectionsPerHost; //Minimum number of connections per node
private int connectionsPerHost = 100; // Number of connections per node
private int threadsAllowedToBlockForConnectionMultiplier = 5; //Maximum Waiting Thread
private int maxWaitTime = 1000 * 60 * 2; // Get the maximum connection waiting time
private int maxConnectionIdleTime; // Maximum idle time of connection pool
private int maxConnectionLifeTime;
private int connectTimeout = 1000 * 10; // Maximum connection time
private int socketTimeout = 0; // Maximum operating time
private boolean socketKeepAlive = false;
private boolean autoConnectRetry = false;
private long maxAutoConnectRetryTime = 0;
// Heart Rate Detection, Keep TCP Connection
private int heartbeatFrequency = Integer.parseInt(System.getProperty("com.mongodb.updaterIntervalMS", "5000"));
private int minHeartbeatFrequency = Integer.parseInt(System.getProperty("com.mongodb.updaterIntervalNoMasterMS", "500"));
private int heartbeatConnectTimeout = Integer.parseInt(System.getProperty("com.mongodb.updaterConnectTimeoutMS", "20000"));
private int heartbeatSocketTimeout = Integer.parseInt(System.getProperty("com.mongodb.updaterSocketTimeoutMS", "20000"));

You can compare it with the implementation of relational database connection pool

mongo built-in connection pool management is relatively simple, there is no effective connection pool management, through heartbeat interval for a period of time to send data packets to the mongo server, to ensure that the connection is effective, which is a little different from the previous introduction, before the destruction of useless connections. This will add additional network and CPU burdens.

See how mongo initializes the connection pool when it creates MongoClient. Wait until the database operation, then go to the Pooled Connection Provider to get a connection for operation.

summary

Through the above analysis, there is no difference between the Mongo database connection pool under mongo driver 3.x and the relational database connection pool. The connection pool is implemented in different ways, such as a lock and a semaphore. Container selection is also different, but these do not affect most application developers'configuration and understanding of connections. Back to the first few questions, there are naturally answers. As for the problem of server killing cursor, the cursor body is not unique to mongo. The client controls the number of results by cursor, and the cursor body also occupies a lot of resources, so it can not always occupy. The server kills the cursor, so the cursor takes too long. You can view the timeout cursor through db.serverStatus().metrics.cursor to find time-consuming operations and optimize them.

"cursor": {
"timedOut": "NumberLong(99)"
"open": {
"noTimeout": "NumberLong(0)"
"pinned": "NumberLong(3)"
"total": "NumberLong(3)"
}
}