In the previous article< Detailed description of <environments> configuration elements for Mybatis configuration > We already know that there are two configurable elements, one is the configuration of data source and connection pool, and the other is the configuration of transaction manager. In the last article, we just described briefly. From this article, we will start with two blog posts detailing these two issues.
In this article, we first look at the configuration of data sources and connection pools.
(1) Data sources supported in Mybatis
In the previous article, we learned that Mybatis supports three types of data source configurations: UNPOOLED, POOLED and JNDI, as shown in the following red area:
An interface DataSourceFactory is defined within MyRatis, and all three forms of support need to implement this interface. The DataSourceFactory interface is defined as follows:
package org.apache.ibatis.datasource;
import java.util.Properties;
import javax.sql.DataSource;
/**
* @author Clinton Begin
*/
public interface DataSourceFactory {
void setProperties(Properties props);
DataSource getDataSource();
}Corresponding to UNPOOLED, POOLED and JNDI, three classes of DataSourceFactory interface are defined and implemented in mybatis, namely Unpooled DataSourceFactory, Pooled DataSourceFactory and JndiDataSourceFactory.
The specific structure is as follows:
Corresponding to these data source factory classes, corresponding data source objects are defined, in which the Unpooled Data Source Factory and the Pooled Data Source Factory return the Unpooled Data Source and the Pooled Data Source Factory respectively, while the JndiDataSource Factory returns the corresponding data source according to the configuration.
(2) Creation of data sources in mybatis
Start with the configuration file:
<!-- Database connection pool -->
<dataSource type="POOLED">
<property name="driver" value="${driver}" />
<property name="url" value="${url}" />
<property name="username" value="${username}" />
<property name="password" value="${password}" />
</dataSource>(a) When mybatis initializes, when parsing to < dataSource > nodes, the corresponding data source factory class instance is created according to the corresponding type settings, as shown below:
DataSourceFactory dsFactory = dataSourceElement(child.evalNode("dataSource"));private DataSourceFactory dataSourceElement(XNode context) throws Exception {
if (context != null) {
String type = context.getStringAttribute("type");
Properties props = context.getChildrenAsProperties();
DataSourceFactory factory = (DataSourceFactory) resolveClass(type).newInstance();
factory.setProperties(props);
return factory;
}
throw new BuilderException("Environment declaration requires a DataSourceFactory.");
}In the above code, find the corresponding data source factory class according to the type type and instantiate it. Specific classes for each configuration have been declared in the Configuration class, as follows:
typeAliasRegistry.registerAlias("JNDI", JndiDataSourceFactory.class);
typeAliasRegistry.registerAlias("POOLED", PooledDataSourceFactory.class);
typeAliasRegistry.registerAlias("UNPOOLED", UnpooledDataSourceFactory.class);(b) Then, a DataSource object is obtained from the instance of the data source factory class through the getDataSource() method.
(c) After MyBatis creates an instance of DataSource, it is placed in the Environment object within the Configuration object for later use. As follows:
DataSourceFactory dsFactory = dataSourceElement(child.evalNode("dataSource"));
DataSource dataSource = dsFactory.getDataSource();
Environment.Builder environmentBuilder = new Environment.Builder(id)
.transactionFactory(txFactory)
.dataSource(dataSource);
configuration.setEnvironment(environmentBuilder.build());(3) When does the DataSource object of the data source create a database connection
When we need to create the SqlSession object and execute the SQL statement, MyBatis will call the dataSource object to create the java.sql.Connection object. That is, the creation of the java.sql.Connection object is delayed until the execution of the SQL statement.
public void testFindUserById(){
SqlSession sqlSession = getSessionFactory().openSession(true);
UserDao userMapper = sqlSession.getMapper(UserDao.class);
User user = userMapper.findUserById(10);
System.out.println("The records are as follows:"+user);
}For the above code, we will find that in the first two sentences we did not create a database connection, but triggered the creation of a database connection when we executed the userMapper.findUserById() method.
(4) Unpooled Data Source, a non-pooled data source
Let's start with the code directly:
@Override
public Connection getConnection() throws SQLException {
return doGetConnection(username, password);
}private Connection doGetConnection(String username, String password) throws SQLException {
Properties props = new Properties();
if (driverProperties != null) {
props.putAll(driverProperties);
}
if (username != null) {
props.setProperty("user", username);
}
if (password != null) {
props.setProperty("password", password);
}
return doGetConnection(props);
}private Connection doGetConnection(Properties properties) throws SQLException {
initializeDriver();
Connection connection = DriverManager.getConnection(url, properties);
configureConnection(connection);
return connection;
}From the above code, you can see the main process of creating a database connection by Unpooled Data Source. The specific sequence diagram is as follows:
(a) Initialize the driver by calling initializeDriver().
To determine whether the driver driver driver has been loaded into memory, if not, the driver class will be loaded dynamically, and a Driver object will be instantiated. DriverManager.registerDriver() method will be used to register it in memory for subsequent use.
(b) Call DriverManager.getConnection() to get the database connection;
(c) Make some settings for the database Connection and return to the database Connection connection Connection;
Set whether the database connection is automatically submitted, set transaction level, etc.
Some may wonder what username and password are passed to the data source here.
As mentioned above, when mybatis is initialized, the < dataSource > element is parsed and the associated < property > configuration is initialized as the configuration of the data source. That is the following logic:
Properties props = context.getChildrenAsProperties();
DataSourceFactory factory = (DataSourceFactory) resolveClass(type).newInstance();
factory.setProperties(props);So far, the Unpooled Data Source data source is a relatively clear understanding. Let's look at Pooled Data Source with connection pool
(5) Pooled Data Source with connection pool
Why use data sources with connection pools? The most fundamental reason is that it costs a lot to create connections every time. Frequent creation and closure of database connections will seriously affect performance. Therefore, the common practice is to maintain a database connection pool, not to close the database connection directly after each use, and then if you need to create a database connection later, use the released database connection directly to avoid the overhead caused by frequent creation and closure of database connections.
In mybatis, a class PoolState of database connection pool state is defined. In this class, besides maintaining data source instances, database connections are also maintained. Database connections are divided into two state types and stored in two lists: idleConnections and active Connections.
idleConnections:
The idle state PooledConnection object is placed in this collection, indicating that the currently idle and unused PooledConnection collection is preferred when calling the getConnection() method of PooledDataSource. When a java.sql.Connection object is used up, MyBatis wraps it as a PooledConnection object and places it in this collection.
Active Connections: The Pooled Connection object in the active state is placed in an Array List named Active Connections, which indicates that the currently used Pooled Connection collection calls the getConnection() method of the Pooled DataSource, preferring to fetch the Pooled Connection object from the idleConnections collection. If not, it depends on whether the collection is full or not. EdDataSource creates a Pooled Connection, adds it to the collection, and returns it.
Let's see how to get a database connection from the connection pool, or from the PooledDataSource class.
@Override
public Connection getConnection() throws SQLException {
return popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();
}
@Override
public Connection getConnection(String username, String password) throws SQLException {
return popConnection(username, password).getProxyConnection();
}Here you call the popConnection() method and return its proxy object.
private PooledConnection popConnection(String username, String password) throws SQLException {
boolean countedWait = false;
PooledConnection conn = null;
long t = System.currentTimeMillis();
int localBadConnectionCount = 0;
while (conn == null) {
synchronized (state) {
if (!state.idleConnections.isEmpty()) {
// Pool has available connection
conn = state.idleConnections.remove(0);
if (log.isDebugEnabled()) {
log.debug("Checked out connection " + conn.getRealHashCode() + " from pool.");
}
} else {
// Pool does not have available connection
if (state.activeConnections.size() < poolMaximumActiveConnections) {
// Can create new connection
conn = new PooledConnection(dataSource.getConnection(), this);
if (log.isDebugEnabled()) {
log.debug("Created connection " + conn.getRealHashCode() + ".");
}
} else {
// Cannot create new connection
PooledConnection oldestActiveConnection = state.activeConnections.get(0);
long longestCheckoutTime = oldestActiveConnection.getCheckoutTime();
if (longestCheckoutTime > poolMaximumCheckoutTime) {
// Can claim overdue connection
state.claimedOverdueConnectionCount++;
state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime;
state.accumulatedCheckoutTime += longestCheckoutTime;
state.activeConnections.remove(oldestActiveConnection);
if (!oldestActiveConnection.getRealConnection().getAutoCommit()) {
try {
oldestActiveConnection.getRealConnection().rollback();
} catch (SQLException e) {
log.debug("Bad connection. Could not roll back");
}
}
conn = new PooledConnection(oldestActiveConnection.getRealConnection(), this);
oldestActiveConnection.invalidate();
if (log.isDebugEnabled()) {
log.debug("Claimed overdue connection " + conn.getRealHashCode() + ".");
}
} else {
// Must wait
try {
if (!countedWait) {
state.hadToWaitCount++;
countedWait = true;
}
if (log.isDebugEnabled()) {
log.debug("Waiting as long as " + poolTimeToWait + " milliseconds for connection.");
}
long wt = System.currentTimeMillis();
state.wait(poolTimeToWait);
state.accumulatedWaitTime += System.currentTimeMillis() - wt;
} catch (InterruptedException e) {
break;
}
}
}
}
if (conn != null) {
if (conn.isValid()) {
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback();
}
conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password));
conn.setCheckoutTimestamp(System.currentTimeMillis());
conn.setLastUsedTimestamp(System.currentTimeMillis());
state.activeConnections.add(conn);
state.requestCount++;
state.accumulatedRequestTime += System.currentTimeMillis() - t;
} else {
if (log.isDebugEnabled()) {
log.debug("A bad connection (" + conn.getRealHashCode() + ") was returned from the pool, getting another connection.");
}
state.badConnectionCount++;
localBadConnectionCount++;
conn = null;
if (localBadConnectionCount > (poolMaximumIdleConnections + 3)) {
if (log.isDebugEnabled()) {
log.debug("PooledDataSource: Could not get a good connection to the database.");
}
throw new SQLException("PooledDataSource: Could not get a good connection to the database.");
}
}
}
}
}
if (conn == null) {
if (log.isDebugEnabled()) {
log.debug("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
}
throw new SQLException("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
}
return conn;
}Let's look at what the above methods have done.
1. See if there is a Pooled Connection object in idle state, and if there is one, return it directly to an available Pooled Connection object; otherwise proceed to step 2.
2. Check whether the active Pooled Connection pool active Connections is full; if not, create a new Pooled Connection object, then put it into the active Connections pool, and return the Pooled Connection object; otherwise, take the third step;
3. See if the first Pooled Connection object entered into the active Connections pool has expired: if it has expired, remove the object from the active Connections pool, then create a new Pooled Connection object, add it to the active Connections, and then return the object; otherwise, step 4;
4. Threads wait, cycle 2 steps.
The flow chart is as follows:
When we get the database connection Pooled Connection, we usually close the database connection after using it, but for pooling, we close a database connection not really to close the connection, but to put it back into the database connection pool.
How to achieve it? The proxy mode is used in mybatis to solve this problem effectively. That is, the database connection returned to external use is actually a proxy object (the object returned by calling getProxyConnection(). This proxy object is created when a real database connection is created, as follows:
public PooledConnection(Connection connection, PooledDataSource dataSource) {
this.hashCode = connection.hashCode();
this.realConnection = connection;
this.dataSource = dataSource;
this.createdTimestamp = System.currentTimeMillis();
this.lastUsedTimestamp = System.currentTimeMillis();
this.valid = true;
this.proxyConnection = (Connection) Proxy.newProxyInstance(Connection.class.getClassLoader(), IFACES, this);
}When invoking each method of this proxy object, it comes in by reflection from the invoke() method. Let's see:
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
String methodName = method.getName();
if (CLOSE.hashCode() == methodName.hashCode() && CLOSE.equals(methodName)) {
dataSource.pushConnection(this);
return null;
} else {
try {
if (!Object.class.equals(method.getDeclaringClass())) {
// issue #579 toString() should never fail
// throw an SQLException instead of a Runtime
checkConnection();
}
return method.invoke(realConnection, args);
} catch (Throwable t) {
throw ExceptionUtil.unwrapThrowable(t);
}
}
}private static final String CLOSE = "close";
We can see that a special treatment is made here to determine whether the method name is a close() method, and if so, call the pushConnection() method of the data source object to put the database connection back into the connection pool, as follows:
protected void pushConnection(PooledConnection conn) throws SQLException {
synchronized (state) {
state.activeConnections.remove(conn);
if (conn.isValid()) {
if (state.idleConnections.size() < poolMaximumIdleConnections && conn.getConnectionTypeCode() == expectedConnectionTypeCode) {
state.accumulatedCheckoutTime += conn.getCheckoutTime();
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback();
}
PooledConnection newConn = new PooledConnection(conn.getRealConnection(), this);
state.idleConnections.add(newConn);
newConn.setCreatedTimestamp(conn.getCreatedTimestamp());
newConn.setLastUsedTimestamp(conn.getLastUsedTimestamp());
conn.invalidate();
if (log.isDebugEnabled()) {
log.debug("Returned connection " + newConn.getRealHashCode() + " to pool.");
}
state.notifyAll();
} else {
state.accumulatedCheckoutTime += conn.getCheckoutTime();
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback();
}
conn.getRealConnection().close();
if (log.isDebugEnabled()) {
log.debug("Closed connection " + conn.getRealHashCode() + ".");
}
conn.invalidate();
}
} else {
if (log.isDebugEnabled()) {
log.debug("A bad connection (" + conn.getRealHashCode() + ") attempted to return to the pool, discarding connection.");
}
state.badConnectionCount++;
}
}
}Simply put the logic of the above method:
1. First, remove the current database connection from the active database connection set, activeConnections.
2. Judge whether the current database connection is valid, if it is invalid, jump to step 4; if it is valid, continue the following judgment;
3. Determine whether the number of idle database connections in the current idleConnections collection does not exceed the set threshold and is a link created by the current database connection pool. If so, place the database connection back into the idleConnections collection and notify the requesting object thread waiting on the database connection pool. If not, close the database connection.
4. Return the number of bad database connections in connection pool + 1.
(6) JndiDataSourceFactory, a data source factory of JNDI type
For JNDI type data source acquisition is relatively simple, mybatis defines a JndiDataSourceFactory class to create data sources created through JNDI form. The source code for this class is as follows:
public class JndiDataSourceFactory implements DataSourceFactory {
public static final String INITIAL_CONTEXT = "initial_context";
public static final String DATA_SOURCE = "data_source";
public static final String ENV_PREFIX = "env.";
private DataSource dataSource;
@Override
public void setProperties(Properties properties) {
try {
InitialContext initCtx;
Properties env = getEnvProperties(properties);
if (env == null) {
initCtx = new InitialContext();
} else {
initCtx = new InitialContext(env);
}
if (properties.containsKey(INITIAL_CONTEXT)
&& properties.containsKey(DATA_SOURCE)) {
Context ctx = (Context) initCtx.lookup(properties.getProperty(INITIAL_CONTEXT));
dataSource = (DataSource) ctx.lookup(properties.getProperty(DATA_SOURCE));
} else if (properties.containsKey(DATA_SOURCE)) {
dataSource = (DataSource) initCtx.lookup(properties.getProperty(DATA_SOURCE));
}
} catch (NamingException e) {
throw new DataSourceException("There was an error configuring JndiDataSourceTransactionPool. Cause: " + e, e);
}
}
@Override
public DataSource getDataSource() {
return dataSource;
}
private static Properties getEnvProperties(Properties allProps) {
final String PREFIX = ENV_PREFIX;
Properties contextProperties = null;
for (Entry<Object, Object> entry : allProps.entrySet()) {
String key = (String) entry.getKey();
String value = (String) entry.getValue();
if (key.startsWith(PREFIX)) {
if (contextProperties == null) {
contextProperties = new Properties();
}
contextProperties.put(key.substring(PREFIX.length()), value);
}
}
return contextProperties;
}
}Because this piece is not clear, not too familiar with JNDI, so this piece does not explain, back to the understanding of this piece after the supplementary explanation. If you know something, you can leave a message to explain it. Thank you.
If you want to know my latest blog, please pay attention to my blog, thank you. If you want to see more related technical articles and would like to support me to continue writing, please give me a reward. Thank you for your reward.
