Spring Dynamic Data Source + Mybatis Interceptor for Database Read-Write Separation

Keywords: Apache Mybatis Spring Java

In the project encountered a need to do read-write separation scenario.
For old projects, to minimize code intrusion and achieve read-write separation at the bottom is crashing.

There are two main points in the technology used:

  • spring dynamic data source
  • mybatis interceptor

spring dynamic data source

In the case of multiple data sources, spring provides dynamic data sources

org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource

Dynamic data sources can obtain different data sources by configuring key values.

But one thing to note: Dynamic data sources are not real data sources!

AbstractRouting Data Source, as its name implies, only provides the function of data source routing. Specific data sources need to be configured separately. So in our implementation, we also need to implement the configuration and generation of data sources.

The configuration of data sources is very simple. In the implementation class DynamicDataSource, three sets of data sources are declared:

 //Direct Given Data Source
    private List<DataSource> roDataSources;
    private List<DataSource> woDataSources;
    private List<DataSource> rwDataSources;

When used, the configured data source is injected through spring, and then three collections are traversed to specify different key s according to the configuration.
In order to manage keys uniformly, the generation and assignment of data source keys are managed in a single OPCountMapper class. According to the set of data sources, three keys and numbers are given respectively: read-only, read-write and write-only. Each data source in each key is invoked in turn according to the type of operation. That is to say, it has a simple load balancing function.

import static com.kingsoft.multidb.MultiDbConstants.*;


import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

/**
 * Data source key management
 * Each data source corresponds to a separate key, for example:
 * ro_0,rw_1,wo_2
 * And so on.
 * This mapping class operates by selecting an available database by operation type.
 * When no data source is available for the corresponding type, read and write data sources are used.
 * Created by SHIZHIDA on 2017/7/4.
 */
public class OPCountMapper {

    private Map<String,Integer> countMapper = new ConcurrentHashMap<>();
    private Map<String,Integer> lastRouter = new ConcurrentHashMap<>();

    public OPCountMapper(){
        countMapper.put(RO,0);
        countMapper.put(RW,0);
        countMapper.put(WO,0);
        lastRouter.put(RO,0);
        lastRouter.put(RW,0);
        lastRouter.put(WO,0);
    }

    public String getCurrentRouter(String key){
        int total = countMapper.get(key);
        if(total==0){
            if(!key.equals(RW))
                return getCurrentRouter(RW);
            else{
                return null;
            }
        }
        int last = lastRouter.get(key);
        return key+"_"+(last+1)%total;
    }


    public String appendRo() {
        return appendKey(RO);
    }
    public String appendWo() {
        return appendKey(WO);
    }
    public String appendRw() {
        return appendKey(RW);
    }

    private String appendKey(String key){
        int total = countMapper.get(key);
        String sk = key+"_"+total++;
        countMapper.put(key,total);
        return sk;
    }
}

Finally, specify the current data source in use, using the ThreadLocal class of java. This class maintains a separate member variable for each thread. When in use, you can specify the type of data source to be used in this thread according to the current operation:

/**
 * Database Selection
 * Created by SHIZHIDA on 2017/7/4.
 */
public final class DataSourceSelector {

    private static ThreadLocal<String> currentKey = new ThreadLocal<>();

    public static String getCurrentKey(){
        String key = currentKey.get();
        if(StringUtils.isNotEmpty(key))
            return key;
        else return RW;
    }

    public static void setRO(){
        setCurrenKey(RO);
    }
    public static void setRW(){
        setCurrenKey(RW);
    }
    public static void setWO(){
        setCurrenKey(WO);
    }

    public static void setCurrenKey(String key){
        if(Arrays.asList(RO,WO,RW).indexOf(key)>=0){
            currentKey.set(key);
        }else{
            currentKey.set(RW);
            warn("undefined key:"+key);
        }
    }
    
}

Mybatis interceptor

The configuration and selection of data sources are described above, and the selection function is handed over to Mybatis's interceptor.

First, all of Mybatis's SQL reads and writes are operated through the org.apache.ibatis.executor.Executor class. Tracking code shows that there are only three interfaces for reading and writing in this class, and the functions are clear at a glance.

int update(MappedStatement ms, Object parameter) throws SQLException;

  <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, CacheKey cacheKey, BoundSql boundSql) throws SQLException;

  <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler) throws SQLException;

That is to say, as long as these three interfaces are monitored, corresponding data sources can be assigned to all read and write operations.

The code is also very simple:


import com.kingsoft.multidb.datasource.DataSourceSelector;
import org.apache.ibatis.cache.CacheKey;
import org.apache.ibatis.executor.Executor;
import org.apache.ibatis.mapping.BoundSql;
import org.apache.ibatis.mapping.MappedStatement;
import org.apache.ibatis.plugin.*;
import org.apache.ibatis.session.ResultHandler;
import org.apache.ibatis.session.RowBounds;

import java.util.Properties;

/**
 * Interceptor, write library for update, read library for query
 * Created by SHIZHIDA on 2017/7/4.
 */
@Intercepts({
        @Signature(
                type= Executor.class,
                method = "update",
                args = {MappedStatement.class,Object.class}),
        @Signature(
                type= Executor.class,
                method = "query",
                args = {MappedStatement.class,Object.class, RowBounds.class, ResultHandler.class,CacheKey.class,BoundSql.class}),
        @Signature(
                type= Executor.class,
                method = "query",
                args = {MappedStatement.class,Object.class,RowBounds.class, ResultHandler.class}),
})
public class DbSelectorInterceptor implements Interceptor {

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        String name = invocation.getMethod().getName();
        if(name.equals("update"))
            DataSourceSelector.setWO();
        if(name.equals("query"))
            DataSourceSelector.setRO();
        return invocation.proceed();
    }

    @Override
    public Object plugin(Object target) {
        if(target instanceof Executor)
            return Plugin.wrap(target,this);
        else return target;
    }

    @Override
    public void setProperties(Properties properties) {

    }

}

summary

So far, a simple database read-write separation function has been implemented, as long as the data source is configured in spring, and the following configuration is made for mybatis's SqlSession Factory:

<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
    <property name="dataSource" ref="dynamicDataSource"/>
    <property name="configLocation" value="classpath:mybatis-config.xml"/>
    <property name="plugins" ref="dbSelectorInterceptor"/>
</bean>

In the case of code 0 intrusion, read-write separation can be achieved, with the function of multi-database load balancing.

Posted by thatsme on Sat, 08 Jun 2019 16:38:19 -0700