1. Sharding sphere version 4.0.0-rc1 is divided into annual tables (to be optimized later)

1.1. overview

As for the tables of LogShardingAlgorithm in the previous article, I used to initialize them when I called for the first time. Although this can realize the function, I would use this if judgment every time I called. Although the performance loss is not big, I don't think this is the logical order for the business. My ideal is to automatically initialize tables after the LogShardingAlgorithm is instantiated

Now the problem is that the instantiation of LogShardingAlgorithm is executed during Spring initialization, and its creation is not generated by Spring's @ Component and other annotations, but by reflection. If initialization is performed at the beginning of instantiation, that is, when the construction method is executed, the applicationContext has not been initialized, the environment parameters cannot be obtained, and even the Datasource has not been initialized

1.2. Solutions

After transformation, the code is as follows: a single initialization method is singled out, and after the class object is instantiated, it is called.

/**
 * @author: laoliangliang
 * @description: Log fragmentation
 * @create: 2020/1/2 10:19
 **/
@Slf4j
public class LogShardingAlgorithm implements PreciseShardingAlgorithm, RangeShardingAlgorithm<Integer> {

    /**
     * Cache existing tables
     */
    private List<String> tables;

    private final String systemLogHead = "system_log_";

    public void init(){
        tables = DBUtil.getAllSystemLogTable();
    }

    @Override
    public String doSharding(Collection availableTargetNames, PreciseShardingValue shardingValue) {
        String target = shardingValue.getValue().toString();
        String year = target.substring(target.lastIndexOf("_") + 1, target.lastIndexOf("_") + 5);
        if (!tables.contains(systemLogHead + year)) {
            DBUtil.createLogTable(year);
            tables.add(year);
        }
        return shardingValue.getLogicTableName() + "_" + year;
    }

    @Override
    public Collection<String> doSharding(Collection<String> availableTargetNames, RangeShardingValue<Integer> shardingValue) {
        Collection<String> availables = new ArrayList<>();
        Range valueRange = shardingValue.getValueRange();
        for (String target : tables) {
            Integer shardValue = Integer.parseInt(target.substring(target.lastIndexOf("_") + 1, target.lastIndexOf("_") + 5));
            if (valueRange.hasLowerBound()) {
                String lowerStr = valueRange.lowerEndpoint().toString();
                Integer start = Integer.parseInt(lowerStr.substring(0, 4));
                if (start - shardValue > 0) {
                    continue;
                }
            }
            if (valueRange.hasUpperBound()) {
                String upperStr = valueRange.upperEndpoint().toString();
                Integer end = Integer.parseInt(upperStr.substring(0, 4));
                if (end - shardValue < 0) {
                    continue;
                }
            }
            availables.add(target);
        }
        return availables;
    }
}

The init method is called through another class instantiation, and the difficulty is how to get the LogShardingAlgorithm that is instantiated.

import cn.hutool.core.util.ReflectUtil;
import com.google.common.base.Optional;
import com.onegene.platform.system.log.LogShardingAlgorithm;
import org.apache.shardingsphere.core.rule.ShardingRule;
import org.apache.shardingsphere.core.rule.TableRule;
import org.apache.shardingsphere.core.strategy.route.ShardingStrategy;
import org.apache.shardingsphere.shardingjdbc.jdbc.core.ShardingContext;
import org.apache.shardingsphere.shardingjdbc.jdbc.core.datasource.ShardingDataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import javax.sql.DataSource;
/**
 * @author: laoliangliang
 * @description:
 * @create: 2020/1/18 8:29
 **/
@Component
public class StartupConfig {
    @Autowired
    private DataSource dataSource;

    @PostConstruct
    public void init() {
        this.loadLogInit();
    }

    private void loadLogInit() {
        if (dataSource instanceof ShardingDataSource) {
            ShardingDataSource sds = (ShardingDataSource) dataSource;
            ShardingContext shardingContext = sds.getShardingContext();
            ShardingRule shardingRule = shardingContext.getShardingRule();
            Optional<TableRule> systemLog = shardingRule.findTableRule("system_log");
            TableRule tableRule = systemLog.orNull();
            if (tableRule != null) {
                ShardingStrategy tableShardingStrategy = tableRule.getTableShardingStrategy();
                LogShardingAlgorithm preciseShardingAlgorithm = (LogShardingAlgorithm) ReflectUtil.getFieldValue(tableShardingStrategy, "preciseShardingAlgorithm");
                LogShardingAlgorithm rangeShardingAlgorithm = (LogShardingAlgorithm) ReflectUtil.getFieldValue(tableShardingStrategy, "rangeShardingAlgorithm");
                preciseShardingAlgorithm.init();
                rangeShardingAlgorithm.init();
            }
        }
    }
}

1.3. summary

It can be seen from the source code that the object instantiated by LogShardingAlgorithm is put into ShardingDataSource. Then we need to take it out. If it doesn't provide get method normally, we need to use reflection hardware to take it out

From the above code, we can see that range segmentation and precise segmentation need to instantiate two classes. I want to find out if they can be combined into one class. I also found that some versions can realize range and precise segmentation query at the same time by using complexkeys shardingalgorithm. But after my actual test, the current version 4.0.0 is not good, because the following code is Complex fragment source code

public final class ComplexShardingStrategy implements ShardingStrategy {
    
    @Getter
    private final Collection<String> shardingColumns;
    
    private final ComplexKeysShardingAlgorithm shardingAlgorithm;
    
    public ComplexShardingStrategy(final ComplexShardingStrategyConfiguration complexShardingStrategyConfig) {
        Preconditions.checkNotNull(complexShardingStrategyConfig.getShardingColumns(), "Sharding columns cannot be null.");
        Preconditions.checkNotNull(complexShardingStrategyConfig.getShardingAlgorithm(), "Sharding algorithm cannot be null.");
        shardingColumns = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
        shardingColumns.addAll(Splitter.on(",").trimResults().splitToList(complexShardingStrategyConfig.getShardingColumns()));
        shardingAlgorithm = complexShardingStrategyConfig.getShardingAlgorithm();
    }
    
    @SuppressWarnings("unchecked")
    @Override
    public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<RouteValue> shardingValues) {
        Map<String, Collection<Comparable<?>>> columnShardingValues = new HashMap<>(shardingValues.size(), 1);
        String logicTableName = "";
        for (RouteValue each : shardingValues) {

            // Key point here is that he forcibly converted the value of each to ListRouteValue, and the range query corresponds to BetweenRouteValue, so it is stuck at the source code level. Unless the policy is rewritten, this cannot be used as before
            columnShardingValues.put(each.getColumnName(), ((ListRouteValue) each).getValues());
            logicTableName = each.getTableName();
        }
        Collection<String> shardingResult = shardingAlgorithm.doSharding(availableTargetNames, new ComplexKeysShardingValue(logicTableName, columnShardingValues));
        Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
        result.addAll(shardingResult);
        return result;
    }
}