A global unique id algorithm of 1 million generated per second by self-study

Keywords: Redis network Zookeeper Database

background

The System tps is 3000, distributed deployment is adopted, and the business order number needs to be globally unique, such as order number. There are many common distributed global id schemes on the Internet, including the following:

  • UUID
  • redis atomic self increasing
  • zookeeper node directory
  • SnowFlake (SnowFlake algorithm)
  • Algorithms based on snowflake algorithm in large domestic factories, such as meituan's leaf

The above solutions are roughly divided into three categories

  1. UUID
  2. Middleware (redis, zk)
  3. Snowflake like algorithm

UUID
The implementation of uuid is relatively simple, but irregular, and the length is too long. On the one hand, it can not be perfectly combined with business, on the other hand, the storage space of database is too large.

middleware
There is basically no problem with the way of middleware, but it depends on the third-party service. Once the third-party service is unavailable, it is basically cool.

Snowflake like algorithm
There is basically no problem. It can be encapsulated as a method in the system, called directly, and expanded with its own business. Snowflake algorithm has a disadvantage: the implementation of snowflake algorithm depends on the system time. If the system time is called back, the global id will no longer be unique.

Introduction to snowflake algorithm

SnowFlake algorithm is an open-source distributed ID generation algorithm of Twitter. The core idea is to use a 64 bit long number as the global unique ID. It is widely used in distributed system, and time stamp is introduced into ID, which is basically self increasing.

One of the 64 bits is not used, and then 41 bit is used as the millisecond, 10 bit is used as the working machine id, and 12 bit is used as the serial number

The core of snowflake algorithm has two points:

  1. Introduce machine number
  2. Introduce serial number

Advantages of introducing machine number
By introducing machine number, the global id generated by different nodes in the system can be distinguished, and the concurrency between different nodes has no effect on the global id of the system.

Introduce serial number
If the current second is the same as the previous second, the serial number will increase automatically, otherwise, the serial number will be reset to zero. The introduction of the serial number solves the limitation of the same node's dense concurrent MS precision.

Self realization global id

Self realization of global id is also based on two excellent ideas of snowflake algorithm: machine number and serial number

Global ID design

This kind of design can generate 10000 unique IDs in a second for the same machine. If the system has 100 nodes in total, it can generate 1 million global IDS in a second. The number of digits of serial number and machine number can be expanded for subsequent business increase.

code implementation

	private static Long lasttime = -1L;
    private static Integer sequence = 0;
    private static Integer sequeBit = 5;

    /**
     * 15 Bit millisecond time series number local ip end network segment 12 bit random number 1 bit suffix sign 15 + 3 + 5 + 1 = 24
     * Current problems
     * Each machine can generate up to 10000 order numbers per second
     * @return
     */
    public static synchronized String createOrderSn()  {
        Date date = new Date();
        SimpleDateFormat format = new SimpleDateFormat("yyyyMMddHHmmsss");
        String timeStr = format.format(date);
        long curr = System.currentTimeMillis() / 1000;
        if (curr == lasttime){
            // Current millisecond
            sequence ++;
        }else {
            sequence = 0;
        }
        lasttime = curr;
        String orderSn = null;
        try {
            orderSn = timeStr+ intToString(sequence) +getLocalIpEnd() +"S";
        } catch (UnknownHostException e) {
            e.printStackTrace();
            throw new RuntimeException("Generate order No. exception");
        }
        return orderSn;
    }

    /**
     * Get the last network segment of the machine, or convert it to node number according to ip
     * @return
     * @throws UnknownHostException
     */
    private static String getLocalIpEnd() throws UnknownHostException {
        InetAddress inetAddress= InetAddress.getLocalHost();
        //Get native Ip
        String ip=inetAddress.getHostAddress().toString();
        return ip.substring(ip.lastIndexOf(".") +1,ip.length());
    }

    /**
     * Keep serial number digits equal
     * @param n
     * @return
     */
    private static String intToString(int n){
        String s = String.valueOf(n);
        String prefix = "";
        if (s.length() < sequeBit){
            for (int i = 0; i < 5-s.length(); i++) {
                prefix += 0;
            }
        }
        return prefix + s;
    }

Example of generated serial number


Is this serial number elegant? You can see at a glance the time when the business happened, and you can also see which machine the business happened on. It will be very convenient to troubleshoot problems.

Remarks: official account No. 1
Scan code to pay attention to official account

Published 26 original articles, won praise 3, visited 823
Private letter follow

Posted by Bleej on Sun, 23 Feb 2020 02:06:09 -0800