Record the principle and Java implementation of snowflake algorithm

1. Basic understanding:

SnowFlake algorithm is an open source distributed id generation algorithm for Twitter. The core idea is to use a 64 bit long number as the globally unique id.

It is widely used in distributed systems, and ID introduces timestamp, which basically keeps self increasing. There are detailed comments in the following code.

  2. Interpretation:
Among the 64 bits, one bit is not used, and then 41 bit is used as the number of milliseconds, 10 bit as the working machine id and 12 bit as the serial number.
For example:

0 0001100 10100011 10111110 10001001 00 10001 1 1001 0000 00000000

For example, the following 64 bit long number:

    The first part is a bit: 0, which is meaningless.

    The second part is 41 bit s: it represents the timestamp.

    The third part is five bit s: it represents the machine room id, 10001.

    The fourth part is 5 bit s: it represents the machine id, 1 1001.

    The fifth part is a 12 bit serial number, which refers to the serial number of IDs generated simultaneously within one millisecond on a machine in a machine room, 0000 0000000.

 

Question: 1 bit: No, why?

Because if the first bit in the binary is 1, it is all negative, but the IDS we generate are all positive, so the first bit is 0.

 

41 bit: indicates the timestamp in milliseconds.

41 bit can represent up to 2 ^ 41 - 1, that is, it can identify 2 ^ 41 - 1 milliseconds. Conversion to adulthood means 69 years.

 

10 bit: record the working machine id, which means that the service can be deployed on 2 ^ 10 machines at most, that is, 1024 machines.

In 10 bit s, 5 bits represent the machine room id and 5 bits represent the machine id. It means that it can represent up to 2 ^ 5 machine rooms (32 machine rooms), and each machine room can represent 2 ^ 5 machines (32 machines), which can also be determined according to the actual situation of your company.

12 bit: This is used to record different IDs generated in the same millisecond.

The maximum positive integer that 12 bits can represent is 2 ^ 12 - 1 = 4096, that is, 4096 different IDS in the same millisecond can be distinguished by the number represented by 12 bits.

 

Simply put, if a service of yours is supposed to generate a globally unique id, it can send a request to the system deployed with the SnowFlake algorithm, and the SnowFlake algorithm system will generate the unique id.
The SnowFlake algorithm system must first know its own computer room and machine, such as computer room id = 17 and machine id = 12.

Then, after receiving the request, the SnowFlake algorithm system will first generate a 64 bit long id by binary bit operation. The first bit of the 64 bits is meaningless.

After 41 bits, you can use the current timestamp (in milliseconds), then set the machine room id for 5 bits, and set the machine id for 5 bits.

Finally, judge the number of requests on the machine in the current machine room in one millisecond. Add a sequence number to the request for id generation as the last 12 bit s.

Finally, a 64 bit id is displayed, similar to:
0 0001100 10100011 10111110 10001001 00 10001 1 1001 0000 00000000

 

***********

This algorithm can guarantee that a unique id is generated on a machine in a computer room within the same millisecond. Multiple IDs may be generated in a millisecond, but they are distinguished by the sequence number of the last 12 bit s.
Let's take a brief look at a code implementation of the SnowFlake algorithm. This is an example. If you understand this meaning, you can try to transform the algorithm yourself in the future.
In short, each bit of a 64 bit number is used to set different flag bits to distinguish each id

 

The implementation code of SnowFlake JAVA algorithm is as follows:

public class IdWorker{

    //The following two 5 bits each add up to a 10 bit work machine id
    private long workerId;    //Job id
    private long datacenterId;   //Data id
    //12 digit serial number
    private long sequence;

    public IdWorker(long workerId, long datacenterId, long sequence){
        // sanity check for workerId
        if (workerId > maxWorkerId || workerId < 0) {
            throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0",maxWorkerId));
        }
        if (datacenterId > maxDatacenterId || datacenterId < 0) {
            throw new IllegalArgumentException(String.format("datacenter Id can't be greater than %d or less than 0",maxDatacenterId));
        }
        System.out.printf("worker starting. timestamp left shift %d, datacenter id bits %d, worker id bits %d, sequence bits %d, workerid %d",
                timestampLeftShift, datacenterIdBits, workerIdBits, sequenceBits, workerId);

        this.workerId = workerId;
        this.datacenterId = datacenterId;
        this.sequence = sequence;
    }

    //Initial timestamp
    private long twepoch = 1288834974657L;

    //The length is 5 digits
    private long workerIdBits = 5L;
    private long datacenterIdBits = 5L;
    //Maximum
    private long maxWorkerId = -1L ^ (-1L << workerIdBits);
    private long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
    //Serial number id length
    private long sequenceBits = 12L;
    //Maximum serial number
    private long sequenceMask = -1L ^ (-1L << sequenceBits);
    
    //Number of bits of work id to be shifted left, 12 bits
    private long workerIdShift = sequenceBits;
   //The data id needs to be shifted left by 12 + 5 = 17 bits
    private long datacenterIdShift = sequenceBits + workerIdBits;
    //The timestamp needs to be shifted left by 12 + 5 + 5 = 22 bits
    private long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
    
    //Last timestamp, initial value is negative
    private long lastTimestamp = -1L;

    public long getWorkerId(){
        return workerId;
    }

    public long getDatacenterId(){
        return datacenterId;
    }

    public long getTimestamp(){
        return System.currentTimeMillis();
    }

     //Next ID generation algorithm
    public synchronized long nextId() {
        long timestamp = timeGen();

        //Get the current timestamp. If it is less than the last timestamp, it indicates that the timestamp acquisition is abnormal
        if (timestamp < lastTimestamp) {
            System.err.printf("clock is moving backwards.  Rejecting requests until %d.", lastTimestamp);
            throw new RuntimeException(String.

Posted by mpiaser on Wed, 24 Nov 2021 06:36:20 -0800