Previous chapter This chapter mainly introduces how a single producer writes data to RingBuffer data, how not to make rings overlap, notify consumers after writing, batch processing at one end of the producer, and how multiple producers work together. This chapter mainly introduces how multiple producers write data to RingBuffer data.

1. The MultiProducerSequencer requests the next node

Unlike the single producer, the next method directly sets the sequence of the producer's cursor cursor through the cursor.compareAndSet(current, next). You may ask whether consumers can obtain data in the multi producer scenario after setting the producer's cursor and before submitting data. The answer is No. the method of getHighestPublishedSequence implemented by MultiProducerSequencer is different from that of a single producer, which will be explained in detail later.

2. Multi producer MultiProducerSequencer submits data

The difference from a single producer is to use setAvailable to set the data to an available state.

In the case of multiple producers, other things are needed to track the serial number. This serial number refers to the currently writable serial number. Note that this is different from "add 1 to the cursor of RingBuffer". If you have more than one producer writing to RingBuffer at the same time, some entries may be written by the producer but not submitted.

Producer 1 gets serial number 14 and producer 2 gets serial number 15. Now suppose that producer 1 did not have time to submit data for some reason.

Producer 2 requests to submit data through setAvailable(15), as shown in the figure.

At this time, when the consumer passes waitFor(14), the returned result will be 13, which is good for any event processing.

When producer 1 completes submitting data through setAvailable(14), as shown in the figure.

The run implementation of BatchEventProcessor will process the data at positions 14 and 15, and obtain the available data through waitFor(16) next time.

3. Musiproducersequencer producer class diagram.

• Musiproducersequencer inherits AbstractSequencer and implements the Sequencer interface.

• Sequencer provides adding and deleting consumer sequences, creating SequenceBarrier, and obtaining the minimum sequence number and the maximum published sequence number.

• Cursored gets the current cursor.

• Sequenced get the current ringbuffer size, get a sequence number, and submit the data interface.

The relationship between consumers and producers is the same as that of a single producer.

4. Multiple producers obtain the next available serial number through next

public long next(int n) {
    if (n <1) {
        throw new IllegalArgumentException("n must be > 0");
    }
    long current;
    long next;
    do {
        // ringbuffer current producer cursor
        current = cursor.get();
        // Next available sequence number
        next = current + n;
        // Overlapping point position
        long wrapPoint = next - bufferSize;
        // Sequence number of cached consumer processing
        long cachedGatingSequence = gatingSequenceCache.get();
        // wrapPoint > cachedGatingSequence,
        // The overlapping position is greater than the sequence number of the cached consumer processing, which indicates that some consumers have not completed the processing and cannot prevent the data
        // cachedGatingSequence > nextValue
        // Only in https://github.com/LMAX-Exchange/disruptor/issues/76 In case of
        if (wrapPoint > cachedGatingSequence || cachedGatingSequence > current) {
            // Get the smallest sequence number of consumers and producers
            long gatingSequence = Util.getMinimumSequence(gatingSequences, current);
            // Still overlap
            if (wrapPoint > gatingSequence) {
                // Notify the consumer to handle the event
                waitStrategy.signalAllWhenBlocking();
                // After the producer waits for spin, the subsequent strategy needs to be used
                LockSupport.parkNanos(1);
                continue;
            }
            // If there is no overlap, set the consumer cache
            gatingSequenceCache.set(gatingSequence);
        }
        // Without overlap, set the sequence number of RingBuffer directly to next
        else if (cursor.compareAndSet(current, next)) {
            break;
        }
    }
    while (true);
    // Returns the available sequence number
    return next;
}

5. Multiple producers submit data through publish

public void publish(final long sequence) {
    // Set sequence to available state
    setAvailable(sequence);
    // Notify the consumer to handle the event
    waitStrategy.signalAllWhenBlocking();
}

Multiple producers have set cusor in the next method to obtain the sequence number. When submitting data, the sequence is set to the available state before it can be used by consumers.

6. Consumer consumption data

Recall the waitFor function of ProcessingSequenceBarrier, which calls to sequencer.getHighestPublishedSequence(sequence,availableSequence).

public long waitFor(final long sequence)
        throws AlertException, InterruptedException, TimeoutException {
    // Check for clert exceptions
    checkAlert();
    // Obtain the available Sequence number through the waitStrategy policy. cursorSequence is the current Sequence and dependentSequence is the dependent Sequence []
    long availableSequence = waitStrategy.waitFor(sequence, cursorSequence, dependentSequence, this);
    // The generated sequence is smaller than expected, and the sequence number may be reset back to the old oldSequence value
    //Can refer to https://github.com/LMAX-Exchange/disruptor/issues/76
    if (availableSequence <sequence) {
        return availableSequence;
    }
    // Get the largest available published sequence, which may be smaller than the sequence
    // It will appear in multiple producers. When producer 1 obtains serial number 13 and producer 2 obtains serial number 14; If producer 1 does not publish and producer 2 publishes, the available sequence number obtained will be 12 and the sequence will be 13
    return sequencer.getHighestPublishedSequence(sequence, availableSequence);
}

Gets the largest available published sequence

public long getHighestPublishedSequence(long lowerBound, long availableSequence) {
    for (long sequence = lowerBound; sequence <= availableSequence; sequence++) {
        // Judge whether it is available
        if (!isAvailable(sequence)) {
            return sequence - 1;
        }
    }
    return availableSequence;
}

isAvailable is judged through the availableBuffer

public boolean isAvailable(long sequence) {
    // Calculate ((int) sequence) & amp; index of indexmask
    int index = calculateIndex(sequence);
    // Calculate the number of slot settings of (int) (sequence > > > indexshift) ringbuffer
    int flag = calculateAvailabilityFlag(sequence);
    // The offset of index in the array
    long bufferAddress = (index * SCALE) + BASE;
    // If it is equal to flag, it indicates that it is available
    return UNSAFE.getIntVolatile(availableBuffer, bufferAddress) == flag;
}

The variables used internally are as follows.

// The availableBuffer tracks the status of the slot of each ringbuffer and whether it is available
private final int[] availableBuffer = new int[bufferSize]; // The initial value is - 1
private final int indexMask = bufferSize - 1;
private final int indexShift = Util.log2(bufferSize);

Through the above methods, you can judge whether the current sequence is available.

By directly returning the available availableSequence in the getHighestPublishedSequence method of the musiproducer sequencer, the consumer is notified of the consumption data, and the producer and consumer cooperate.