6,PriorityBlockingQueue
An unbounded blocking queue that supports prioritization. The elements entering the queue will be sorted according to priority.
public class PriorityBlockingQueue<E> extends AbstractQueue<E>
implements BlockingQueue<E>, java.io.Serializable
The unbounded priority blocking queue uses an array to store data internally. When the capacity is reached, the capacity will be expanded automatically, and the placed elements will be sorted according to priority. There are four construction methods:
//The default construction method. The default initialization capacity is 11
public PriorityBlockingQueue() {
this(DEFAULT_INITIAL_CAPACITY, null);
}
//Specifies the initialization capacity of the queue
public PriorityBlockingQueue(int initialCapacity) {
this(initialCapacity, null);
}
//Specify the initialization capacity of the queue and the comparator to put into the element
public PriorityBlockingQueue(int initialCapacity,
Comparator<? super E> comparator) {
if (initialCapacity < 1)
throw new IllegalArgumentException();
this.lock = new ReentrantLock();
this.notEmpty = lock.newCondition();
this.comparator = comparator;
this.queue = new Object[initialCapacity];
}
//The incoming collection is put into the queue to initialize the queue. The incoming collection can implement the SortedSet interface or PriorityQueue interface for sorting. If these two interfaces are not implemented, it will be put into the queue in the normal order
public PriorityBlockingQueue(Collection<? extends E> c) {
this.lock = new ReentrantLock();
this.notEmpty = lock.newCondition();
boolean heapify = true; // true if not known to be in heap order
boolean screen = true; // true if must screen for nulls
if (c instanceof SortedSet<?>) {
SortedSet<? extends E> ss = (SortedSet<? extends E>) c;
this.comparator = (Comparator<? super E>) ss.comparator();
heapify = false;
}
else if (c instanceof PriorityBlockingQueue<?>) {
PriorityBlockingQueue<? extends E> pq =
(PriorityBlockingQueue<? extends E>) c;
this.comparator = (Comparator<? super E>) pq.comparator();
screen = false;
if (pq.getClass() == PriorityBlockingQueue.class) // exact match
heapify = false;
}
Object[] a = c.toArray();
int n = a.length;
// If c.toArray incorrectly doesn't return Object[], copy it.
if (a.getClass() != Object[].class)
a = Arrays.copyOf(a, n, Object[].class);
if (screen && (n == 1 || this.comparator != null)) {
for (int i = 0; i < n; ++i)
if (a[i] == null)
throw new NullPointerException();
}
this.queue = a;
this.size = n;
if (heapify)
heapify();
}
When the priority queue is put into the elements, it will be sorted, so we need to specify the sorting rules. There are two ways:
- Create PriorityBlockingQueue to specify Comparator
- The placed elements need to implement the Comparable interface
One of the above two methods must be selected. If there are both, the first rule will be followed.
- Important member variables
// Default array size private static final int DEFAULT_INITIAL_CAPACITY = 11; // Maximum size limit for allocable arrays private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; // An array of elements private transient Object[] queue; // Number of elements private transient int size; // Priority queue comparator private transient Comparator<? super E> comparator; // Lock for read / write operation private final ReentrantLock lock; // Blocking control of empty operation private final Condition notEmpty; // Spinlock for allocation, acquired via CAS. // Spin lock for allocation, obtained through CAS. private transient volatile int allocationSpinLock; // PriorityQueue for serialization only private PriorityQueue<E> q;
Demand: it's still a push business. At present, push is sent according to the order of putting in. For example, some announcements are urgent and have high priority. They need to be sent quickly. What's the matter? At this point, PriorityBlockingQueue comes in handy. The code is as follows:
public class PriorityBlockingQueueDemo {
//Push information encapsulation
static class Msg implements Comparable<Msg> {
//The lower the priority, the higher the priority
private int priority;
//Push message
private String msg;
public Msg(int priority, String msg) {
this.priority = priority;
this.msg = msg;
}
@Override
public int compareTo(Msg o) {
return Integer.compare(this.priority, o.priority);
}
@Override
public String toString() {
return "Msg{" +
"priority=" + priority +
", msg='" + msg + '\'' +
'}';
}
}
//Push queue
static PriorityBlockingQueue<Msg> pushQueue = new PriorityBlockingQueue<Msg>();
static {
//Start a thread for real push
new Thread(() -> {
while (true) {
Msg msg;
try {
long starTime = System.currentTimeMillis();
//Gets a push message. This method blocks until the result is returned
msg = pushQueue.take();
//Simulation push time
TimeUnit.MILLISECONDS.sleep(100);
long endTime = System.currentTimeMillis();
System.out.printf("[%s,%s,take time consuming:%s],%s,send message:%s%n", starTime, endTime, (endTime - starTime), Thread.currentThread().getName(), msg);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
}
//For push messages, if you need to send a push message, call this method to add the push information to the push queue first
public static void pushMsg(int priority, String msg) throws InterruptedException {
pushQueue.put(new Msg(priority, msg));
}
public static void main(String[] args) throws InterruptedException {
for (int i = 5; i >= 1; i--) {
String msg = "Let's learn together java High concurrency,The first" + i + "day";
PriorityBlockingQueueDemo.pushMsg(i, msg);
}
}
}
Run the above code and output the result:
[1638020133761,1638020133870,take time consuming:109],Thread-0,send message:Msg{priority=3, msg='Let's learn together java High concurrency,Day 3'}
[1638020133915,1638020134031,take time consuming:116],Thread-0,send message:Msg{priority=1, msg='Let's learn together java High concurrency,Day 1'}
[1638020134033,1638020134136,take time consuming:103],Thread-0,send message:Msg{priority=2, msg='Let's learn together java High concurrency,Day 2'}
[1638020134137,1638020134247,take time consuming:110],Thread-0,send message:Msg{priority=4, msg='Let's learn together java High concurrency,Day 4'}
[1638020134248,1638020134353,take time consuming:105],Thread-0,send message:Msg{priority=5, msg='Let's learn together java High concurrency,Day 5'}
Five push messages are put into main, i is put in as the priority of the message according to the flashback, and the final output result is output according to the priority from small to large. Note that Msg implements the Comparable interface and has the comparison function.
6.1 put (E) method
public void put(E e) {
offer(e); // never need to block
}
public boolean offer(E e) {
if (e == null)
throw new NullPointerException();
final ReentrantLock lock = this.lock;
lock.lock();
int n, cap;
Object[] array;
while ((n = size) >= (cap = (array = queue).length))
tryGrow(array, cap); // Capacity expansion in case of insufficient capacity
try {
Comparator<? super E> cmp = comparator;
if (cmp == null)
siftUpComparable(n, e, array);
else
siftUpUsingComparator(n, e, array, cmp);
size = n + 1;
notEmpty.signal(); // Wake up threads blocked when reading elements
} finally {
lock.unlock();
}
return true;
}
6.2 tryGrow method
Try increasing the array to hold at least one element (but it usually expands by about 50%)
private void tryGrow(Object[] array, int oldCap) {
lock.unlock(); // must release and then re-acquire main lock
Object[] newArray = null;
if (allocationSpinLock == 0 &&
UNSAFE.compareAndSwapInt(this, allocationSpinLockOffset,
0, 1)) {
try {
int newCap = oldCap + ((oldCap < 64) ?
(oldCap + 2) : // grow faster if small
(oldCap >> 1));
if (newCap - MAX_ARRAY_SIZE > 0) { // possible overflow
int minCap = oldCap + 1;
if (minCap < 0 || minCap > MAX_ARRAY_SIZE)
throw new OutOfMemoryError();
newCap = MAX_ARRAY_SIZE;
}
// Create a new array according to the expanded capacity
if (newCap > oldCap && queue == array)
newArray = new Object[newCap];
} finally {
allocationSpinLock = 0;
}
}
if (newArray == null) // back off if another thread is allocating
Thread.yield();
lock.lock();
// Update the record array of the queue
if (newArray != null && queue == array) {
queue = newArray;
System.arraycopy(array, 0, newArray, 0, oldCap);
}
}
6.3 take() method
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly(); // Lock interruptible lock
E result;
try {
while ( (result = dequeue()) == null)
// When the queue element is null, the thread is blocked. When it is not empty, the element is removed and returned
notEmpty.await();
} finally {
// Release lock
lock.unlock();
}
return result;
}
private E dequeue() {
int n = size - 1;
if (n < 0)
return null;
else {
Object[] array = queue;
E result = (E) array[0]; // Get root element
E x = (E) array[n];
array[n] = null;
Comparator<? super E> cmp = comparator;
if (cmp == null)
siftDownComparable(0, x, array, n);
else
siftDownUsingComparator(0, x, array, n, cmp);
size = n;
return result;
}
}
The source code of other methods should be understood by yourself.
7,SynchronousQueue
Synchronous blocking queue has no capacity. Unlike other blockingqueues, synchronous queue is a BlockingQueue that does not store elements. Each put operation must wait for a take operation, otherwise elements cannot be added, and vice versa.
public class SynchronousQueue<E> extends AbstractQueue<E>
implements BlockingQueue<E>, java.io.Serializable
- Synchronous blocking queue has no capacity. Unlike other blockingqueues, synchronous queue is a BlockingQueue that does not store elements. Each put operation must wait for a take operation, otherwise elements cannot be added, and vice versa.
- SynchronousQueue is rarely used in reality. It has been used in the thread pool. This queue is used in the implementation of Executors.newCachedThreadPool(). When a task is dropped into the thread pool, if the created working threads are busy processing tasks, a new line program will be created to process the tasks dropped into the queue.
Let's take a look at its construction method:
// Unfair strategy
public SynchronousQueue() {
this(false);
}
public SynchronousQueue(boolean fair) {
transferer = fair ? new TransferQueue<E>() : new TransferStack<E>();
}
- Here's a sample code
public class SynchronousQueueDemo {
static SynchronousQueue<String> queue = new SynchronousQueue<>();
public static void main(String[] args) throws InterruptedException {
new Thread(() -> {
try {
long starTime = System.currentTimeMillis();
queue.put("java High concurrency series, passerby a Java!");
long endTime = System.currentTimeMillis();
System.out.println(String.format("[%s,%s,take time consuming:%s],%s", starTime, endTime, (endTime - starTime), Thread.currentThread().getName()));
} catch (InterruptedException e) {
e.printStackTrace();
}
}).start();
//After sleeping for 5 seconds, take an element from the queue
TimeUnit.SECONDS.sleep(5);
System.out.println(System.currentTimeMillis() + "call take Get and remove elements," + queue.take());
}
}
Run the above code and output the result:
1638020884179 call take Get and remove elements,Learn together java High concurrency! [1638020879165,1638020884181,take time consuming:5016],Thread-0
A thread is started in the main method. The queue.put method is called to drop a piece of data into the queue. Blocking occurs when calling. It can be seen from the output results that the put method does not return to normal from the blocking state until the take method is called.
7.1 put (E) method
public void put(E e) throws InterruptedException {
if (e == null) throw new NullPointerException();
if (transferer.transfer(e, false, 0) == null) {
Thread.interrupted();
throw new InterruptedException();
}
}
7.2 take() method
public E take() throws InterruptedException {
E e = transferer.transfer(null, false, 0);
if (e != null) // Reading element succeeded
return e;
Thread.interrupted();
throw new InterruptedException();
}
8,DelayQueue
DelayQueue is an unbounded blocking queue that supports delaying the acquisition of elements. All elements in the queue are "delayable" elements. The elements in the column header are the first "expired" elements. If there is no element expiration in the queue, the elements cannot be obtained from the column header, even if there are elements, that is, the elements can be retrieved from the queue only when the delay period expires.
We have already talked about DelayQueue in the previous blog. What you want to know is https://blog.csdn.net/qq_43605444/article/details/121661830 Let's review it with an example.
Requirement: it's also a push business. Sometimes we want to push at 9 a.m. or other specified time. How can we achieve it? DelayQueue comes in handy.
public class DelayQueueDemo {
//Push information encapsulation
static class Msg implements Delayed {
//The lower the priority, the higher the priority
private int priority;
//Push message
private String msg;
//Timing sending time, in millisecond format
private long sendTimeMs;
public Msg(int priority, String msg, long sendTimeMs) {
this.priority = priority;
this.msg = msg;
this.sendTimeMs = sendTimeMs;
}
@Override
public String toString() {
return "Msg{" +
"priority=" + priority +
", msg='" + msg + '\'' +
", sendTimeMs=" + sendTimeMs +
'}';
}
@Override
public long getDelay(TimeUnit unit) {
return unit.convert(this.sendTimeMs - Calendar.getInstance().getTimeInMillis(), TimeUnit.MILLISECONDS);
}
@Override
public int compareTo(Delayed o) {
if (o instanceof Msg) {
Msg c2 = (Msg) o;
return Integer.compare(this.priority, c2.priority);
}
return 0;
}
}
//Push queue
static DelayQueue<Msg> pushQueue = new DelayQueue<Msg>();
static {
//Start a thread for real push
new Thread(() -> {
while (true) {
Msg msg;
try {
//Gets a push message. This method blocks until the result is returned
msg = pushQueue.take();
//Real push can be done here
long endTime = System.currentTimeMillis();
System.out.printf("Scheduled sending time:%s,Actual sending time:%s,send message:%s%n", msg.sendTimeMs, endTime, msg);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
}
//For push messages, if you need to send a push message, call this method to add the push information to the push queue first
public static void pushMsg(int priority, String msg, long sendTimeMs) throws InterruptedException {
pushQueue.put(new Msg(priority, msg, sendTimeMs));
}
public static void main(String[] args) throws InterruptedException {
for (int i = 5; i >= 1; i--) {
String msg = "Let's learn together java High concurrency,The first" + i + "day";
DelayQueueDemo.pushMsg(i, msg, Calendar.getInstance().getTimeInMillis() + i * 2000);
}
}
}
Run the above code and output the result:
Timing transmission time: 1638022563331,Actual sending time: 1638022563338,send message:Msg{priority=1, msg='Let's learn together java High concurrency,Day 1', sendTimeMs=1638022563331}
Timing sending time: 1638022565331,Actual sending time: 1638022565347,send message:Msg{priority=2, msg='Let's learn together java High concurrency,Day 2', sendTimeMs=1638022565331}
Timing sending time: 1638022567331,Actual sending time: 1638022567335,send message:Msg{priority=3, msg='Let's learn together java High concurrency,Day 3', sendTimeMs=1638022567331}
Timing transmission time: 1638022569330,Actual sending time: 1638022569333,send message:Msg{priority=4, msg='Let's learn together java High concurrency,Day 4', sendTimeMs=1638022569330}
Timing sending time: 1638022571295,Actual sending time: 1638022571302,send message:Msg{priority=5, msg='Let's learn together java High concurrency,Day 5', sendTimeMs=1638022571295}
It can be seen that the sending time is basically the same as the scheduled sending time. Msg in the code needs to implement the Delayed interface, focusing on the getDelay method, which returns the remaining delay time. this.sendTimeMs is used in the code to subtract the millisecond format time of the current time to obtain the remaining delay time.
9,LinkedTransferQueue
LinkedTransferQueue is an unbounded blocked TransferQueue queue composed of a linked list structure. Compared with other blocking queues, LinkedTransferQueue has more tryTransfer and transfer methods.
LinkedTransferQueue adopts a preemptive mode. This means that when the consumer thread takes an element, if the queue is not empty, it directly takes the data. If the queue is empty, it generates a node (the node element is null) to join the queue. Then the consumer thread is waiting on this node. When the producer thread joins the queue, it finds a node with null element, and the producer thread does not join the queue, Directly fill the element into the node and wake up the waiting thread of the node. The awakened consumer thread takes the element and returns from the called method. We call this node operation "matching".
LinkedTransferQueue is a superset of concurrent linkedqueue, synchronous Queue (transfer elements in fair mode) and LinkedBlockingQueue (basic method of blocking Queue). Moreover, LinkedTransferQueue is more useful because it not only integrates the functions of these classes, but also provides a more efficient implementation.
The LinkedTransferQueue class inherits from the AbstractQueue abstract class and implements the TransferQueue interface:
public class LinkedTransferQueue<E> extends AbstractQueue<E>
implements TransferQueue<E>, java.io.Serializable
public interface TransferQueue<E> extends BlockingQueue<E> {
// If there is a consumer waiting to receive it, the specified element is transmitted immediately. Otherwise, it returns false and does not enter the queue.
boolean tryTransfer(E e);
// If there is a consumer waiting to receive it, the specified element is transmitted immediately, otherwise wait until the element is received by the consumer.
void transfer(E e) throws InterruptedException;
// Set the timeout based on the above method
boolean tryTransfer(E e, long timeout, TimeUnit unit)
throws InterruptedException;
// Returns true if at least one consumer is waiting
boolean hasWaitingConsumer();
// Gets the number of consuming threads waiting to get all elements
int getWaitingConsumerCount();
}
Take another look at the above methods. The transfer(E e) method is similar to the put method of SynchronousQueue. You need to wait for the consumer to take the element. Otherwise, you have to wait all the time.
public void transfer(E e) throws InterruptedException {
if (xfer(e, true, SYNC, 0) != null) {
Thread.interrupted(); // failure possible only due to interrupt
throw new InterruptedException();
}
}
Other methods are similar to those in ArrayBlockingQueue and LinkedBlockingQueue.
- LinkedTransferQueue has only two construction methods, which will not be described in detail here:
public LinkedTransferQueue() {
}
public LinkedTransferQueue(Collection<? extends E> c) {
this();
addAll(c);
}
The TransferQueue interface inherits and extends the BlockingQueue interface, and defines some methods needed by the LinkedTransferQueue class.
The nodes in the TransferQueue queue are all Node types:
static final class Node {
// If it is the node requested by the consumer, isData is false, otherwise it is the production (data) node and true
final boolean isData; // false if this is a request node
// The value of the data node. If it is a consumer node, the item is null
volatile Object item; // initially non-null if isData; CASed to match
// Point to next node
volatile Node next;
// Wait thread
volatile Thread waiter; // null until waiting
// CAS settings next
final boolean casNext(Node cmp, Node val) {
return UNSAFE.compareAndSwapObject(this, nextOffset, cmp, val);
}
// CAS set item
final boolean casItem(Object cmp, Object val) {
// assert cmp == null || cmp.getClass() != Node.class;
return UNSAFE.compareAndSwapObject(this, itemOffset, cmp, val);
}
// Construction method
Node(Object item, boolean isData) {
UNSAFE.putObject(this, itemOffset, item); // relaxed write
this.isData = isData;
}
// Point next to yourself
final void forgetNext() {
UNSAFE.putObject(this, nextOffset, this);
}
// It will be called when the matching or node is cancelled, set the item self connection, and the waiter is null
final void forgetContents() {
UNSAFE.putObject(this, itemOffset, this);
UNSAFE.putObject(this, waiterOffset, null);
}
// Has the node been matched
final boolean isMatched() {
Object x = item;
return (x == this) || ((x == null) == isData);
}
// Is it a unmatched request node
// If yes, isData is false and the item is null, because if it is matched, the item is no longer null, but points to itself
final boolean isUnmatchedRequest() {
return !isData && item == null;
}
// Returns true if the given node cannot connect after the current node
final boolean cannotPrecede(boolean haveData) {
boolean d = isData;
Object x;
return d != haveData && (x = item) != this && (x != null) == d;
}
// Match a data node
final boolean tryMatchData() {
// assert isData;
Object x = item;
if (x != null && x != this && casItem(x, null)) {
LockSupport.unpark(waiter);
return true;
}
return false;
}
private static final long serialVersionUID = -3375979862319811754L;
// Unsafe mechanics
private static final sun.misc.Unsafe UNSAFE;
private static final long itemOffset;
private static final long nextOffset;
private static final long waiterOffset;
static {
try {
UNSAFE = sun.misc.Unsafe.getUnsafe();
Class<?> k = Node.class;
itemOffset = UNSAFE.objectFieldOffset
(k.getDeclaredField("item"));
nextOffset = UNSAFE.objectFieldOffset
(k.getDeclaredField("next"));
waiterOffset = UNSAFE.objectFieldOffset
(k.getDeclaredField("waiter"));
} catch (Exception e) {
throw new Error(e);
}
}
}
The change of node item before and after matching, where node1 is the data node and node2 is the placeholder node requested by the consumer:
| Node | node1(isData-item) | node2(isData-item) |
|---|---|---|
| Before matching | true-item | false-null |
| After matching | true-null | false-this |
- Data node, the item before matching is not null and is not itself, and is set to null after matching.
- The placeholder request node, the item before matching is null, and it is self connected after matching.
Important fields in the LinkedTransferQueue class are as follows:
// Is it multi-core private static final boolean MP = Runtime.getRuntime().availableProcessors() > 1; // The number of spins of the first waiting node before blocking private static final int FRONT_SPINS = 1 << 7; // The number of spins of the current node before blocking when the precursor node is processing private static final int CHAINED_SPINS = FRONT_SPINS >>> 1; // Threshold of sweepVotes static final int SWEEP_THRESHOLD = 32; // Queue head node transient volatile Node head; // Queue tail node private transient volatile Node tail; // Number of failed disconnections of deleted nodes private transient volatile int sweepVotes; // Possible values of how parameter of xfer method // For poll and tryTransfer without waiting private static final int NOW = 0; // for untimed poll, tryTransfer // Used for offer, put, add private static final int ASYNC = 1; // for offer, put, add // For blocking transfer and take without timeout private static final int SYNC = 2; // for transfer, take // poll, tryTransfer for timeout waiting private static final int TIMED = 3; // for timed poll, tryTransfer
9.1 xfer method
private E xfer(E e, boolean haveData, int how, long nanos) {
// If haveData but e is null, a NullPointerException exception is thrown
if (haveData && (e == null))
throw new NullPointerException();
// s is the node to be added, if necessary
Node s = null; // the node to append, if needed
retry:
for (;;) { // restart on append race
// Match from first node
for (Node h = head, p = h; p != null;) { // find & match first node
boolean isData = p.isData;
Object item = p.item;
// Judge whether the nodes have been matched
// item != There are two cases of null: one is the put operation, and the other is that the item of take is modified (matching is successful)
// (itme! = null) = = isdata indicates that p is either a put operation or a take operation that has not been matched successfully
if (item != p && (item != null) == isData) { // unmatched
// The node is consistent with this operation mode and cannot be matched
if (isData == haveData) // can't match
break;
// Match successful
if (p.casItem(item, e)) { // match
for (Node q = p; q != h;) {
Node n = q.next; // update by 2 unless singleton
// Update the head to the next node of the matching node
if (head == h && casHead(h, n == null ? q : n)) {
// Connect old nodes to themselves
h.forgetNext();
break;
} // advance and retry
if ((h = head) == null ||
(q = h.next) == null || !q.isMatched())
break; // unless slack < 2
}
// If the match is successful, the blocked thread will wake up
LockSupport.unpark(p.waiter);
// Type conversion to return elements matching nodes
return LinkedTransferQueue.<E>cast(item);
}
}
// If the node has already been matched, look back for the next unmatched node
Node n = p.next;
// If the current node has left the queue, search from the head
p = (p != n) ? n : (h = head); // Use head if p offlist
}
// If no matching node is found after the entire queue is traversed, subsequent processing will be performed
// Add the current node to the end of the queue
if (how != NOW) { // No matches available
if (s == null)
s = new Node(e, haveData);
// Add the new node s to the end of the queue and return the predecessor node of S
Node pred = tryAppend(s, haveData);
// If the precursor node is null, it indicates that other threads compete and the queue is modified, restart from retry
if (pred == null)
continue retry; // lost race vs opposite mode
// If it is not the ASYNC method, the synchronization block waits
if (how != ASYNC)
return awaitMatch(s, pred, e, (how == TIMED), nanos);
}
// how == NOW, return immediately
return e; // not waiting
}
}
10. Summary
- It is important to understand all the methods in BlockingQueue and their differences
- Focus on the usage scenarios of ArrayBlockingQueue, LinkedBlockingQueue, PriorityBlockingQueue and DelayQueue
- If the task to be processed has priority, use PriorityBlockingQueue
- If the processing task needs to be delayed, use DelayQueue