Thread pool
If too many threads are created in the JVM, the system may run out of resources due to excessive memory consumption or "excessive switching".
In order to solve this problem, the first thought is pooling, so there is the concept of thread pool.
The core logic of the thread pool is to create several threads in advance and put them in a container. If a task needs to be processed, the task will be directly assigned to the thread in the thread pool for execution. After the task is processed, the thread will not be destroyed, but wait for subsequent task allocation, so as to reduce the overhead caused by thread creation and destruction.
characteristic
Reasonable use of thread pool can bring some benefits
- Reduce the performance overhead of creating and destroying threads
- Improve the response speed. When a new task needs to be executed, it can be executed immediately without waiting for the thread to be created
- Reasonably setting the thread pool pair size can avoid the problem caused by the number of threads exceeding the hardware resource bottleneck
use
Java has provided several different thread pool pairs for the JUC package. Using the provided API can quickly meet the daily development needs.
Example
/**
* Create a thread pool with a fixed number of threads of 5 and execute 10 tasks
*/
public class TestThreadPool {
public static void main(String[] args) {
ExecutorService threadPool = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
threadPool.execute(new TestTask());
}
threadPool.shutdown();
}
static class TestTask implements Runnable{
@Override
public void run() {
System.out.println(Thread.currentThread().getName() + " is running");
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
API
Executors
JUC provides a tool class java.util.concurrent.Executors, which provides many methods to create thread pool and thread factory objects.
There are four common strategies for thread pooling:
- newFixedThreadPool
- newSingleThreadExecutor
- newCachedThreadPool
- newScheduledThreadPool
newFixedThreadPool
A thread pool with a fixed number of threads. The number of threads remains unchanged. When a task is submitted to the thread pool, if there are idle threads, it will be executed immediately. If there are no idle threads, it will be added to the waiting queue.
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
newSingleThreadExecutor
The thread pool with a fixed number of threads is 1. If the thread is idle, the task will be executed, otherwise the task will be added to the waiting queue.
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
newCachedThreadPool
One can adjust the number of threads according to the actual situation. For the thread pool, the maximum number of threads is not limited. If there are idle threads, the task will be executed, and each idle thread will be automatically recycled after 60 seconds; If there are no tasks, no threads are created.
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
newScheduledThreadPool
A thread pool with latency and periodic execution tasks.
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}
public ScheduledThreadPoolExecutor(int corePoolSize) {
super(corePoolSize, Integer.MAX_VALUE,
DEFAULT_KEEPALIVE_MILLIS, MILLISECONDS,
new DelayedWorkQueue());
}
ThreadPoolExecutor
The above four thread pools are implemented based on java.util.concurrent.ThreadPoolExecutor. The difference is that the construction parameter pairs are different.
There are 7 input parameters in the construction method, and the thread pool operation strategy is configured according to these input parameters.
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler)
- corePoolSize is the number of core threads. The number of threads held in the thread pool. Unless the allowCoreThreadTimeOut property is set, idle core threads will always be waiting
- maximumPoolSize maximum number of threads, the maximum number of threads allowed in the pool
- keepAliveTime survival time. When the number of threads in the pool is greater than the core capacity, the survival time of redundant idle threads will not exceed this value
- Unit time unit
- workQueue work queue is used to store unexecuted tasks. The task type can only be Runnable
- For example, newScheduledThreadPool implements delayed tasks through a delay queue
- The threadFactory thread project is used to create new threads
- handler rejects the case when the processor is used to handle the maximum capacity of threads and queues
ThreadPoolExecutor is the core of thread pool and provides the implementation of thread pool. Based on ThreadPoolExecutor, we can implement thread pool according to customization.
principle
When an asynchronous task is submitted to the thread pool, how does the thread pool handle it?
The following figure shows this process
Submit task
The first step is to submit a task to the thread pool, that is, execute the java.util.concurrent.ThreadPoolExecutor#execute method. The ThreadPoolExecutor judges the processing method of the task according to the configured parameters such as the number of core threads, queue and maximum threads.
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
//1. If the number of currently running threads is less than the number of core threads, create a new thread to execute the task
if (addWorker(command, true))
return;
c = ctl.get();
}
//2. If the core pool is full, try to add the task to the queue
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
//3. Failed to join the queue, indicating that the queue is full. Try to create a new thread
else if (!addWorker(command, false))
//4. If it still fails, enable the reject policy
reject(command);
}
Thread pool status
In the execute method, you can see that a variable ctl has a high exposure rate. It runs through the whole life cycle of the thread pool. Its main function is to save the number of threads and the state of the thread pool.
The following code is the variables and functions used to represent the thread pool state in the TreadPoolExecutor.
//It is used to save the status and number of threads of the thread pool. The high three bits represent the status of the thread pool and the low 29 bits represent the number of threads
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int COUNT_MASK = (1 << COUNT_BITS) - 1;
//The status of thread pool is represented by the top three bits of ctl
private static final int RUNNING = -1 << COUNT_BITS;
private static final int SHUTDOWN = 0 << COUNT_BITS;
private static final int STOP = 1 << COUNT_BITS;
private static final int TIDYING = 2 << COUNT_BITS;
private static final int TERMINATED = 3 << COUNT_BITS;
// Judge the thread pool status and the number of threads according to the ctl value
private static int runStateOf(int c) { return c & ~COUNT_MASK; }
private static int workerCountOf(int c) { return c & COUNT_MASK; }
private static int ctlOf(int rs, int wc) { return rs | wc; }
private static boolean runStateLessThan(int c, int s) {
return c < s;
}
private static boolean runStateAtLeast(int c, int s) {
return c >= s;
}
private static boolean isRunning(int c) {
return c < SHUTDOWN;
}
- RUNNING status, accept new tasks and execute tasks in the queue
- SHUTDOWN does not accept new tasks and executes tasks in the queue
- STOP does not accept new tasks, does not execute tasks in the queue, and interrupts the executing thread
- TIDYING all tasks have ended, the number of threads is 0, and the terminate() method will be called
- TERMINATED terminate() finished executing
Create a new thread
When the number of working threads in the thread pool is less than the number of core threads, a new thread will be created to process the task, and the addWorker method will be called.
The addWorker method mainly does two things:
1. Increase the number of working threads by one;
for (int c = ctl.get();;) {
//Only when the thread pool is in running state or SHUTDOWN state and the queue is not empty can a new thread be created. Otherwise, false is returned directly
if (runStateAtLeast(c, SHUTDOWN)
&& (runStateAtLeast(c, STOP)
|| firstTask != null
|| workQueue.isEmpty()))
return false;
for (;;) { //spin
//If the number of working threads is greater than the default maximum number, false is returned directly
if (workerCountOf(c)
>= ((core ? corePoolSize : maximumPoolSize) & COUNT_MASK))
return false;
if (compareAndIncrementWorkerCount(c)) //Try cas to increase the number of working threads by one. If it fails, continue the loop. If it succeeds, jump out of the loop and continue to create threads
break retry;
c = ctl.get(); // Re-read ctl
if (runStateAtLeast(c, SHUTDOWN))
continue retry;
}
}
2. Create a new thread and enable it.
Worker w = null;
try {
//Build a woker
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
int c = ctl.get();
if (isRunning(c) ||
(runStateLessThan(c, STOP) && firstTask == null)) {
if (t.getState() != Thread.State.NEW)
throw new IllegalThreadStateException();
//Only when the thread pool is in running state or SHUTDOWN state and the incoming task is empty can it be added to the wokers collection
workers.add(w);
workerAdded = true;
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
}
} finally {
mainLock.unlock();
}
if (workerAdded) {
t.start();
workerStarted = true;
}
}
} finally {
if (! workerStarted)
//If the addition fails, the number of threads added before will be reduced back
addWorkerFailed(w);
}
woker
The thread created by the thread pool is a woker object.
private final class Worker
extends AbstractQueuedSynchronizer
implements Runnable{
...
Worker(Runnable firstTask) {
setState(-1); // inhibit interrupts until runWorker
this.firstTask = firstTask;
this.thread = getThreadFactory().newThread(this);
}
...
}
Woker inherits AQS and implements Runnable, so it is a thread and implements the function of exclusive lock, which is to ensure that the thread executing the task cannot be interrupted.
In the construction method, threads are created using the factory configured by the thread pool.
Perform tasks
The entry of the running task of the thread pool is the execute method. The core is to create the thread method addWoker, and the runWoker method in Woker actually executes the task.
After a woker is created, until it is destroyed by the thread pool, it is in the loop, constantly trying to get tasks from the queue for execution.
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
//woker can be interrupted when it does not execute a task, so it needs to release the exclusive lock
w.unlock(); // allow interrupts
boolean completedAbruptly = true;
try {
//woker threads constantly try to obtain tasks, so as to realize thread reuse
while (task != null || (task = getTask()) != null) {
//woker in operation is not allowed to be interrupted
w.lock();
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
try {
//The pre operation is not implemented by default. If necessary, you can inherit and implement it yourself
beforeExecute(wt, task);
try {
//Perform tasks
task.run();
afterExecute(task, null);
} catch (Throwable ex) {
afterExecute(task, ex);
throw ex;
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
//Destroy threads without tasks
processWorkerExit(w, completedAbruptly);
}
}
Idle thread recycling
After the tasks submitted to the queue in the thread pool are executed, naturally there will be some threads, some of which will be destroyed, and only a few core threads will be retained (if allowCoreThreadTimeOut is true, the core threads will also be destroyed), and the idle judgment is realized in getTask.
private Runnable getTask() {
boolean timedOut = false; // Did the last poll() time out?
for (;;) {
int c = ctl.get();
if (runStateAtLeast(c, SHUTDOWN)
&& (runStateAtLeast(c, STOP) || workQueue.isEmpty())) {
decrementWorkerCount();
return null;
}
int wc = workerCountOf(c);
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
if ((wc > maximumPoolSize || (timed && timedOut))
&& (wc > 1 || workQueue.isEmpty())) {
//After the timeout, the number of worker threads is reduced by one and then destroyed
if (compareAndDecrementWorkerCount(c))
return null;
continue;
}
try {
//When woker obtains a task, a timeout time is added, that is, keepAliveTime. If it is not obtained, it is considered idle
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
//Idle threads will be destroyed on the next spin
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
}
}
The implementation of idle thread recycling is to add a timeout time to the poll method of the queue, that is, keepAliveTime. If the thread whose task has not been obtained after the timeout is considered idle, it will be destroyed at the end of the runWoker method.
Reject policy
When a new task cannot be received in the thread pool, the java.util.concurrent.RejectedExecutionHandler#rejectedExecution() method is executed.
In JUC, there are four implementations of reject policy processors:
- AbortPolicy throws exceptions directly. It is the default rejection policy
- DiscardPolicy discards tasks directly, but does not throw exceptions
- DiscardOldestPolicy discards the first task waiting in the queue and calls the execute() method again
- CallerRunsPolicy executes the task directly with the thread of the caller
In addition to the 4 rejection policies provided, they can also be customized by implementing the RejectedExecutionHandler interface.
summary
In Alibaba Java development manual, there is a provision on thread pool
In our daily development work, we often use thread pool. Therefore, after understanding the principle of thread pool, we should try our best to configure the parameters of thread pool according to the actual scene.
Rational allocation
How to set the thread pool size is reasonable?
CPU intensive
If it is CPU intensive and mainly performs computing tasks with fast response time, and the CPU utilization of such tasks is very high, the number of threads should be set according to the number of CPU cores.
Too many threads will lead to context switching, which will reduce the efficiency. Then the maximum number of threads in the thread pool can be configured as the number of CPU cores + 1.
IO intensive
If it is Io intensive, it is mainly for IO operations. If the execution time of IO operations is long, the CPU idle time will be long, resulting in low CPU utilization. In this case, the size of thread pool can be increased appropriately.
Generally, you can configure twice the number of CPU cores.