Asynchronous mode - worker thread
1. Definitions
Let a limited number of worker threads take turns to process an unlimited number of tasks asynchronously. It can also be classified as division of labor mode. Its typical implementation is thread pool, which also reflects the sharing mode in the classical design mode.
For example, the waiter (thread) of Haidilao takes turns to handle each guest's order (task). If each guest is equipped with a dedicated waiter, the cost is too high (compared with another multi-threaded design mode: thread per message). Note that different thread pools should be used for different task types, which can avoid hunger and improve efficiency. For example, If a restaurant worker has to greet guests (task type A) and cook in the back kitchen (task type B), it is obviously inefficient. It is more reasonable to divide into waiter (thread pool a) and chef (thread pool B). Of course, you can think of a more detailed division of labor.
2. Hunger
A fixed size thread pool will starve
- Two workers are two threads in the same thread pool
- What they have to do is: order for the guests and cook in the back kitchen, which are two stages of work
- Guests' ordering: they must finish ordering first, wait for the dishes to be ready and serve. During this period, the workers handling the ordering must wait
- Kitchen cooking: nothing to say, just do it
- For example, worker A handles the order task, and then it waits for worker B to finish the dishes and serve them. They also cooperate very well
- But now there are two guests at the same time. At this time, worker A and worker B go to order. At this time, no one cooks and is hungry
Starvation code demonstration caused by insufficient threads
(1) One thread orders and one thread cooks without hunger
@Test
public void test_starvation() {
ExecutorService pool = Executors.newFixedThreadPool(2);
//One thread orders and one thread cooks. You won't be hungry
pool.execute(() -> {
log.debug("Start ordering");
Future<String> future = pool.submit(() -> {
log.debug("cook a dish...");
return cooking();
});
try {
log.debug("Serve:{}", future.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
while (true) ;
}
-------------output--------------
20:56:45.524 [pool-1-thread-1] DEBUG c.Test_ThreadPoolStarvation - Start ordering
20:56:45.564 [pool-1-thread-2] DEBUG c.Test_ThreadPoolStarvation - cook a dish...
20:56:45.564 [pool-1-thread-1] DEBUG c.Test_ThreadPoolStarvation - Serve:Sauteed Potato, Green Pepper and Eggplant
(2) When two orders are executed, both threads run to order, and there is no thread to cook, resulting in hunger
static final List<String> MENU = Arrays.asList("Sauteed Potato, Green Pepper and Eggplant", "Spicy chicken", "cola");
static Random RANDOM = new Random();
static String cooking() {
return MENU.get(RANDOM.nextInt(MENU.size()));
}
@Test
public void test_starvation() {
ExecutorService pool = Executors.newFixedThreadPool(2);
//One thread orders and one thread cooks. You won't be hungry
pool.execute(() -> {
log.debug("Start ordering");
Future<String> future = pool.submit(() -> {
log.debug("cook a dish...");
return cooking();
});
try {
log.debug("Serve:{}", future.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
pool.execute(() -> {
log.debug("Start ordering");
Future<String> future = pool.submit(() -> {
log.debug("cook a dish...");
return cooking();
});
try {
log.debug("Serve:{}", future.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
while (true) ;
}
---------------output----------------
20:59:47.424 [pool-1-thread-2] DEBUG c.Test_ThreadPoolStarvation - Start ordering
20:59:47.414 [pool-1-thread-1] DEBUG c.Test_ThreadPoolStarvation - Start ordering
3. Solutions to hunger
Different task types use different thread pools! It can effectively avoid hunger
static final List<String> MENU = Arrays.asList("Sauteed Potato, Green Pepper and Eggplant", "Spicy chicken", "cola");
static Random RANDOM = new Random();
static String cooking() {
return MENU.get(RANDOM.nextInt(MENU.size()));
}
@Test
public void test_starvationResolving() {
//2 ordering threads
ExecutorService waiter = Executors.newFixedThreadPool(2);
//2 cooking threads
ExecutorService cooker = Executors.newFixedThreadPool(2);
waiter.execute(() -> {
log.debug("Start ordering...");
Future<String> future = cooker.submit(() -> {
log.debug("cook...");
return cooking();
});
try {
log.debug("Serving:{}", future.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
waiter.execute(() -> {
log.debug("Start ordering...");
Future<String> future = cooker.submit(() -> {
log.debug("cook...");
return cooking();
});
try {
log.debug("Serving:{}", future.get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
while (true) ;
}
--------------output-----------------
21:09:44.706 [pool-1-thread-2] DEBUG c.Test_ThreadPoolStarvation - Start ordering...
21:09:44.698 [pool-1-thread-1] DEBUG c.Test_ThreadPoolStarvation - Start ordering...
21:09:44.711 [pool-2-thread-2] DEBUG c.Test_ThreadPoolStarvation - cook...
21:09:44.712 [pool-1-thread-1] DEBUG c.Test_ThreadPoolStarvation - Serving: Coke
21:09:44.714 [pool-2-thread-1] DEBUG c.Test_ThreadPoolStarvation - cook...
21:09:44.714 [pool-1-thread-2] DEBUG c.Test_ThreadPoolStarvation - Serving: ground three delicacies
4. How many process pools are appropriate to create
- Too small will lead to the program can not make full use of system resources, easy to lead to hunger
- Too large will lead to more thread context switching and occupy more memory
(1) CPU intensive operation
Generally, the CPU core count + 1 can achieve the optimal CPU utilization. The + 1 ensures that when the thread is suspended due to page loss fault (operating system) or other reasons, the additional thread can be pushed up to ensure that the CPU clock cycle is not wasted.
(2) I/O intensive operation
The CPU is not always busy. For example, when you perform business computing, CPU resources will be used, but when you perform I/O operations, remote RPC calls, including database operations, the CPU will be idle. You can use multithreading to improve its utilization.
The empirical formula is as follows:
Number of threads = number of cores * expected CPU utilization * total time (CPU calculation time + waiting time) / CPU calculation time
For example, the calculation time of 4-core CPU is 50%, and the other waiting time is 50%. It is expected that the CPU will be 100% utilized. Apply the formula
4 * 100% * 100% / 50% = 8
For example, the calculation time of 4-core CPU is 10%, and the other waiting time is 90%. It is expected that the CPU will be 100% utilized. Apply the formula
4 * 100% * 100% / 10% = 40