We know that single thread computing is serial, and only after the last task is finished can the next task be executed, so the execution efficiency is relatively low.

Then, if multithreading is used to execute tasks, more tasks can be executed in unit time, and master worker is a way of multithreading parallel computing.

Its idea is to start two processes to work together: Master and Worker processes.

Master is responsible for task receiving and assignment, and Worker is responsible for specific sub task execution. After each Worker completes the task, the result is returned to the master, and the master summarizes the result. (in fact, it is also a divide and rule idea, which is similar to the calculation framework of forkjoin. See: Parallel task computing framework)

The working diagram of master worker is as follows:

Next, use master worker to calculate the sum of squares of 1-100. The idea is as follows:

1. Define a Task class to store data for each Task.
2. Master produces a fixed number of workers, and stores all workers in the workers variable (map). Master needs to store the queue workqueue(ConcurrentLinkedQueue) of all tasks and the result set resultMap(ConcurrentHashMap) returned by all subtasks.
3. Each Worker performs its own subtasks, and then stores the results in the resultMap.
4. The Master summarizes the data in the resultMap and returns it to the Client.
5. In order to extend the function of Worker, a MyWorker inherits the specific method of Worker rewriting task processing.

Class Task:

package com.thread.masterworker;
public class Task {
    private int id;
    private String name;
    private int num;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getNum() {
        return num;
    }

    public void setNum(int num) {
        this.num = num;
    }
}

Master implementation:

package com.thread.masterworker;

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;

public class Master {
    //Queues for all tasks
    private ConcurrentLinkedQueue<Task> workerQueue = new ConcurrentLinkedQueue<Task>();

    //All worker
    private HashMap<String,Thread> workers = new HashMap<String,Thread>();

    //Shared variable, result returned by worker
    private ConcurrentHashMap<String,Object> resultMap = new ConcurrentHashMap<String,Object>();

    //Constructor, initializing all worker s
    public Master(Worker worker,int workerCount){
        worker.setWorkerQueue(this.workerQueue);
        worker.setResultMap(this.resultMap);

        for (int i = 0; i < workerCount; i++) {
            Thread t = new Thread(worker);
            this.workers.put("worker-"+i,t);
        }
    }

    //Submission of tasks
    public void submit(Task task){
        this.workerQueue.add(task);
    }

    //Perform tasks
    public int execute(){
        for (Map.Entry<String, Thread> entry : workers.entrySet()) {
            entry.getValue().start();
        }

        //Loop until the result returns
        while (true){
            if(isComplete()){
                return getResult();
            }
        }

    }

    //Determine whether all threads have completed execution
    public boolean isComplete(){
        for (Map.Entry<String, Thread> entry : workers.entrySet()) {
            //Return false as long as any thread does not end
            if(entry.getValue().getState() != Thread.State.TERMINATED){
                return false;
            }
        }
        return true;
    }

    //Processing result set returns final result
    public int getResult(){
        int res = 0;
        for (Map.Entry<String,Object> entry : resultMap.entrySet()) {
            res += (Integer) entry.getValue();
        }
        return res;
    }

}

Parent Worker:

package com.thread.masterworker;

import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;

public class Worker implements Runnable {

    private ConcurrentLinkedQueue<Task> workerQueue;

    private ConcurrentHashMap<String,Object> resultMap;

    public void setWorkerQueue(ConcurrentLinkedQueue<Task> workerQueue) {
        this.workerQueue = workerQueue;
    }

    public void setResultMap(ConcurrentHashMap<String, Object> resultMap) {
        this.resultMap = resultMap;
    }

    @Override
    public void run() {
        while(true){
            //Remove a task from the task queue
            Task task = workerQueue.poll();
            if(task == null) break;
            //Handling specific tasks
            Object res = doTask(task);
            //Put the result of each processing into the result set. Here, take the num value as the result
            resultMap.put(String.valueOf(task.getId()),res);
        }

    }

    public Object doTask(Task task) {
        return null;
    }
}

The subclass MyWorker inherits the parent class Worker and rewrites the doTask method to implement the specific logic:

package com.thread.masterworker;

public class MyWorker extends Worker {
    @Override
    public Object doTask(Task task) {
        //Pause for 0.5 seconds to simulate task processing
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        //Calculate the square of the number
        int num = task.getNum();
        return num * num;
    }
}

Client:

package com.thread.masterworker;

import java.util.Random;

public class Client {
    public static void main(String[] args) {

        Master master = new Master(new MyWorker(), 10);

        //Submit n tasks to task queue
        for (int i = 0; i < 100; i++) {
            Task task = new Task();
            task.setId(i);
            task.setName("task"+i);
            task.setNum(i+1);
            master.submit(task);
        }

        //Perform tasks
        long start = System.currentTimeMillis();
        int res = master.execute();
        long time = System.currentTimeMillis() - start;
        System.out.println("Result:"+res+",time consuming:"+time);
    }
}

In the above, we use 10 threads to perform subtasks, and the Master will finally calculate the sum (1-100 square sum). Each thread pauses for 500ms and calculates the square value of the number.

There are 100 tasks in total and 10 threads for parallel computing, which is equivalent to 10 tasks for each thread. The time of one task is about 500ms, so the time of 10 tasks is 5000ms, plus the time of calculating the square value, so it is slightly greater than 5000ms. The results are as follows,

Results: 338350, time consuming: 5084