Understanding of parallel pipeline

Keywords: Java Attribute less

Ideas sharing

Now there are two variables i,j. to calculate the result value of (i+j)*i/2, we can first find that each step of the calculation depends on the result of the previous step. For example, we have to add, then multiply, and then divide. It seems that we can't split it into parallel. This is normal because there is strong coupling between the calculation steps. However, if thousands of i and j variables need to calculate the result value of (i+j)*i/2, if the idea of serial is used normally, the beginning of addition of each pair of variables i and j must be based on the end of division of the previous pair of variables i and j, but actually there is no dependency between each pair of variables i and j, only the same pair of variables i and j need to pay attention to the order and steps when calculating. Therefore, we can try this split calculation step, which is divided into addition, multiplication and division. At the same time, variables i and j are abstracted into objects, which are stored and passed in each step. Here, we need to introduce queues. In fact, this is also a kind of idea of producers and consumers. Addition produces the result of adding variables i and j, and then uses multiplication to consume it, and then produces the result after multiplication So, the final division is to consume the result of multiplication and make the final output.

Sample code

There is a class abstracted from variables i and j:

/**
 * Calculated objects
 */
public class CalculateMsg {
    private double i;
    private double j;
    private String str;

    public double getI() {
        return i;
    }

    public void setI(double i) {
        this.i = i;
    }

    public double getJ() {
        return j;
    }

    public void setJ(double j) {
        this.j = j;
    }

    public String getStr() {
        return str;
    }

    public void setStr(String str) {
        this.str = str;
    }

    public CalculateMsg(double i, double j, String str) {
        this.i = i;
        this.j = j;
        this.str = str;
    }
}

Add thread:

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * Add thread
 */
public class Plus implements Runnable {
    public static BlockingQueue<CalculateMsg> plusBq = new LinkedBlockingQueue<>();

    @Override
    public void run() {
        while (true) {
            try {
                ////Take the object that needs to be added
                CalculateMsg calculateMsg = plusBq.take();
                //Calculate the addition and set the value to the j attribute
                calculateMsg.setJ(calculateMsg.getI() + calculateMsg.getJ());
                //It is then passed to the multiply thread
                Multiply.multiplyBq.add(calculateMsg);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

Multiply thread:

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * Multiply thread
 */
public class Multiply implements Runnable {

    public static BlockingQueue<CalculateMsg> multiplyBq = new LinkedBlockingQueue<>();

    @Override
    public void run() {
        while (true) {
            try {
                //Take the object that needs multiplication
                CalculateMsg calculateMsg = multiplyBq.take();
                calculateMsg.setI(calculateMsg.getI() * calculateMsg.getJ());
                Div.divBq.add(calculateMsg);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

Divide thread:

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * Division thread
 */
public class Div implements Runnable {
    //Provides a latch for calculating the total time
    private static CountDownLatch countDownLatch = new CountDownLatch(10000);
    //Division object queue
    public static BlockingQueue<CalculateMsg> divBq = new LinkedBlockingQueue<>();

    @Override
    public void run() {
        while (true) {
            try {
                //Take the object to divide
                CalculateMsg calculateMsg = divBq.take();
                calculateMsg.setI(calculateMsg.getI() / 2);
                System.out.println(calculateMsg.getStr() + "=" + calculateMsg.getI());
                //Division is finished, indicating that the whole calculation logic of an object is finished, and the door latch is minus one
                countDownLatch.countDown();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

test method

    public static void main(String[] args) throws InterruptedException {
        //Start each thread
        new Thread(new Plus()).start();
        new Thread(new Multiply()).start();
        new Thread(new Div()).start();
        //start time
        long startTime = System.currentTimeMillis();
        for (int i = 0; i < 100; i++) {
            for (int j = 0; j < 100; j++) {
                CalculateMsg calculateMsg = new CalculateMsg(i, j, "(" + i + "+" + j + ")" + "*" + i + "/" + "2");
                Plus.plusBq.add(calculateMsg);
            }
        }
        //Wait until all objects are calculated before releasing the latch
        countDownLatch.await();
        long endTime = System.currentTimeMillis();
        System.out.println("Time consumed" + (endTime - startTime) + "ms");
    }

The operation results are as follows:

summary

The main purpose of using countDownLatch here is to do a time test, and then find that the time of serial computing i and j results is not slower than that of parallel computing. At least 10 million times of computing are tested, and it is not found that the time of parallel using is less than that of serial computing. So the above idea is just for sharing, not to say that the parallel efficiency will be higher than that of serial computing. The above example is clear evidence, but It is always good to understand the optimization idea of parallel pipeline.

Posted by d00dle on Sun, 28 Jun 2020 21:46:56 -0700