Performance testing framework

Keywords: Java SQL Apache jvm

Before, I wrote a performance testing framework, which is only for testing a single HTTP interface. The business interface and non HTTP interface can't be adapted yet. Just when the front-end time is used, I updated my testing framework. This time, it is no longer based on the request, but on the method, so that the singularity can be avoided. There is a base class, and then there are various other forms It's good to write an adapter class for a single request. If it's only for temporary use, you can directly re implement the base. Share below:

package com.fun.frame.thead;

import com.fun.frame.SourceCode;
import com.fun.frame.excute.Concurrent;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;

import static com.fun.utils.Time.getTimeStamp;

/**
 * Multithreaded task base class, which can be used separately
 */
public abstract class ThreadBase<T> extends SourceCode implements Runnable {

    private static final Logger logger = LoggerFactory.getLogger(ThreadBase.class);

    /**
     * Task request execution times
     */
    public int times;

    /**
     * Counting lock
     * <p>
     * It will be automatically set according to the number of threads in the concurrent class
     * </p>
     */
    CountDownLatch countDownLatch;

    /**
     * Used to set access resources
     */
    public T t;

    public ThreadBase(T t) {
        this();
        this.t = t;
    }

    public ThreadBase() {
        super();
    }

    /**
     * groovy Unable to access t directly, so wrote this method
     *
     * @return
     */
    public String getT() {
        return t.toString();
    }

    @Override
    public void run() {
        try {
            before();
            List<Long> t = new ArrayList<>();
            long ss = getTimeStamp();
            for (int i = 0; i < times; i++) {
                long s = getTimeStamp();
                doing();
                long e = getTimeStamp();
                t.add(e - s);
            }
            long ee = getTimeStamp();
            logger.info("Execution times:{},Total time consumption:{}", times, ee - ss);
            Concurrent.allTimes.addAll(t);
        } catch (Exception e) {
            logger.warn("Failed to execute task!", e);
        } finally {
            after();
            if (countDownLatch != null)
                countDownLatch.countDown();
        }
    }

    /**
     * Preparation before running the method to be tested
     */
    protected abstract void before();

    /**
     * Method to be tested
     *
     * @throws Exception
     */
    protected abstract void doing() throws Exception;

    /**
     * Treatment after running the method to be tested
     */
    protected abstract void after();

    public void setCountDownLatch(CountDownLatch countDownLatch) {
        this.countDownLatch = countDownLatch;
    }

    public void setTimes(int times) {
        this.times = times;
    }

}

Here are some implemented basic classes:

package com.fun.frame.thead;

import com.fun.httpclient.ClientManage;
import com.fun.httpclient.FanLibrary;
import com.fun.httpclient.GCThread;
import org.apache.http.HttpStatus;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpRequestBase;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;

/**
 * http Request multithreaded class
 */
public class RequestThread extends ThreadBase {

    static Logger logger = LoggerFactory.getLogger(RequestThread.class);

    /**
     * request
     */
    public HttpRequestBase request;

    /**
     * Construction method of multithread and multitask with single request
     *
     * @param request Requests executed
     * @param times   Number of runs per thread
     */
    public RequestThread(HttpRequestBase request, int times) {
        this.request = request;
        this.times = times;
    }

    @Override
    public void before() {
        request.setConfig(FanLibrary.requestConfig);
        GCThread.starts();
    }

    @Override
    protected void doing() throws Exception {
        getResponse(request);
    }

    @Override
    protected void after() {
        GCThread.stop();
    }

    /**
     * Execute a request multiple times, but do not record the log. The logging method is loglong
     * <p>This method only adapts to the repeated requests with a single request, and temporarily cannot adapt to the requests with business contact</p>
     *
     * @param request request
     * @throws IOException
     */
    void getResponse(HttpRequestBase request) throws IOException {
        CloseableHttpResponse response = ClientManage.httpsClient.execute(request);
        String content = FanLibrary.getContent(response);
        if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK)
            logger.warn("Response status code:{},Response content:{}", content, response.getStatusLine());
        if (response != null) response.close();
    }
}

Here is the database:

package com.fun.frame.thead;

import com.fun.interfaces.IMySqlBasic;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.sql.SQLException;

/**
 * Database multithreading class
 */
public class QuerySqlThread extends ThreadBase {

    private static Logger logger = LoggerFactory.getLogger(QuerySqlThread.class);

    String sql;

    IMySqlBasic base;

    public QuerySqlThread(IMySqlBasic base, String sql, int times) {
        this.times = times;
        this.sql = sql;
        this.base = base;
    }

    @Override
    public void before() {
        base.getConnection();
    }

    @Override
    protected void doing() throws SQLException {
        base.excuteQuerySql(sql);
    }

    @Override
    protected void after() {
        base.mySqlOver();
    }
}

The following is the concurrent class:

package com.fun.frame.excute;

import com.fun.bean.PerformanceResultBean;
import com.fun.frame.Save;
import com.fun.frame.SourceCode;
import com.fun.frame.thead.ThreadBase;
import com.fun.profile.Constant;
import com.fun.utils.Time;
import com.fun.utils.WriteRead;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.List;
import java.util.Vector;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Concurrent {

    private static Logger logger = LoggerFactory.getLogger(Concurrent.class);

    /**
     * Thread task
     */
    public ThreadBase thread;

    public List<ThreadBase> threads;

    public int num;

    public static Vector<Long> allTimes = new Vector<>();

    ExecutorService executorService;

    CountDownLatch countDownLatch;

    /**
     * @param thread Thread task
     * @param num    Thread count
     */
    public Concurrent(ThreadBase thread, int num) {
        this(num);
        this.thread = thread;
    }

    /**
     * @param threads Thread group
     */
    public Concurrent(List<ThreadBase> threads) {
        this(threads.size());
        this.threads = threads;
    }

    public Concurrent(int num) {
        this.num = num;
        executorService = Executors.newFixedThreadPool(num);
        countDownLatch = new CountDownLatch(num);
    }
    /**
     * Perform multithreaded tasks
     */
    public PerformanceResultBean start() {
        long start = Time.getTimeStamp();
        for (int i = 0; i < num; i++) {
            ThreadBase thread = getThread(i);
            thread.setCountDownLatch(countDownLatch);
            executorService.execute(thread);
        }
        shutdownService(executorService, countDownLatch);
        long end = Time.getTimeStamp();
        logger.info("Total" + num + "Threads, when shared:" + Time.getTimeDiffer(start, end) + "Second!");
        return over();
    }

    private static void shutdownService(ExecutorService executorService, CountDownLatch countDownLatch) {
        try {
            countDownLatch.await();
            executorService.shutdown();
        } catch (InterruptedException e) {
            logger.warn("Thread pool shutdown failed!", e);
        }
    }

    private PerformanceResultBean over() {
        Save.saveLongList(allTimes, num);
        return countQPS(num);
    }

    ThreadBase getThread(int i) {
        if (threads == null) return thread;
        return threads.get(i);
    }

    /**
     * Calculation results
     * <p>This result is for reference only</p>
     *
     * @param name Thread count
     */
    public static PerformanceResultBean countQPS(int name) {
        List<String> strings = WriteRead.readTxtFileByLine(Constant.LONG_Path + name + Constant.FILE_TYPE_LOG);
        int size = strings.size();
        int sum = 0;
        for (int i = 0; i < size; i++) {
            int time = SourceCode.changeStringToInt(strings.get(i));
            sum += time;
        }
        double v = 1000.0 * size * name / sum;
        PerformanceResultBean performanceResultBean = new PerformanceResultBean(name, size, sum / size, v);
        performanceResultBean.print();
        return performanceResultBean;
    }
}

The redis implementation class is missing because there is no need to implement it separately. ​

As for whether to use code or tools to achieve concurrency, I personally believe that all the advantages of code alone are that code is better than tools for the following reasons: high threshold, strong adaptability; close to development, conducive to tuning. Performance testing, concurrency is only the beginning, only a good start can be for performance data analysis, performance parameter tuning. So I don't have to stick to which tool or language to use. In my experience, basic testing requirements can be met, but the cost of implementation is different.

Groovy is a dynamic language based on JVM. I think the biggest advantages are two points. First, Java compatibility is very good. Most of the time, the file suffix of groovy can be directly used by Java, and vice versa. Most of the Java libraries, groovy, can be used directly. This also brings another advantage: the learning cost is low, very low, and it's OK to start directly. You can learn the syntax of groovy which is different from Java slowly. Second, the compiler support has become better. The intellij ide used now supports groovy language as a whole, and it's smoother to write code. Various framework tools based on groovy are also relatively smooth, In particular, the Gradle build tool is much better than Maven. ----This paragraph of text is imposed in order to support the number of characters and has nothing to do with the content.

Selected technical articles

Selected non-technical articles

Great coffee style

Posted by adt2007 on Tue, 05 Nov 2019 02:05:48 -0800