In the last three articles, we used the automated tools to integrate Jenkins Docker Cloud with Robot Framework + Selenium + Chome, and realized the automated testing of the Web in Pipeline, from the installation of the Web automated testing tools under windows to the containerization of the Web automated testing tool Robot Framework + Selenium + Chome.
For links to the three articles, see the following: uuuuuuuuuuu
Devops Key Tools and Technologies (6) - Pipeline-based Web Automated Testing (Installation of Automated Testing Tools under Windows)
Devops Key Tools and Technologies (6) - Pipeline-based Web Automated Testing (Web Automated Testing Tool Containerization)
Devops Key Tools and Technologies (6) - Pipeline-based Web Automation Testing (Pipeline Integration of Web Automation Testing)
With automated testing, natural performance testing also needs to be part of Pipeline. The common tools for performance testing are Jmeter and LoadRunner. We will choose Jmeter as our protagonist. Because it's free. Since it is relatively simple to install Jmeter under Windows, we will introduce two articles in Jmeter performance testing. Namely Containerization of Jmeter Performance Testing,Pipeline Integration of Jmeter Performance Testing.
In this article, we will first introduce the pipeline integration of Jmeter performance testing.
1. Performance Test Node
We will refer to it. Devops Key Tools and Technologies (I) - Jenkins Containerization and Devops Key Tools and Technologies (6) - Pipeline-based Web Automation Testing (Pipeline Integration of Web Automation Testing) By adding Slave and Robot nodes, Jmeter performance test nodes are added to Jenkins in order to make Jmeter performance test part of the pipeline.
Before that, we need to containerize Jmeter tests, which can be referred to in detail. Devops Key Tools and Technologies (7) - Pipeline-based Jmeter Performance Testing (Jmeter Containerization).
- Information on configuring Jmeter performance test nodes on JenkinsMaster
Before configuring, we also need to configure the joining nodes. The specific operation can be learned from the two articles mentioned above. There is no much explanation here. Refer to the screenshot below.
2. Pipeline pipeline integration
Based on the previous pipeline of Web automation testing, we added Stage for Jmeter performance testing. Before that, we added Checkout Code, Mvn Build, Sonar, Bash Deploy, Robot Framework and other Stages to our Pipeline. This time, we added Jmeter performance test Stage to our Pipeline. As follows:
- Performance test script
The script can be found here: https://github.com/zbbkeepgoing/springboot-demo/blob/master/jmeter/demo.jmx
<?xml version="1.0" encoding="UTF-8"?> <jmeterTestPlan version="1.2" properties="5.0" jmeter="5.0 r1840935"> <hashTree> <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Demo" enabled="true"> <stringProp name="TestPlan.comments"></stringProp> <boolProp name="TestPlan.functional_mode">false</boolProp> <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp> <boolProp name="TestPlan.serialize_threadgroups">false</boolProp> <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true"> <collectionProp name="Arguments.arguments"/> </elementProp> <stringProp name="TestPlan.user_define_classpath"></stringProp> </TestPlan> <hashTree> <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true"> <stringProp name="ThreadGroup.on_sample_error">continue</stringProp> <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true"> <boolProp name="LoopController.continue_forever">false</boolProp> <stringProp name="LoopController.loops">10</stringProp> </elementProp> <stringProp name="ThreadGroup.num_threads">2</stringProp> <stringProp name="ThreadGroup.ramp_time">0</stringProp> <boolProp name="ThreadGroup.scheduler">false</boolProp> <stringProp name="ThreadGroup.duration"></stringProp> <stringProp name="ThreadGroup.delay"></stringProp> </ThreadGroup> <hashTree> <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="index" enabled="true"> <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true"> <collectionProp name="Arguments.arguments"/> </elementProp> <stringProp name="HTTPSampler.domain">192.168.88.130</stringProp> <stringProp name="HTTPSampler.port">8080</stringProp> <stringProp name="HTTPSampler.protocol">http</stringProp> <stringProp name="HTTPSampler.contentEncoding"></stringProp> <stringProp name="HTTPSampler.path">/</stringProp> <stringProp name="HTTPSampler.method">GET</stringProp> <boolProp name="HTTPSampler.follow_redirects">true</boolProp> <boolProp name="HTTPSampler.auto_redirects">false</boolProp> <boolProp name="HTTPSampler.use_keepalive">true</boolProp> <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp> <stringProp name="HTTPSampler.embedded_url_re"></stringProp> <stringProp name="HTTPSampler.connect_timeout"></stringProp> <stringProp name="HTTPSampler.response_timeout"></stringProp> </HTTPSamplerProxy> <hashTree/> <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="rightaway" enabled="true"> <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true"> <collectionProp name="Arguments.arguments"/> </elementProp> <stringProp name="HTTPSampler.domain">192.168.88.130</stringProp> <stringProp name="HTTPSampler.port">8080</stringProp> <stringProp name="HTTPSampler.protocol">http</stringProp> <stringProp name="HTTPSampler.contentEncoding"></stringProp> <stringProp name="HTTPSampler.path">/rightaway</stringProp> <stringProp name="HTTPSampler.method">GET</stringProp> <boolProp name="HTTPSampler.follow_redirects">true</boolProp> <boolProp name="HTTPSampler.auto_redirects">false</boolProp> <boolProp name="HTTPSampler.use_keepalive">true</boolProp> <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp> <stringProp name="HTTPSampler.embedded_url_re"></stringProp> <stringProp name="HTTPSampler.connect_timeout"></stringProp> <stringProp name="HTTPSampler.response_timeout"></stringProp> </HTTPSamplerProxy> <hashTree/> <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="sleep" enabled="true"> <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true"> <collectionProp name="Arguments.arguments"/> </elementProp> <stringProp name="HTTPSampler.domain">192.168.88.130</stringProp> <stringProp name="HTTPSampler.port">8080</stringProp> <stringProp name="HTTPSampler.protocol">http</stringProp> <stringProp name="HTTPSampler.contentEncoding"></stringProp> <stringProp name="HTTPSampler.path">/sleep</stringProp> <stringProp name="HTTPSampler.method">GET</stringProp> <boolProp name="HTTPSampler.follow_redirects">true</boolProp> <boolProp name="HTTPSampler.auto_redirects">false</boolProp> <boolProp name="HTTPSampler.use_keepalive">true</boolProp> <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp> <stringProp name="HTTPSampler.embedded_url_re"></stringProp> <stringProp name="HTTPSampler.connect_timeout"></stringProp> <stringProp name="HTTPSampler.response_timeout"></stringProp> </HTTPSamplerProxy> <hashTree/> <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true"> <boolProp name="ResultCollector.error_logging">false</boolProp> <objProp> <name>saveConfig</name> <value class="SampleSaveConfiguration"> <time>true</time> <latency>true</latency> <timestamp>true</timestamp> <success>true</success> <label>true</label> <code>true</code> <message>true</message> <threadName>true</threadName> <dataType>true</dataType> <encoding>false</encoding> <assertions>true</assertions> <subresults>true</subresults> <responseData>false</responseData> <samplerData>false</samplerData> <xml>false</xml> <fieldNames>true</fieldNames> <responseHeaders>false</responseHeaders> <requestHeaders>false</requestHeaders> <responseDataOnError>false</responseDataOnError> <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage> <assertionsResultsToSave>0</assertionsResultsToSave> <bytes>true</bytes> <sentBytes>true</sentBytes> <url>true</url> <threadCounts>true</threadCounts> <idleTime>true</idleTime> <connectTime>true</connectTime> </value> </objProp> <stringProp name="filename"></stringProp> </ResultCollector> <hashTree/> <ResultCollector guiclass="SummaryReport" testclass="ResultCollector" testname="Summary Report" enabled="true"> <boolProp name="ResultCollector.error_logging">false</boolProp> <objProp> <name>saveConfig</name> <value class="SampleSaveConfiguration"> <time>true</time> <latency>true</latency> <timestamp>true</timestamp> <success>true</success> <label>true</label> <code>true</code> <message>true</message> <threadName>true</threadName> <dataType>true</dataType> <encoding>false</encoding> <assertions>true</assertions> <subresults>true</subresults> <responseData>false</responseData> <samplerData>false</samplerData> <xml>false</xml> <fieldNames>true</fieldNames> <responseHeaders>false</responseHeaders> <requestHeaders>false</requestHeaders> <responseDataOnError>false</responseDataOnError> <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage> <assertionsResultsToSave>0</assertionsResultsToSave> <bytes>true</bytes> <sentBytes>true</sentBytes> <url>true</url> <threadCounts>true</threadCounts> <idleTime>true</idleTime> <connectTime>true</connectTime> </value> </objProp> <stringProp name="filename"></stringProp> </ResultCollector> <hashTree/> <ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true"> <boolProp name="ResultCollector.error_logging">false</boolProp> <objProp> <name>saveConfig</name> <value class="SampleSaveConfiguration"> <time>true</time> <latency>true</latency> <timestamp>true</timestamp> <success>true</success> <label>true</label> <code>true</code> <message>true</message> <threadName>true</threadName> <dataType>true</dataType> <encoding>false</encoding> <assertions>true</assertions> <subresults>true</subresults> <responseData>false</responseData> <samplerData>false</samplerData> <xml>false</xml> <fieldNames>true</fieldNames> <responseHeaders>false</responseHeaders> <requestHeaders>false</requestHeaders> <responseDataOnError>false</responseDataOnError> <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage> <assertionsResultsToSave>0</assertionsResultsToSave> <bytes>true</bytes> <sentBytes>true</sentBytes> <url>true</url> <threadCounts>true</threadCounts> <idleTime>true</idleTime> <connectTime>true</connectTime> </value> </objProp> <stringProp name="filename"></stringProp> </ResultCollector> <hashTree/> </hashTree> </hashTree> </hashTree> </jmeterTestPlan>
- Pipeline content
Content can also be found in Github
https://github.com/zbbkeepgoing/pipeline-sample
pipeline { agent none stages { stage('Preparation') { agent { node { label 'master' } } steps { checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]]) } } stage('Build') { agent { node { label 'master' } } steps { dir(env.WORKSPACE){ sh "mvn clean install" junit allowEmptyResults: true, keepLongStdio: true, testResults: 'target/surefire-reports/*.xml' sh "mv target/sample-0.0.1-SNAPSHOT.jar target/sample.jar" } } } stage('Sonarqube') { agent { node { label 'master' } } steps { dir(env.WORKSPACE){ sh "mvn sonar:sonar -Dsonar.host.url=http://192.168.88.130:9000 -Dsonar.login=65607ba9d0f54590cf55fe8e60134fb5e87c557d" } } } stage('Deploy') { agent { node { label 'master' } } steps { dir(env.WORKSPACE){ sh 'sshpass -p 1qaz@WSX scp -o StrictHostKeychecking=no target/sample.jar root@192.168.88.130:/opt/ansible' sh 'sshpass -p 1qaz@WSX scp -o StrictHostKeychecking=no deploy.sh root@192.168.88.130:/opt/ansible' sh 'sshpass -p 1qaz@WSX ssh -o StrictHostKeychecking=no root@192.168.88.130 "bash /opt/ansible/deploy.sh"' sh 'sleep 8s' } } } stage('Robot Framework') { agent { node { label 'robot' } } steps { dir(env.WORKSPACE){ checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]]) sh "pybot -d /opt/workspace/CI+Sonar+Sh+Robot+Jmeter robot/demo.robot" step([$class: 'RobotPublisher', disableArchiveOutput: false, logFileName: 'log.html', otherFiles: '', outputFileName: 'output.xml', outputPath: '.', passThreshold: 40, reportFileName: 'report.html', unstableThreshold: 0]); } } } stage('Jmeter') { agent { node { label 'jmeter' } } steps { sh "rm -rf /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/*" //The directory that dropped the target last time, if the following command is executed to output the file, the file will continue to exist and error will be reported, and the file can not be generated. checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']// pull the latest code because the script is in our repo sh "mkdir -p /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/output" //Create the output directory generated by Jmeter sh "jmeter.sh -n -t /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.jmx -l /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.jtl -j /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.log -e -o /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/output" //Command Line for Performance Testing step([$class: 'ArtifactArchiver', artifacts: 'jmeter/*,jmeter/output/*']) //File all the files generated by Jmeter and provide page view directly after Jenkins Build perfReport "jmeter/demo.jtl" //Generate a Jmeter report (which Jenkins analyzed through the performance plug-in). Jmeter's own report also needs to look at the files in output. } } } }
-
New Pipeline
-
Execute Pipeline
Started by user admin Running in Durability level: MAX_SURVIVABILITY ...... Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] checkout ...... Commit message: "Update deploy.sh" > git rev-list --no-walk 1aa03a20b88565b775a23a759fde2701cabe8592 # timeout=10 ...... [Pipeline] node Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] dir Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + mvn clean install ...... ------------------------------------------------------- T E S T S ------------------------------------------------------- ...... Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 [INFO] ...... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 31.154 s [INFO] Finished at: 2018-10-20T09:27:22+00:00 [INFO] Final Memory: 29M/69M [INFO] ------------------------------------------------------------------------ [Pipeline] junit Recording test results [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + mv target/sample-0.0.1-SNAPSHOT.jar target/sample.jar ...... [Pipeline] { (Sonarqube) [Pipeline] node Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] dir Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + mvn sonar:sonar -Dsonar.host.url=http://192.168.88.130:9000 -Dsonar.login=65607ba9d0f54590cf55fe8e60134fb5e87c557d [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building demo 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ ...... [INFO] Task total time: 22.811 s [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 32.828 s [INFO] Finished at: 2018-10-20T09:28:02+00:00 [INFO] Final Memory: 32M/154M [INFO] ------------------------------------------------------------------------ ...... [Pipeline] { (Deploy) [Pipeline] node Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] dir Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + sshpass -p 1qaz@WSX scp -o StrictHostKeychecking=no target/sample.jar root@192.168.88.130:/opt/ansible [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + sshpass -p 1qaz@WSX scp -o StrictHostKeychecking=no deploy.sh root@192.168.88.130:/opt/ansible [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + sshpass -p 1qaz@WSX ssh -o StrictHostKeychecking=no root@192.168.88.130 bash /opt/ansible/deploy.sh [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + sleep 8s ...... [Pipeline] { (Robot Framework) [Pipeline] node Still waiting to schedule task robot-0000nio3k3235 is offline Running on robot-0000nio3k3235 on 192.168.88.130robot in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] dir Running in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] checkout ...... Commit message: "Update deploy.sh" [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + pybot -d /opt/workspace/CI+Sonar+Sh+Robot+Jmeter robot/demo.robot ============================================================================== Demo ============================================================================== Demo | PASS | ------------------------------------------------------------------------------ Baidu | PASS | ------------------------------------------------------------------------------ Demo | PASS | 2 critical tests, 2 passed, 0 failed 2 tests total, 2 passed, 0 failed ============================================================================== Output: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/output.xml Log: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/log.html Report: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/report.html [Pipeline] step Robot results publisher started... -Parsing output xml: Done! -Copying log files to build dir: Done! -Assigning results to build: Done! -Checking thresholds: Done! Done publishing Robot results. ...... [Pipeline] { (Jmeter) [Pipeline] node Running on jmeter-0000nkxischdk on 192.168.88.130jmeter in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter [Pipeline] { [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + rm -rf /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/* [Pipeline] checkout Cloning the remote Git repository Cloning repository https://github.com/zbbkeepgoing/springboot-demo.git ...... Checking out Revision 1aa03a20b88565b775a23a759fde2701cabe8592 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 1aa03a20b88565b775a23a759fde2701cabe8592 Commit message: "Update deploy.sh" [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + mkdir -p /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/output [Pipeline] sh [CI+Sonar+Sh+Robot+Jmeter] Running shell script + jmeter.sh -n -t /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.jmx -l /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.jtl -j /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.log -e -o /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/output Oct 20, 2018 9:32:29 AM java.util.prefs.FileSystemPreferences$1 run INFO: Created user preferences directory. Creating summariser <summary> Created the tree successfully using /opt/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter/demo.jmx Starting the test @ Sat Oct 20 09:32:31 UTC 2018 (1540027951645) Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445 summary + 1 in 00:00:01 = 1.1/s Avg: 182 Min: 182 Max: 182 Err: 0 (0.00%) Active: 2 Started: 2 Finished: 0 ...... Tidying up ... @ Sat Oct 20 09:33:50 UTC 2018 (1540028030668) ... end of run [Pipeline] archiveArtifacts Archiving artifacts [Pipeline] perfReport Performance: Recording JMeterCsv reports 'jmeter/demo.jtl' ...... Performance: File demo.jtl reported 0.0% of errors [SUCCESS]. Build status is: SUCCESS [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // stage [Pipeline] End of Pipeline Finished: SUCCESS
- View and obtain Jmeter test reports
There are three places where we can view and get reports:
1. The results of Jenkins construction are as follows:
2. Click on the last successful product in the picture above. You can download the original report for viewing.
3. Under the Host Directory
[root@localhost jmeter]# ll total 40 -rw-r--r--. 1 root root 10762 Oct 20 17:32 demo.jmx -rw-r--r--. 1 root root 6810 Oct 20 17:33 demo.jtl -rw-r--r--. 1 root root 17438 Oct 20 17:33 demo.log drwxr-xr-x. 4 root root 79 Oct 20 17:33 output [root@localhost jmeter]# pwd /opt/jmeter/workspace/CI+Sonar+Sh+Robot+Jmeter/jmeter [root@localhost jmeter]# cd output/ [root@localhost output]# ls content index.html README.TXT sbadmin2-1.0.7 [root@localhost output]# ll total 28 drwxr-xr-x. 5 root root 40 Oct 20 17:33 content -rw-r--r--. 1 root root 9465 Oct 20 17:33 index.html -rw-r--r--. 1 root root 15437 Oct 20 17:33 README.TXT drwxr-xr-x. 5 root root 89 Oct 20 17:33 sbadmin2-1.0.7 [root@localhost output]#
That's all about our Jmeter performance test integration with Pipeline.