This article summarizes and introduces the development of interface test framework. The environment uses python3+selenium3+unittest+ddt+requests test framework and DDT data-driven, integrates test data functions such as Excel management test cases, and generates test reports using HTMLTestRunner. There are open source interface test tools such as poman and Jmeter. Why should we develop interface test framework?There are also a few shortcomings with interface testing tools.
- Test data is uncontrollable.For example, if the data returned by an interface is uncontrollable, it is impossible to automatically assert the data returned by the interface, whether it is caused by the interface program or by errors caused by changes in the test data, so it is necessary to do some initialization of the test data.Interface tools do not have the ability to initialize test data, and can not achieve true interface test automation.
- The cryptographic interface could not be tested.In practical projects, most interfaces are not callable at will, and encryption algorithms cannot be simulated and generated in general.For example, timestamps and MDB encryption algorithms, general interface tools cannot be simulated.
- Insufficient expansion capacity.Open source interface testing tools cannot implement extended functionality.For example, we want to generate test reports in different formats, send test reports to specified mailboxes, and integrate interface tests into CI for continuous integration timing tasks.
Test Framework Processing Process
The test framework is handled as follows:
- First, initialize data that empties the database tables and insert test data into the database.
- Call the interface provided by the system under test to read a row of excel use case data first by data driver;
- Send the request data and query the database to get the corresponding data according to the passed parameter data.
- Assemble the results of the query into data in JSON format, compare the returned data values with those of Excel, and write the results to the specified Excel test case table.
- Asserts the data returned by the interface through the unit test framework, generates the test report, and finally sends the HTML file that generates the latest test report to the specified mailbox.
Introduction to the test framework structure catalog
The directory structure is described below:
- config/:* File path configuration
- database/:* Test Case Template Files and database and Send Mailbox Profiles
- db_fixture/:* Initialize interface test data
- lib/:* Core program modules.Contains excel to parse, read, write, send mailbox, send request, generate latest test report file
- package/:* Store third-party library packages.For example, HTMLTestRunner, used to generate HTML format test reports
- report/:* Generate interface automated test reports
- Tescase/:* Used to write interface automated test cases
- run_demo.py:) The main program that executes all interface test cases
- GitHub project address:] https://github.com/yingoja/DemoAPI
Database Encapsulation
1 [tester] 2 name = Jason 3 4 [mysqlconf] 5 host = 127.0.0.1 6 port = 3306 7 user = root 8 password = 123456 9 db_name = guest 10 11 [user] 12 # Send Mailbox Server 13 HOST_SERVER = smtp.163.com 14 # Mail sender 15 FROM = 111@163.com 16 # mail recipient 17 TO = 222@126.com 18 # Send Mailbox User Name/Password 19 user = aaa 20 password = aaa 21 # Mail Subject 22 SUBJECT = Release automated test report for system interface
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 from config import setting 8 from pymysql import connect,cursors 9 from pymysql.err import OperationalError 10 import configparser as cparser 11 12 # --------- read config.ini configuration file --------------- 13 cf = cparser.ConfigParser() 14 cf.read(setting.TEST_CONFIG,encoding='UTF-8') 15 host = cf.get("mysqlconf","host") 16 port = cf.get("mysqlconf","port") 17 user = cf.get("mysqlconf","user") 18 password = cf.get("mysqlconf","password") 19 db = cf.get("mysqlconf","db_name") 20 21 class DB: 22 """ 23 MySQL basic operation 24 """ 25 def __init__(self): 26 try: 27 # Connect to database 28 self.conn = connect(host = host, 29 user = user, 30 password = password, 31 db = db, 32 charset = 'utf8mb4', 33 cursorclass = cursors.DictCursor 34 ) 35 except OperationalError as e: 36 print("Mysql Error %d: %s" % (e.args[0],e.args[1])) 37 38 # Clear Table Data 39 def clear(self,table_name): 40 real_sql = "delete from " + table_name + ";" 41 with self.conn.cursor() as cursor: 42 # Remove foreign key constraints from tables 43 cursor.execute("SET FOREIGN_KEY_CHECKS=0;") 44 cursor.execute(real_sql) 45 self.conn.commit() 46 47 # Insert Table Data 48 def insert(self, table_name, table_data): 49 for key in table_data: 50 table_data[key] = "'"+str(table_data[key])+"'" 51 key = ','.join(table_data.keys()) 52 value = ','.join(table_data.values()) 53 real_sql = "INSERT INTO " + table_name + " (" + key + ") VALUES (" + value + ")" 54 55 with self.conn.cursor() as cursor: 56 cursor.execute(real_sql) 57 self.conn.commit() 58 59 # close database 60 def close(self): 61 self.conn.close() 62 63 # Initialize data 64 def init_data(self, datas): 65 for table, data in datas.items(): 66 self.clear(table) 67 for d in data: 68 self.insert(table, d) 69 self.close()
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import sys, time, os 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 from db_fixture.mysql_db import DB 8 9 # Define Past Time 10 past_time = time.strftime("%Y-%m-%d %H:%M:%S",time.localtime(time.time()-100000)) 11 # Define the future time 12 future_time = time.strftime("%Y-%m-%d %H:%M:%S",time.localtime(time.time()+10000)) 13 14 # Create test data 15 datas = { 16 # Release table data 17 'sign_event':[ 18 {'id':1,'name':'Red rice Pro Release conference','`limit`':2000,'status':1,'address':'Beijing Convention and Exhibition Center','start_time':future_time}, 19 {'id':2,'name':'Apple iphon6 Release conference','`limit`':1000,'status':1,'address':'Bao'an Gymnasium','start_time':future_time}, 20 {'id':3,'name':'Huawei Glory 8 Conference','`limit`':2000,'status':0,'address':'Shenzhen Futian Convention and Exhibition Center','start_time':future_time}, 21 {'id':4,'name':'Apple iphon8 Release conference','`limit`':2000,'status':1,'address':'Shenzhen Bay Sports Center','start_time':past_time}, 22 {'id':5,'name':'Millet 5 Press','`limit`':2000,'status':1,'address':'Beijing National Convention Center','start_time':future_time}, 23 ], 24 # Guest Table Data 25 'sign_guest':[ 26 {'id':1,'realname':'Tom','phone':13511886601,'email':'alen@mail.com','sign':0,'event_id':1}, 27 {'id':2,'realname':'Jason','phone':13511886602,'email':'sign@mail.com','sign':1,'event_id':1}, 28 {'id':3,'realname':'Jams','phone':13511886603,'email':'tom@mail.com','sign':0,'event_id':5}, 29 ], 30 } 31 32 # Test data insertion table 33 def init_data(): 34 DB().init_data(datas)
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys 6 BASE_DIR = os.path.dirname(os.path.dirname(__file__)) 7 sys.path.append(BASE_DIR) 8 9 # configuration file 10 TEST_CONFIG = os.path.join(BASE_DIR,"database","config.ini") 11 # Test Case Template File 12 SOURCE_FILE = os.path.join(BASE_DIR,"database","DemoAPITestCase.xlsx") 13 # excel Test Case Result File 14 TARGET_FILE = os.path.join(BASE_DIR,"report","excelReport","DemoAPITestCase.xlsx") 15 # Test Case Report 16 TEST_REPORT = os.path.join(BASE_DIR,"report") 17 # Test Case Program File 18 TEST_CASE = os.path.join(BASE_DIR,"testcase")
Program Core Modules
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os 6 7 def new_report(testreport): 8 """ 9 Generate the latest test report file 10 :param testreport: 11 :return:Return to file 12 """ 13 lists = os.listdir(testreport) 14 lists.sort(key=lambda fn: os.path.getmtime(testreport + "\\" + fn)) 15 file_new = os.path.join(testreport,lists[-1]) 16 return file_new
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import xlrd 6 7 class ReadExcel(): 8 """read excel file data""" 9 def __init__(self,fileName, SheetName="Sheet1"): 10 self.data = xlrd.open_workbook(fileName) 11 self.table = self.data.sheet_by_name(SheetName) 12 13 # Get total rows, columns 14 self.nrows = self.table.nrows 15 self.ncols = self.table.ncols 16 def read_data(self): 17 if self.nrows > 1: 18 # Get the contents of the first line, list format 19 keys = self.table.row_values(0) 20 listApiData = [] 21 # Get the contents of each line, list format 22 for col in range(1, self.nrows): 23 values = self.table.row_values(col) 24 # keys,values Combination to Dictionary 25 api_dict = dict(zip(keys, values)) 26 listApiData.append(api_dict) 27 return listApiData 28 else: 29 print("Table is empty data!") 30 return None
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys,json 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 8 9 class SendRequests(): 10 """Send Request Data""" 11 def sendRequests(self,s,apiData): 12 try: 13 #Parameters that get the response from the read table as a pass 14 method = apiData["method"] 15 url = apiData["url"] 16 if apiData["params"] == "": 17 par = None 18 else: 19 par = eval(apiData["params"]) 20 if apiData["headers"] == "": 21 h = None 22 else: 23 h = eval(apiData["headers"]) 24 if apiData["body"] == "": 25 body_data = None 26 else: 27 body_data = eval(apiData["body"]) 28 type = apiData["type"] 29 v = False 30 if type == "data": 31 body = body_data 32 elif type == "json": 33 body = json.dumps(body_data) 34 else: 35 body = body_data 36 37 #Send Request 38 re = s.request(method=method,url=url,headers=h,params=par,data=body,verify=v) 39 return re 40 except Exception as e: 41 print(e)
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 from config import setting 8 import smtplib 9 from lib.newReport import new_report 10 import configparser 11 from email.mime.text import MIMEText 12 from email.mime.multipart import MIMEMultipart 13 14 15 def send_mail(file_new): 16 """ 17 Define Send Mail 18 :param file_new: 19 :return: Success: Print send mailbox successfully; Failure: Return failure information 20 """ 21 f = open(file_new,'rb') 22 mail_body = f.read() 23 f.close() 24 #Send Attachments 25 con = configparser.ConfigParser() 26 con.read(setting.TEST_CONFIG,encoding='utf-8') 27 report = new_report(setting.TEST_REPORT) 28 sendfile = open(report,'rb').read() 29 # --------- read config.ini configuration file --------------- 30 HOST = con.get("user","HOST_SERVER") 31 SENDER = con.get("user","FROM") 32 RECEIVER = con.get("user","TO") 33 USER = con.get("user","user") 34 PWD = con.get("user","password") 35 SUBJECT = con.get("user","SUBJECT") 36 37 att = MIMEText(sendfile,'base64','utf-8') 38 att["Content-Type"] = 'application/octet-stream' 39 att.add_header("Content-Disposition", "attachment", filename=("gbk", "", report)) 40 41 msg = MIMEMultipart('related') 42 msg.attach(att) 43 msgtext = MIMEText(mail_body,'html','utf-8') 44 msg.attach(msgtext) 45 msg['Subject'] = SUBJECT 46 msg['from'] = SENDER 47 msg['to'] = RECEIVER 48 49 try: 50 server = smtplib.SMTP() 51 server.connect(HOST) 52 server.starttls() 53 server.login(USER,PWD) 54 server.sendmail(SENDER,RECEIVER,msg.as_string()) 55 server.quit() 56 print("Mail sent successfully!") 57 except Exception as e: 58 print("fail: " + str(e))
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 import shutil 8 from config import setting 9 from openpyxl import load_workbook 10 from openpyxl.styles import Font,Alignment 11 from openpyxl.styles.colors import RED,GREEN,DARKYELLOW 12 import configparser as cparser 13 14 # --------- read config.ini configuration file --------------- 15 cf = cparser.ConfigParser() 16 cf.read(setting.TEST_CONFIG,encoding='UTF-8') 17 name = cf.get("tester","name") 18 19 class WriteExcel(): 20 """File Write Data""" 21 def __init__(self,fileName): 22 self.filename = fileName 23 if not os.path.exists(self.filename): 24 # If the file does not exist, copy the template file to the specified report directory 25 shutil.copyfile(setting.SOURCE_FILE,setting.TARGET_FILE) 26 self.wb = load_workbook(self.filename) 27 self.ws = self.wb.active 28 29 def write_data(self,row_n,value): 30 """ 31 Write test results 32 :param row_n:Number of rows of data 33 :param value: Test Result Value 34 :return: nothing 35 """ 36 font_GREEN = Font(name='Song Style', color=GREEN, bold=True) 37 font_RED = Font(name='Song Style', color=RED, bold=True) 38 font1 = Font(name='Song Style', color=DARKYELLOW, bold=True) 39 align = Alignment(horizontal='center', vertical='center') 40 # Number of rows to get 41 L_n = "L" + str(row_n) 42 M_n = "M" + str(row_n) 43 if value == "PASS": 44 self.ws.cell(row_n, 12, value) 45 self.ws[L_n].font = font_GREEN 46 if value == "FAIL": 47 self.ws.cell(row_n, 12, value) 48 self.ws[L_n].font = font_RED 49 self.ws.cell(row_n, 13, name) 50 self.ws[L_n].alignment = align 51 self.ws[M_n].font = font1 52 self.ws[M_n].alignment = align 53 self.wb.save(self.filename)
Interface Test Case Writing
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 import os,sys 6 sys.path.append(os.path.dirname(os.path.dirname(__file__))) 7 import unittest,requests,ddt 8 from config import setting 9 from lib.readexcel import ReadExcel 10 from lib.sendrequests import SendRequests 11 from lib.writeexcel import WriteExcel 12 13 testData = ReadExcel(setting.SOURCE_FILE, "Sheet1").read_data() 14 15 @ddt.ddt 16 class Demo_API(unittest.TestCase): 17 """Release system""" 18 def setUp(self): 19 self.s = requests.session() 20 21 def tearDown(self): 22 pass 23 24 @ddt.data(*testData) 25 def test_api(self,data): 26 # Obtain ID Field value, truncate the end number and remove the start 0 27 rowNum = int(data['ID'].split("_")[2]) 28 # Send Request 29 re = SendRequests().sendRequests(self.s,data) 30 # Get the value returned by the server 31 self.result = re.json() 32 # Obtain excel Status codes and messages for tabular data 33 readData_code = int(data["status_code"]) 34 readData_msg = data["msg"] 35 if readData_code == self.result['status'] and readData_msg == self.result['message']: 36 OK_data = "PASS" 37 WriteExcel(setting.TARGET_FILE).write_data(rowNum + 1,OK_data) 38 if readData_code != self.result['status'] or readData_msg != self.result['message']: 39 NOT_data = "FAIL" 40 WriteExcel(setting.TARGET_FILE).write_data(rowNum + 1,NOT_data) 41 self.assertEqual(self.result['status'], readData_code, "Return the actual result->:%s" % self.result['status']) 42 self.assertEqual(self.result['message'], readData_msg, "Return the actual result->:%s" % self.result['message']) 43 44 if __name__=='__main__': 45 unittest.main()
Integrated Test Report
1 #!/usr/bin/env python 2 # _*_ coding:utf-8 _*_ 3 __author__ = 'YinJia' 4 5 6 import os,sys 7 sys.path.append(os.path.dirname(__file__)) 8 from config import setting 9 import unittest,time 10 from HTMLTestRunner import HTMLTestRunner 11 from lib.sendmail import send_mail 12 from lib.newReport import new_report 13 from db_fixture import test_data 14 from package.HTMLTestRunner import HTMLTestRunner 15 16 def add_case(test_path=setting.TEST_CASE): 17 """Load all test cases""" 18 discover = unittest.defaultTestLoader.discover(test_path, pattern='*API.py') 19 return discover 20 21 def run_case(all_case,result_path=setting.TEST_REPORT): 22 """Execute all test cases""" 23 24 # Initialize interface test data 25 test_data.init_data() 26 27 now = time.strftime("%Y-%m-%d %H_%M_%S") 28 filename = result_path + '/' + now + 'result.html' 29 fp = open(filename,'wb') 30 runner = HTMLTestRunner(stream=fp,title='Release automated test report for system interface', 31 description='Environmental Science: windows 7 Browser: chrome', 32 tester='Jason') 33 runner.run(all_case) 34 fp.close() 35 report = new_report(setting.TEST_REPORT) #Call module to generate latest report 36 send_mail(report) #Invoke Send Mail Module 37 38 if __name__ =="__main__": 39 cases = add_case() 40 run_case(cases)
Test Result Display
- HTML Test Results Report:
- Excel Test Case Results
- Test report received by mail