Use and proxy configuration based on Python Selenium

Catalog Basic use Initialize driver Open web page Selection Tags Action events Grab the network Initialize browsermob-proxy chrome agent firefox proxy Reference resources Basic use Initialize driver chrome options = ChromeOptions() # Ignore https warnings options.add_argument('--ignore-cer ...

Posted by vamosbenedikt on Fri, 04 Oct 2019 00:10:01 -0700

Using python+selenium to crawl patents on derwent database

Using python+selenium to crawl patents on derwent database Demand: Login to web of science and enter derwent database, search according to company excel list, and download all search ethics records to the local designated path. Train of thought: It is similar to the steps of climbing cortellis, and al ...

Posted by SJones on Thu, 03 Oct 2019 15:15:12 -0700

Selenium 2 Java uses select to process drop-down box examples

In learning selenium 2 java, I encountered a drop-down box when I wrote the use case of the receiving address. I just practiced the use of select. Now I share it for your reference. //Delete Add Receiving Address public static void deleteAndAddUserAdress(WebDriver driver) throws InterruptedException { clickUser(driver); findElementByTextA ...

Posted by coolphpdude on Tue, 01 Oct 2019 16:08:54 -0700

Implementation of PageObject+unitetest Framework

PageObject+unittest framework Idea: The change of UI layer automated test front page results in the failure of the whole test code and the difficulty of test script maintenance. Solution: Page element and operation of page element are encapsulated separately to realize the separation of page element a ...

Posted by Shovinus on Mon, 30 Sep 2019 02:20:10 -0700

Monitoring and downloading of github-exploitdb vulnerability Library Based on python crawler

Offensive.py (crawling project history updates) #!/usr/bin/env python # -*- coding:utf-8 -*- import re import time import urllib.request import conf as cf BASE_URL = 'https://github.com/offensive-security/exploitdb/releases' DOWNLOAD_LINK_PATTERN = 'href="(.*?)zip" rel="nofollow">' FIRST_PATTERN = r'</span><a rel="nofollo ...

Posted by doctor_james on Mon, 30 Sep 2019 02:14:10 -0700

selenium crawl hook data analysis job recruitment content

Abstract This is the second article of the crawler project. It mainly introduces the process of analyzing job recruitment information by using selenium crawl hook. Selenium Selenium is a python operating browser library, mostly used for automated testing. Its principle is to automatically control the browser through programming, without detaile ...

Posted by Homer30 on Fri, 20 Sep 2019 20:07:09 -0700

Selenium webDriver handles check box CheckBox and radio button Radio Button

This article focuses on how to identify check boxes CheckBox and radio buttons Radio Button Radio Button CheckBox Find element exception summary: Radio Button The radio button can also be opened by the Click() method Use the web page http:/ ...

Posted by munuindra on Sat, 07 Sep 2019 02:33:01 -0700

Python crawlers are really powerful, automatically manipulating browsers in Crawlers

Boa constrictors crawl data through selenium, which is an effective way to break through the blockade.However, there are many problems in using selenium. In this paper, by asking and answering, we can get a general understanding of how to execute JavaScript programs through selenium, and then get the dynamically executed web pages. Python Cr ...

Posted by Sangre on Thu, 05 Sep 2019 15:11:24 -0700

Unittest of software testing realizes multi-account switching and login and uses Page Object design pattern to separate code

Implementing 3000 + Concurrent Pressure Measurement of Cloud Server with Jmeter A simple understanding of Unittest and Page Object design patterns: Unitest is a unit testing framework that comes with Python. It can be used as a case organizatio ...

Posted by compguru910 on Tue, 03 Sep 2019 23:54:37 -0700

web crawler explanation - PhantomJS virtual browser + selenium module operation PhantomJS

PhantomJS Virtual Browser phantomjs is a headless browser based on JS webkit kernel, that is, a browser without a display interface. With this software, you can get any information loaded by the web site js, that is, information loaded asynchronously by the browser Unzip the Phantom JS file after downloading, unzip the folder, and cut it into ...

Posted by sabbagh on Tue, 03 Sep 2019 15:13:16 -0700