Multi thread crawling agent and verifying

Preface One of the most common ways in anti crawler is to determine the frequency of your requests. If you send a large number of requests in a short period of time, whether you are a person or not, first seal your account or IP for a period of time. At this time, in order to achieve the goal of their ...

Posted by Jen_u41 on Wed, 29 Jan 2020 21:34:19 -0800

Solution of "running package script" for MAC installing MacPorts card

Click the upper right corner to force the end of "installer" Open terminal, enter the command "ps -ef | grep install", find the legacy process and kill it Install Xcode Enter the command "xcodebuild -license" and agree MacPorts is stuck because the installation script uses Rsync update during the installation pro ...

Posted by swathin2 on Thu, 09 Jan 2020 07:11:03 -0800

Brute force cracking adds form request for token verification

In this module, a hidden field user "token is added to prevent brute force cracking, and a new token value will be generated every time the page is visited. The submitted request must carry this value and each value can only be used once. In this case, burp cannot be used directly for simple enumeration. You can write a python script to e ...

Posted by dta on Tue, 31 Dec 2019 12:27:07 -0800

Mobile (mobile) page adaptive solution - rem layout

Let's assume that the size of the design draft that my sister gave us is 750 * 1340. Combined with the dynamic font size attribute, design draft size and collaboration process between the front end and design on the html element of the home page of Netease and Taobao mobile terminal, they are generally divided into the following two types: 1. N ...

Posted by AlGale on Mon, 30 Dec 2019 23:47:58 -0800

Use the python analysis fund to analyze the data properly to make money at the starting line!

Preface The text and pictures in this article are from the Internet. They are for study and communication only. They do not have any commercial use. The copyright is owned by the original author. If you have any questions, please contact us in time for processing. Author: White Jade without Ice PS: If you need Python learning materials for your ...

Posted by markspec87 on Mon, 09 Dec 2019 21:06:17 -0800

Crawling knowledge planet with python

Last year, we did a community activity called "question and answer for seniors", which precipitated a lot of valuable interactive information, and then it was terminated for various reasons. Today, I started to talk with Tu Teng, and thought that the information was so quiet and wasteful. So try to use python to crawl the content of t ...

Posted by kylera on Sat, 07 Dec 2019 21:05:17 -0800

Basic crawler, who can learn? Use requests and regular expressions to crawl the top 250 movie data of Douban!

Crawls to take the Douban top 250 movie score, the poster, the movie review and so on data!   this project is the most basic and simple example among reptiles; Later, there will be more advanced and automated crawlers using the crawler framework.   the process of this project is to use requests request library to get html, and then us ...

Posted by V-Man on Wed, 04 Dec 2019 02:47:50 -0800

Automating your git workflow with git-flow

Introduce to you The git flow branch model is believed to have been heard more or less by everyone. Let's start with Zhang Tu Townhouse: It doesn't matter if I don't understand the diagram above (I don't know==), but today I'm talking about the git-flow command line tool developed from this branch model.Just remember a few simple commands to s ...

Posted by lostnucleus on Sun, 17 Nov 2019 18:18:02 -0800

Under the shader of OpenGL learning

Previous section On the shaders of OpenGL learning In this section, we will create a shader class and use the C + + file stream to read the contents of the shader. I. create shader class We can directly right-click the solution and select new class. This will automatically generate a. h and a. cpp file. We can also create a. h a ...

Posted by vinnier on Tue, 12 Nov 2019 07:23:00 -0800

UA pool and IP proxy pool use

UA pool: user agent pool -Function: disguise the requests in the project as different types of browser identity as much as possible. -Operation process: 1. Intercept requests in download Middleware 2. Disguise the UA in the intercepted request header information 3. Enable download Middleware in configuration file Code display #Guide b ...

Posted by zymosan on Thu, 31 Oct 2019 17:45:30 -0700