< attacking entomologist > data acquisition of public opinion monitoring

Public opinion monitoring is the behavior of monitoring and predicting the public opinions and opinions on the Internet. Most monitoring technologies are based on crawlers. If we search the keywords of relevant hot events with search engines and save the results locally, the first link of public opinion monitoring is realized: rea ...

Posted by andre&rachel on Sat, 04 Apr 2020 08:56:05 -0700

Learning notes CB005: key words, corpus extraction

Keyword extraction. The pynlpir library implements keyword extraction. # coding:utf-8 import sys import importlib importlib.reload(sys) import pynlpir pynlpir.open() s = 'How to delete the junk files in the computer' key_words = pynlpir.get_key_words(s, weighted=True) for key_word in key_words: print(key_word[0], 't', key_word[1]) pynl ...

Posted by Cloud9247 on Thu, 02 Apr 2020 09:32:17 -0700

[ASP.NET Core 3.1] Browser sniffing solves missing cookies in some browsers

[ASP.NET Core 3.1] Browser sniffing solves missing cookies in some browsersStudents who have read the previous article should know that browsers such as Sogou and 360 redirect repeatedly in single sign-on, and eventually fail and make mistakes. The reason is that non-Chrome80+ browsers do not recognize the SameSite=none attribute value on the C ...

Posted by jallard on Mon, 30 Mar 2020 19:53:37 -0700

What did Runtime do during the startup of Runtime objc4-779.1 App?

Knowledge preparation First of all, we need to know a scenario. We click on the application on the screen, and then we can see that the application is fully displayed and can be operated. In this process, the system, runtime and our own code have done a lot of work, and many excellent blogs have descr ...

Posted by smerny on Tue, 10 Mar 2020 04:09:32 -0700

Multi thread crawling agent and verifying

Preface One of the most common ways in anti crawler is to determine the frequency of your requests. If you send a large number of requests in a short period of time, whether you are a person or not, first seal your account or IP for a period of time. At this time, in order to achieve the goal of their ...

Posted by Jen_u41 on Wed, 29 Jan 2020 21:34:19 -0800

Solution of "running package script" for MAC installing MacPorts card

Click the upper right corner to force the end of "installer" Open terminal, enter the command "ps -ef | grep install", find the legacy process and kill it Install Xcode Enter the command "xcodebuild -license" and agree MacPorts is stuck because the installation script uses Rsync update during the installation pro ...

Posted by swathin2 on Thu, 09 Jan 2020 07:11:03 -0800

Brute force cracking adds form request for token verification

In this module, a hidden field user "token is added to prevent brute force cracking, and a new token value will be generated every time the page is visited. The submitted request must carry this value and each value can only be used once. In this case, burp cannot be used directly for simple enumeration. You can write a python script to e ...

Posted by dta on Tue, 31 Dec 2019 12:27:07 -0800

Mobile (mobile) page adaptive solution - rem layout

Let's assume that the size of the design draft that my sister gave us is 750 * 1340. Combined with the dynamic font size attribute, design draft size and collaboration process between the front end and design on the html element of the home page of Netease and Taobao mobile terminal, they are generally divided into the following two types: 1. N ...

Posted by AlGale on Mon, 30 Dec 2019 23:47:58 -0800

Use the python analysis fund to analyze the data properly to make money at the starting line!

Preface The text and pictures in this article are from the Internet. They are for study and communication only. They do not have any commercial use. The copyright is owned by the original author. If you have any questions, please contact us in time for processing. Author: White Jade without Ice PS: If you need Python learning materials for your ...

Posted by markspec87 on Mon, 09 Dec 2019 21:06:17 -0800

Crawling knowledge planet with python

Last year, we did a community activity called "question and answer for seniors", which precipitated a lot of valuable interactive information, and then it was terminated for various reasons. Today, I started to talk with Tu Teng, and thought that the information was so quiet and wasteful. So try to use python to crawl the content of t ...

Posted by kylera on Sat, 07 Dec 2019 21:05:17 -0800