Classical Crawler Learning-Traditional request Module Implements Three Different Ways to Simulate Landing on Renren Network (Elite)
Keywords:
Session
Mac
OS X
encoding
This case simulates the landing of Renren in three different ways. It is a classic case of traditional simulated landing. Readers are invited to consult it carefully.
- Setting session to simulate login
- Include cookies in headers for simulated Login
- Simulate login by specifying cookies separately
1. The method of setting session to simulate login (need to specify the password of login account)
import requests
session = requests.session()
post_url = "http://www.renren.com/PLogin.do"
post_data = {"email":"***", "password":"***"}
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36"
}
session.post(post_url,data=post_data,headers=headers)
r = session.get("http://www.renren.com/855311809/profile",headers=headers)
with open("renren1.html","w",encoding="utf-8") as f:
f.write(r.content.decode())
2. Include cookies in headers for simulated Login
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
"Cookie":"***"
}
r = requests.get("http://www.renren.com/855311809/profile",headers=headers)
with open("renren2.html","w",encoding="utf-8") as f:
f.write(r.content.decode())
3. Simulate login by specifying cookies separately
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
}
cookies="***"
cookies = {i.split("=")[0]:i.split("=")[1] for i in cookies.split("; ")}
print(cookies)
r = requests.get("http://www.renren.com/855311809/profile",headers=headers,cookies=cookies)
with open("renren3.html","w",encoding="utf-8") as f:
f.write(r.content.decode())
Posted by Telemachus on Wed, 03 Apr 2019 16:45:29 -0700