Classical Crawler Learning-Traditional request Module Implements Three Different Ways to Simulate Landing on Renren Network (Elite)

Keywords: Session Mac OS X encoding

This case simulates the landing of Renren in three different ways. It is a classic case of traditional simulated landing. Readers are invited to consult it carefully.

  • Setting session to simulate login
  • Include cookies in headers for simulated Login
  • Simulate login by specifying cookies separately

1. The method of setting session to simulate login (need to specify the password of login account)

# coding=utf-8
import requests

session = requests.session()
post_url = "http://www.renren.com/PLogin.do"
post_data = {"email":"***", "password":"***"}
headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36"
}
#Send a post request using session, where cookie s are saved
session.post(post_url,data=post_data,headers=headers)

#Addresses accessible only after requesting login using session
r = session.get("http://www.renren.com/855311809/profile",headers=headers)

#Save page
with open("renren1.html","w",encoding="utf-8") as f:
    f.write(r.content.decode())

2. Include cookies in headers for simulated Login

# coding=utf-8
import requests
# There are injection problems
headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
    "Cookie":"***"
}

r = requests.get("http://www.renren.com/855311809/profile",headers=headers)

#Save page
with open("renren2.html","w",encoding="utf-8") as f:
    f.write(r.content.decode())

3. Simulate login by specifying cookies separately

# coding=utf-8
import requests

headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
}

cookies="***"
cookies = {i.split("=")[0]:i.split("=")[1] for i in cookies.split("; ")}
print(cookies)

r = requests.get("http://www.renren.com/855311809/profile",headers=headers,cookies=cookies)

#Save page
with open("renren3.html","w",encoding="utf-8") as f:
    f.write(r.content.decode())

Posted by Telemachus on Wed, 03 Apr 2019 16:45:29 -0700