Boa constrictors crawl data through selenium, which is an effective way to break through the blockade.However, there are many problems in using selenium. In this paper, by asking and answering, we can get a general understanding of how to execute JavaScript programs through selenium, and then get the dynamically executed web pages.

Question:

Xiao Wang began to learn Python's crawler programming, as if the entire Internet data were almost in his pocket.Today, he is trying to complete another difficult action. He wants the following HTML to be captured in selenium and executed the JS script automatically, mimicking the mouse to automatically execute a click action.But to his disappointment, it was useless!

I don't know what to add to my learning
 python learning communication deduction qun, 784758214
 There are good learning video tutorials, development tools and e-books in the group.
Share with you the current talent needs of the python enterprise and how to learn Python from a zero-based perspective, and what to learn
1.  <div class = "vbseo_liked" > 
2.  <a href= "http://www.jamiiforums.com/member.php?u=8355" rel= "nofollow"> Nyaralego </a> 
3.  ， 
4.  <a href= "http://www.jamiiforums.com/member.php?u=8870" rel= "nofollow"> Sikonge </a> 
5.  ， 
6.  <a href= "http://www.jamiiforums.com/member.php?u=8979" rel= "nofollow"> Ab-Titchaz </a> 
7. and 
8. <a onclick= "return vbseoui.others_click(this)" href= "http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html#">11 others</a> 
9. Like this. 
10.  </ DIV> 

This is the code he executed.

1.  browser.execute_script("document.getElement(By.xpath(\"// div [@ class ='vbseo_liked'] / a [contains (@onclick,'return vbseoui.others_click (this)')] ".Click ()') 

It's useless and unresponsive.What was wrong?

Python's big answer:

Key Answers:

Find elements using selenium and pass them to execute_script() to click:

1.  link = browser.find_element_by_xpath('// div [@ class ="vbseo_liked"] / a [contains(@onclick，"return vbseoui.others_click(this)")]' ) 
2.  browser.execute_script('arguments [0] .click();' ，link) 

If you want to solve this problem from scratch, here's a list of things you need to know about it:

• How do I use simulated JavaScript clicks?

That's what I do It's simple, but it works:

1.  function  eventFire(el，etype){ 
2.  if(el.fireEvent){ 
3.  el.fireEvent('on'  + etype); 
4.  } else  { 
5.  var evObj = document.createEvent('Events' ); 
6.  evObj.initEvent(etype， true ， false ); 
7.  el.dispatchEvent(evObj); 
8.  } 
9.  } 

Usage:

1.  eventFire(document.getElementById('mytest1' )， 'click' ); 

• How to simulate clicks in Python? First, set a custom expectation and wait for the element to be "executed"?

1.  class wait_for_text_not_to_end_with(object):  
2.  def __init __(self，locator，text):  
3.  self.locator = locator 
4.  self.text =Written words 
5.  def __call __(self，driver):  
6.  Try: 
7.  element_text = EC._find_element(driver，self.locator).text.strip() 
8.  return not  element_text.endswith(self.text)  
9.  except StaleElementReferenceException:  
10.  Return False  

If you are still confused in the world of programming, you can join us in Python learning to deduct qun:784758214 and see how our forefathers learned.Exchange experience.From basic Python scripts to web development, crawlers, django, data mining, and so on, zero-based to project actual data are organized.For every Python buddy!Share some learning methods and small details that need attention, Click to join us python learner cluster

What?Once defined, how to call this class in a program? Look at the following code:

1.  Come from selenium import webdriver 
2.  from selenium.common.exceptions Import StaleElementReferenceException 
3.  Come from selenium.webdriver.common. Pass by Import 
4.  Come from selenium.webdriver.support.ui Import WebDriverWait 
5.  from selenium.webdriver.support Import expected_conditions As EC 
6.  class wait_for_text_not_to_end_with(object):  
7.  def __init __(self，locator，text):  
8.  self.locator = locator 
9.  self.text =Written words 
10.  def __call __(self，driver):  
11.  Try: 
12.  element_text = EC._find_element(driver，self.locator).text.strip() 
13.  return not  element_text.endswith(self.text)  
14.  except StaleElementReferenceException:  
15.  Return False  
16.  browser = webdriver.PhantomJS() 
17.  browser.maximize_window() 
18.  browser.get("http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html" ) 
19.  username = browser.find_element_by_id("navbar_username" ) 
20.  password  = browser.find_element_by_name("vb_login_password_hint" ) 
21.  username.send_keys("MarioP" ) 
22.  Password.send_keys(" codeswitching " ) 
23.  browser.find_element_by_class_name("loginbutton" ). click () 
24.  wait = WebDriverWait(Browser, 30) 
25.  wait.until(EC.visibility_of_element_located((. XPATH， '// h2 [contains(. ，"Redirecting")]' ))) 
26.  wait.until(EC.title_contains('Kenyan＆Tanzanian' )) 
27.  wait.until(EC.visibility_of_element_located((adopt.ID， 'postlist' ))) 
28.  ＃Click on the Other 11 link 
29.  link = browser.find_element_by_xpath('// div [@ class ="vbseo_liked"] / a [contains(@onclick，"return vbseoui.others_click(this)")]' ) 
30.  link.click() 
31.  browser.execute_script("" " 
32.  function  eventFire(el，etype){ 
33.  if(el.fireEvent){ 
34.  el.fireEvent('on'  + etype); 
35.  } else  { 
36.  var evObj = document.createEvent('Events' ); 
37.  evObj.initEvent(etype， true ， false ); 
38.  el.dispatchEvent(evObj); 
39.  } 
40.  } 
41.  eventFire(arguments [0]， "click" ); 
42.  ""，"Links" 
43.  ＃Waiting for a pair of "lattices" does not end the link with "11 people."      
44.  wait.until(wait_for_text_not_to_end_with((. CLASS_NAME， 'vbseo_liked' )， "Eleven other people do this.) 
45.  Print 'Success!!' 
46.  Browser.Close () 

See how it's so easy to crawl through selenium in a boa constrictor.Get the gist right and start compiling your own crawls.