catalogue

1. Common errors in waiting and packaging of selenium

2. Call the contents in print and compare the data

3. Enterprise wechat group sending and webhook parameter configuration

4. Summary

        

1. Common errors in waiting and packaging of selenium

        When climbing some dynamic data encrypted by the interface, sometimes the demand is one or two time-consuming data. It seems a little clumsy to try hard to decrypt and sort out. It is also a very practical means to browse without a head and take out the data. Here are some problems you have experienced and share with you.

        Explicit waiting and invisible waiting are the most common waiting means. The implicit waiting page is fully loaded. The explicit check whether the element loading is completed. The explicit writing format of personal habits:

        wait = WebDriverWait(web, 30, 2)
        wait.until(EC.presence_of_element_located((By.XPATH, '**')))
        wait.until(EC.presence_of_element_located((By.XPATH, '**')))
        text = web.find_element_by_xpath('**').text
        time = web.find_element_by_xpath('**').text

        You can use xpath or id to locate. You can detect it every 2s. If you don't throw an exception in 30s, you can get the desired text or other content.

        For some strange but recurring problems such as certificate problems and monitoring problems, you can try my parameter configuration.

    opt = Options()
    opt.add_argument("--headless")
    opt.add_argument("--disable-gpu")
    opt.add_argument('--ignore-certificate-errors')
    opt.add_argument('-ignore -ssl-errors')
    opt.add_experimental_option('excludeSwitches', ['enable - logging'])
    web = Chrome(options=opt, executable_path='**')  # Under the path is the driving position of chrome
    web.get('**')

        Most of the problems when packaging with pyinstaller are parameter configuration types, and   C:\Program 'is not an internal or external command   This problem can be written in the following form (those linked to Program Files (x86) must be quoted first). It is recommended to package with - D, which is easy to use. (see the end of the text for specific parameters)

os.system(r'C:/"Program Files (x86)"/Google/Chrome/Application/chromedriver.exe')

pyinstaller -D xxx.py

 

2. Call the contents in print and compare the data

        When we encounter small data errors in the loop, sometimes we need the content in print. We can add it to the file and then call the parameters. (that is, save the contents of print to a file)

 print(y, end='', file=f)

        When we encounter some data such as news with time and text, we can return two values. x = getContent() x[0] is used to output x[1], which is repeated with the previous data. In this way, we can avoid too many variables confusion when using concurrent programming for a large workload.

 

 

3. Enterprise wechat group sending and webhook parameter configuration

        If you want to obtain timely data such as news broadcast and weather forecast in real time every day in the mobile phone group, you can access it with the interface provided by the enterprise wechat, or use some automated libraries to realize mass distribution. Here is a brief talk about the practicability of the enterprise wechat internal group webhook.

        

        Copy webhook parameters after adding:

        

 

         Configure into pycharm

    url = '**'  # webhook address of robot
    headers = {'Content-type': 'application/json'}
    data = {
        "msgtype": "text",
        "text": {
            "content": x,  # x is the text to send
            "mentioned_list": ["@all"]  # Designable person
        }  # For more purposes, please refer to the official website of enterprise wechat

    }
    resp = requests.post(url, headers=headers, json=data) 
    print(datetime.now().strftime("%Y-%m-%d %H:%M:%S"),resp.text)
    resp.close()

        Not only text, but also pictures, graphics and text, specified tasks, sending time and frequency can be sent in by post request.

        If there is no interface in the external group, you can use other libraries to use algorithm operations (if you are interested, you can communicate with me privately)

4. Summary

        On the way, I really encountered some problems and thought of some small skills. Maybe unintentional small mistakes will make me think hard for several days. I hope I can give full play to my strength in your programming.

         Welcome everyone who read this article to put forward valuable opinions and exchange and study together.

Python (VII) concurrent programming - Practice 1 (process, thread, coprocess)

 Enterprise wechat robot - start
Introduction and summary of pyinstaller parameters