Recently, the company's servers have been attacked by a large number of requests, and the other side constantly requests to visit the web site. It takes up the broadband of the server, which makes it inaccessible to ordinary users.
Headache, so I studied for a day, wrote a script with python, extracted a large number of requested ip through nginx access log, and then added the blacklist to prohibit access.

The specific code is as follows:

There are still many optimization places in the code. After that, the basic functions have been realized.

# -*- coding: utf-8 -*-
import os
import signal
import subprocess
import time
from collections import Counter

logFile1 = "http-2018-08-21-access.log"
logFile2 = 'https-2018-08-21-access.log'
# Log file address

#Generally, log files are generated by day. Then, judge the generation date and current time of the files in the program, and replace the monitored log files
#The program is just a simple example. For example, monitor test1.log for 10 seconds, and turn to monitor test2.log
#In fact, multithreading operation is possible, but I haven't seen the content of multithreading recently, which will be added later
def monitorLog(logFile):
    print 'The log files monitored are%s' % logFile
    # Program runs for 10 seconds, monitoring another log
    #Define program exit time
    stoptime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time() + 20))
    #Execute linux command to get access log
    popen = subprocess.Popen('tail -f ' + logFile, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
    pid = popen.pid
    print('Popen.pid:' + str(pid))
    #Create a new empty list to store the ip data of visitors
    ips = []
    while True:
        line = popen.stdout.readline().strip()
        # Judge whether the content is empty
        if line:
            ip = line.split(' ')[0]
            # print(ip)
            # Add ip to the list
            ips.append(ip)
            # Count the number of ip in the list of 20 seconds, dictionary format
            conut = Counter(ips)
            # Take the first data in the list with the most ip times,
            print(conut.items()[0][0]+'----'+str(conut.items()[0][1]))
            # Judge if there are more than 50 requests, write ip to blacklist
            if conut.items()[0][1] >50:
                with open('/data/nginx/sites-enabled/blocklist.conf','a+') as f :
                    f.write('deny '+conut.items()[0][0]+';'+'\n')
                    os.system('docker exec -it nginx nginx -s reload')


        # current time
        thistime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time()))
        if thistime >= stoptime:
            # Terminate subprocess
            popen.kill()
            print 'kill subprocess'
            break
    time.sleep(1)
    monitorLog(logFile1)
    # monitorLog(logFile2)

if __name__ == '__main__':
    monitorLog(logFile1)