Preface
python is a simple and easy language to learn. Powerful third-party libraries enable us to multiply our programming efforts with half the effort. Today, let's talk about python multithreading. Penetration test In this paper, we will write a simple c-segment surviving host scanning script and a python version of the multi-threaded sword-directory scanning tool.

I. python multithreading
There are several ways to write python multithreading
1. thread module
python's multithreaded module, small scripts can be used, but there are flaws, such as instability, the number of threads is not controlled

There is a c-segment surviving host scanning script posted below. This script has been taught by Mr. iChunqiu ado.
Idea: Input an ip, get C segment by character splitting, ping-c 2 IP by multi-threading, judge whether the host survives or not according to the information returned
demo ping_thread.py:

'''
Created on 2017-2-27
 
@author: xiaoye
'''
#coding: utf-8
import thread
import time
from subprocess import Popen,PIPE
 
def scan_ip(ip):
    process = Popen('ping -c 2 ' + ip, stdin=PIPE, stdout=PIPE, shell=True)
    data = process.stdout.read()
    if 'ttl' in data:
        print '%s is live ,now time is %s' % (ip, time.strftime('%H:%M:%S'))
     
if __name__ == '__main__':
    #scan_ip('111.13.147.229')
    ips = raw_input()
    ip_header = '.'.join(ips.split('.')[:3])
    for i in range(1,255):
        ip = ip_header + '.' + str(i)
        #print ip
        thread.start_new_thread(scan_ip, (ip,))
        time.sleep(0.1)

Operation:
                                                                           
The speed is good, the stability is general.

The core of thread module is:

thread.start_new_thread(scan_ip, (ip,))
        time.sleep(0.1)

scan_ip is the function to execute, (ip,) is the parameter passed in, remember sleep

2. Use of threading module:
demo:

'''
Created on 2017-2-28
@author: xiaoye
'''
#coding: utf-8
import threading
import time
 
def test(th):
    print 'i am doing %s %s' % (th, time.strftime('%H:%M:%S'))
     
def main():
    thread = []
    keys = ['movie_th','swim_th','listen_th','learn_th','movie_th','swim_th','listen_th','learn_th','movie_th','swim_th','listen_th','learn_th','movie_th','swim_th','listen_th','learn_th']
    thread_count = len(keys)
    #print thread_count
    for i in range(thread_count):
        t = threading.Thread(target=test, args=(keys[i],))
        thread.append(t)
    for i in range(thread_count):
        thread[i].start()
    for i in range(thread_count):
        thread[i].join()
 
if __name__ == '__main__':
    main()

Operation:

As you can see, it basically runs at the same time. One use of threading.Thread module is as follows:

for i in range(thread_count):
        t = threading.Thread(target=test, args=(keys[i],))
        thread.append(t)
    for i in range(thread_count):
        thread[i].start()
    for i in range(thread_count):
        thread[i].join()

Mode 1.: A list holds all threads, start() executes the threads in the list, join() waits to run.

Mode 1? Is there mode 2?
Of course, mode 2 inherits a sub class from threading.Thread, overrides the parent run method, and implements multithreading to run run run run functions, which is also a very good way to write.
demo:

# -*- coding: utf-8 -*-
import threading
 
class T(threading.Thread):
 
    def __init__(self):
        threading.Thread.__init__(self)
 
    def run(self):                    #Inheritance, threading.Thread subclass, override run method, run method automatically executes after start()
        print 'i love you'
 
 
def main():
    thread = []
    for i in range(10):
      thread.append(T())
    for i in thread:
        i.start()
    for i in thread:
        i.join()
 
if __name__ == '__main__':
    main()

Operation:


2. Data daemon between threads
Queue absolutely protects between threads data security Good choice, about Queue, you can Baidu its own use, I send a point often used:

Queue.qsize() returns the size of the queue
Queue.empty() If the queue is empty, return True, and vice versa False
Queue.full() If the queue is full, return True, and vice versa False
Queue.full corresponds to maxsize size
Queue.get([block[, timeout]) gets the queue, timeout waiting time
Queue.get_nowait() is equivalent to Queue.get(False)
Non-blocking Queue.put(item) writes to the queue, timeout waiting time
Queue.put_nowait(item) is equivalent to Queue.put(item, False)
Queue.task_done() After completing a task, the Queue.task_done() function sends a signal to the queue whose task has been completed.
Queue.join() actually means waiting until the queue is empty before performing other operations


3. Multithread threading.Thread+Queue implementation Penetration Testing Tool To write
The script is released:
1. Multithread c-segment Survival Host Scan:

'''
Created on 2017-2-28
@author: xiaoye
'''
#coding: utf-8
import time
import sys
import threading
import Queue
from subprocess import Popen,PIPE
 
 
class Quethread(threading.Thread):
    def __init__(self, que):
        threading.Thread.__init__(self)
        self._que = que
     
    def run(self):
        while not self._que.empty():
            ip = self._que.get()
            process = Popen('ping -c 2 ' + ip, stdin=PIPE, stdout=PIPE, shell=True)
            data = process.stdout.read()
            if 'ttl' in data:
                sys.stdout.write('%s is live %s\n' % (ip, time.strftime('%H:%M:%S')))
     
     
def main():
    que = Queue.Queue()
    ips = raw_input()
    thread = []
    thread_count = 200
    ip_head = '.'.join(ips.split('.')[:3])
    #print ip_head
    for i in range(1, 255):
        que.put(ip_head + '.' + str(i))
    '''for i in range(1,255):
        print que.get()'''
     
    for i in range(thread_count):
        thread.append(Quethread(que))
         
    for i in thread:
        i.start()
         
    for i in thread:
        i.join()
         
     
if __name__ == '__main__':
    main()

Successful operation under ubuntu, need to modify the commands in Popen under win, screenshot:

It's fast and stable.
Section c host survival script: https://github.com/xiaoyecent/ping_threading_Queue

2. The py version of the multi-threaded sword - Directory scanning - supports custom dictionaries, output file locations and the number of custom threads:

'''
@author: xiaoye
'''
#coding: utf-8
import requests
import sys
import threading
#import time
import Queue
from optparse import OptionParser
 
reload(sys)
sys.setdefaultencoding('utf8')
 
class Doscan(threading.Thread):
    def __init__(self, que):
        threading.Thread.__init__(self)
        self._que = que
         
    def run(self):
        while not self._que.empty():
            d = self._que.get()
            try:
                r = requests.get(url + d, headers=headers, timeout=3)
                sys.stdout.write(d + ' is scan  status:' + str(r.status_code) + '\n')
                if r.status_code == 200:
                    with open(option.outfile, 'a') as f:
                        f.write(url + d + '\n')
            except:
                pass
     
def main():
    thread = []
    thread_count = option.threadcount
    que = Queue.Queue()
     
    with open(option.dictname, 'r') as f:
        for d in f.readlines():
            d = d.strip('\n')
            que.put(d)
     
    for i in range(thread_count):
        thread.append(Doscan(que))
     
    for i in thread:
        i.start()
     
    for i in thread:
        i.join()
         
if __name__ == '__main__':
    parse = OptionParser()
    parse.add_option('-u', '--url', dest='input_url', type='string', help='the url you wan to scan dir')
    parse.add_option('-o', '--out', dest='outfile', type='string', help='output filename', default='result.txt')
    parse.add_option('-s', '--speed', dest='threadcount', type='int', default=60, help='the thread_count')
    parse.add_option('-d', '--dict', dest='dictname', type='string', help='dict filename')
    (option, args) = parse.parse_args()
    headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0'}
    url = option.input_url
    main()
      

Usage: scan_dir.py [options]
 
Options:
  -h, --help            show this help message and exit
  -u INPUT_URL, --url=INPUT_URL
                        the url you wan to scan dir
  -o OUTFILE, --out=OUTFILE
                        output filename
  -s THREADCOUNT, --speed=THREADCOUNT
                        the thread_count
  -d DICTNAME, --dict=DICTNAME
                        dict filename

Paste out the parameter usage
Operation condition
For instance:
-u http://localhost -s 30 -d d://PHP.txt -o d://ichunqiu.txt:


Result:


Running speed depends on the number of threads (default 60) and the actual environment
Source code: https://github.com/xiaoyecent/scan_dir

Four, summary
Multithread plus queue is a good match for data protection between threads. The use of threading.Thread+Queue is expected to be mastered. In addition, inheriting threading.Thread writes subclasses, and rewriting the parent run method to achieve multithreading is also worth learning.