Python 3.4 multiprocessing

Question

This question requires advice as well as help with some code.

I'm currently learning Python with 3.4. I have created a basic network validation tool, I import elements from a text file, and for each of them I want python to check dns (using pydns), ip ping (using a subprocess to call its own ping device )

I am currently checking the IP address from 5,000 to 9,000 thousand and its number of hours, about 4, to return all the results.

I am wondering if I can use multiprocessor or streaming to speed this up, but still return the result to the list so that the string can be written to the csv file at the very end of the script in bulk.

I am new to python, so please tell me if I forgot something that I should have been talking about too.

Main code http://pastebin.com/ZS23XrdE

Class http://pastebin.com/kh65hYhG

Answer 1

Since most of the work seems IO based, you can easily rely on Threads.

Take a look at the Executor.map () function in cocurrent.futures: https://docs.python.org/3/library/concurrent.futures.html

You can pass a list of IP addresses and a function that you want to run for each element, and the return value is a list of the results of this function.

(check_dns_ip os_ping) ThreadPoolExecutor.map.

Answer 2

(ping ) , . HTTP-, . , concurrent.futures DNS-.

/ IP- 5000-9000 (DNS, ICMP). gevent, twisted, asyncio .