Testing domain name availability with pythonwhois

Question

I successfully use pythonwhois (installed with pip install ... ) to check for the availability of .com domains:

 import pythonwhois for domain in ['aaa.com', 'bbb.com', ...]: details = pythonwhois.get_whois(domain) if 'No match for' in str(details): # simple but it works! print domain 

But:

• it's a bit slow (average 2 requests per second)
• Will I be blacklisted by the whois server if I make 26 * 26 * 26 ~ 17000 requests?
  (I check availability of ???mail.com with ? a..z )

Question: is there a better way to check availability than to make one whois request per domain?

---

Edit: the task was completed in 9572 seconds, and here is the complete list of all available domains of the form ???mail.com , since November 2017, if anyone is interested in starting the email service!

Answer 1

You have to parallelize what you are doing. Since most of the time spent by your function is waiting for you, you can immediately check a lot of work (not limited to the number of your processors). Example:

 import pythonwhois from joblib import Parallel, delayed, cpu_count n_jobs = 100 # works in parallel def f(domain): details = pythonwhois.get_whois(domain) if 'No match for' in str(details): # simple but it works! print(domain) return domain else: return None domains= ['aaa.com', 'bbb.com', 'ccc.com', 'bbbaohecraoea.com'] result = Parallel(n_jobs=n_jobs, verbose=10)(delayed(f)(domain) for domain in domains) # create a list with the available domains available_domains=[domains[idx] for idx,r in enumerate(result) if r!=None] print(available_domains) # Result # ['bbbaohecraoea.com']