How to detect exceptions in concurrent.futures in Python3?

Question

I just switched to python3 as a result of its parallel futures module. I was wondering if I can detect errors. I want to use parallel futures for a parallel program, if there are more efficient modules, please let me know.

I do not like multiprocessing, because it is too complicated and not much documentation. It would be great, however, if someone could write Hello World without classes only functions that use multiprocessing for parallel computation, so this is easy to understand.

Here is a simple script:

from concurrent.futures import ThreadPoolExecutor def pri(): print("Hello World!!!") def start(): try: while True: pri() except KeyBoardInterrupt: print("YOU PRESSED CTRL+C") with ThreadPoolExecutor(max_workers=3) as exe: exe.submit(start) 

The above code was just a demonstration of how CTRL + C would not work to print an application.

I want to be able to call a function, there is an error. This error detection must be from the function itself.

Another example

 import socket from concurrent.futures import ThreadPoolExecutor s = socket.socket(socket.AF_INET,socket.SOCK_STREAM) def con(): try: s.connect((x,y)) main() except: socket.gaierror err() def err(): time.sleep(1) con() def main(): s.send("[+] Hello") with ThreadPoolExecutor as exe: exe.submit(con) 

Answer 1

Here is the solution. I'm not sure you like it, but I can't come up with another. I modified your code to make it work.

 from concurrent.futures import ThreadPoolExecutor import time quit = False def pri(): print("Hello World!!!") def start(): while quit is not True: time.sleep(1) pri() try: pool = ThreadPoolExecutor(max_workers=3) pool.submit(start) while quit is not True: print("hei") time.sleep(1) except KeyboardInterrupt: quit = True 

Here are the points:

• When you use with ThreadPoolExecutor(max_workers=3) as exe , it waits until all tasks are completed. Look doc

  If wait is True, then this method will not be returned until all pending futures are executed and the resources associated with the executor are freed. If wait False , then this method will immediately return, and the resources associated with the executor will be freed when all pending futures are completed. Regardless of the value of the wait, the entire Python program will not exit until all pending futures are executed.

  You can not use this method explicitly if you use a with statement that disables Executor (waiting as if Executor.shutdown() were called with the wait set to True )

  It is like calling join() in a stream.
  This is why I replaced it with:

   pool = ThreadPoolExecutor(max_workers=3) pool.submit(start) 

• The main thread must do the “work” in order to be able to catch Ctrl + C. Thus, you cannot just leave the main thread there and exit, the easiest way is to start an endless loop

• Now that you have a loop running in the main thread, when you press CTRL+C , the program will go into the except KeyboardInterrupt block and set quit=True . Then your workflow may exit.

Strictly speaking, this is just a workaround. It seems to me that there can be no other way.

Edit
I'm not sure what bothers you, but you can catch the exception in another thread without problems:

 import socket import time from concurrent.futures import ThreadPoolExecutor s = socket.socket(socket.AF_INET,socket.SOCK_STREAM) def con(): try: raise socket.gaierror main() except socket.gaierror: print("gaierror occurred") err() def err(): print("err invoked") time.sleep(1) con() def main(): s.send("[+] Hello") with ThreadPoolExecutor(3) as exe: exe.submit(con) 

Output

 gaierror occurred err invoked gaierror occurred err invoked gaierror occurred err invoked gaierror occurred ...