How to pass stdout / stderr from a child process using asyncio and get exit code after?

Question

In Python 3.4 on Windows, I need to pass the data written to stdout / stderr as a child process, that is, get its output as it occurs, using asyncio , introduced in Python 3.4. After that, I also need to determine the program exit code. How can i do this?

Answer 1

In the solution that I have so far used, SubprocessProtocol to get the output from the child process and the associated transport to get the process. "I do not know how optimal this is. I based my approach on the answer to a similar question from J. F. Sebastian .

import asyncio import contextlib import os import locale class SubprocessProtocol(asyncio.SubprocessProtocol): def pipe_data_received(self, fd, data): if fd == 1: name = 'stdout' elif fd == 2: name = 'stderr' text = data.decode(locale.getpreferredencoding(False)) print('Received from {}: {}'.format(name, text.strip())) def process_exited(self): loop.stop() if os.name == 'nt': # On Windows, the ProactorEventLoop is necessary to listen on pipes loop = asyncio.ProactorEventLoop() asyncio.set_event_loop(loop) else: loop = asyncio.get_event_loop() with contextlib.closing(loop): # This will only connect to the process transport = loop.run_until_complete(loop.subprocess_exec( SubprocessProtocol, 'python', '-c', 'print(\'Hello async world!\')'))[0] # Wait until process has finished loop.run_forever() print('Program exited with: {}'.format(transport.get_returncode())) 

Answer 2

Since the event loop can see and notify the end of the process before reading the rest of the data for stdout / stderr, we need to check the PIPE close events in addition to the process exit event.

This is the correction of aknuds1 answer:

 class SubprocessProtocol(asyncio.SubprocessProtocol): def __init__(self): self._exited = False self._closed_stdout = False self._closed_stderr = False @property def finished(self): return self._exited and self._closed_stdout and self._closed_stderr def signal_exit(self): if not self.finished: return loop.stop() def pipe_data_received(self, fd, data): if fd == 1: name = 'stdout' elif fd == 2: name = 'stderr' text = data.decode(locale.getpreferredencoding(False)) print('Received from {}: {}'.format(name, text.strip())) def pipe_connection_lost(self, fd, exc): if fd == 1: self._closed_stdout = True elif fd == 2: self._closed_stderr = True self.signal_exit() def process_exited(self): self._exited = True self.signal_exit() 

Answer 3

I suggest using a high-level api :

 proc = yield from asyncio.create_subprocess_exec( 'python', '-c', 'print(\'Hello async world!\')') stdout, stderr = yield from proc.communicate() retcode = proc.returncode 

You can also do more:

 yield from proc.stdin.write(b'data') yield from proc.stdin.drain() stdout = yield from proc.stdout.read() stderr = yield from proc.stderr.read() retcode = yield from proc.wait() 

etc.

But please keep in mind that waiting for, say, stdout when a child process prints nothing can not hang you on a coroutine.