Streaming traffic: how to view application logs?

Question

I see all hadoop logs in my path /usr/local/hadoop/logs

but where can I see application level logs? eg:

mapper.py

 import logging def main(): logging.info("starting map task now") // -- do some task -- // print statement 

reducer.py

 import logging def main(): for line in sys.stdin: logging.info("received input to reducer - " + line) // -- do some task -- // print statement 

Where can I see logging.info or related log statements of my application?
I am using Python and using hadoop-streaming

thanks

Answer 1

In Hadoop streaming mode, STDIN / STDOUT is used to transfer key / value pairs between mappers and gearboxes, so log messages should be written to a specific log file - check the code sample and write documentation in python for more details. This query may also help.

Answer 2

Hadoop collects stderr, which can be viewed on the hasoop sitemap / reduce status. So you can just write stderr.