Geek Answers Handbook

Post Hook for Elastic MapReduce

I wonder if there is an example of a mail process for EMR (Elastic MapReduce)? What I'm trying to achieve is to send an email to a group of people right after Amazon Hadoop has finished work.

+4
source share
2 answers

You want to configure the shutdown notification URL.

jobEnd.notificationUrl

AWS will fall into this URL, presumably with query variables that indicate which job has completed (job ID).

You can then indicate that this URL on your server processes your email notifications, assuming that you have already maintained a connection between email messages and job IDs.

https://issues.apache.org/jira/browse/HADOOP-1111

+5
source

An easier way is to use Amazon CloudWatch (a monitoring system) and Amazon Simple Notification Service (SNS) to track and notify you and others about the status of your EMR tasks.

For example, you can set an alarm for your cluster to check when it is IsIdle. It will be set to 1 after completing the task (or unsuccessful), and you will be able to receive the SNS notification as an email (or SMS). You can set similar alarms in the "Completed Tasks" account and other indicators.

For a complete list of indicators related to EMR, you can see the documentation on EMR

You can see more information about this here: http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/UsingEMR_ViewingMetrics.html

+3
source

More articles:


All Articles