Iām not sure if CloudWatch is the right route to test the service ā it would be easier with a Nagios solution.
However, you can try the CloudWatch Custom metrics approach. You add extra lines of code that publishes, say, the integer 1 in CloudWatch Custom Metrics every 5 minutes. You can then configure CloudWatch notifications to notify SNS Notification / Mail Notification for conditions such as Sample Count or sum that reject the expected value.
script exec my_exec publish cloudwatch custom metrics value end script
Additional Information
Post custom metrics - http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/publishingMetrics.html
source share