Warning message not showing in slack using prometheus and alertmanager

Question

I am trying to get an alert alert found by Prometheus for standby notification using alertmanager .

This is alert.rules file and it works fine

groups: - name: Instances rules: # Alert for any instance that is unreachable for >5 minutes. - alert: InstanceDown expr: up == 0 for: 5m labels: severity: page # Prometheus templates apply here in the annotation and label fields of the alert. annotations: description: '{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes.' summary: 'Instance {{ $labels.instance }} down' 

Shows one instance successfully.

But what is the problem in my alertmanager.yml that it does not send notifications to failure. I also successfully installed the slack webhook and even checked if the hook was working fine by creating the hook using the service provided by slack

alertmanager.yml

 groups: - name: Instances rules: # Alert for any instance that is unreachable for >5 minutes. - alert: InstanceDown expr: up == 0 for: 5m labels: severity: page # Prometheus templates apply here in the annotation and label fields of the alert. annotations: description: '{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes.' summary: 'Instance {{ $labels.instance }} down' [tgurung@ip131 prometheus_graphana_myversion]$ cat alertmanager/alertmanager.yml route: receiver: 'slack-notifications' #group_by: [alertname, datacenter, app] receivers: - name: 'slack-notifications' slack_configs: - api_url: https://hooks.slack.com/services/T52GRFN3F/B93KTCUHH/JC channel: #general send_resolved: true # Alertmanager templates apply here. text: "<!channel> \nsummary: {{ .CommonAnnotations.summary }}\ndescription: {{ .CommonAnnotations.description }}" 

When starting docker-compose, I get the following

 prometheus_1 | level=error ts=2018-02-06T09:36:35.580565429Z caller=notifier.go:454 component=notifier alertmanager=http://xxxx:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://xXxx:9093/api/v1/alerts: dial tcp xxxx:9093: getsockopt: no route to host" 

Solving the error above: To solve the above routing problems, I launched the alertmanager in a completely new instance, and this error was overcome.

Going to API links in error message I see this

 {"status":"success","data":[]} 

And this is from alert_manager and works great.

 alertmanager_1 | level=info ts=2018-02-06T09:36:37.66654544Z caller=main.go:141 msg="Starting Alertmanager" version="(version=0.13.0, branch=HEAD, revision=fb713f6d8239b57c646cae30f78e8b4b8861a1aa)" alertmanager_1 | level=info ts=2018-02-06T09:36:37.66661402Z caller=main.go:142 build_context="(go=go1.9.2, user=root@d83981af1d3d, date=20180112-10:32:46)" alertmanager_1 | level=info ts=2018-02-06T09:36:37.668103448Z caller=main.go:279 msg="Loading configuration file" file=/alertmanager/alertmanager.yml alertmanager_1 | level=info ts=2018-02-06T09:36:37.673288146Z caller=main.go:354 msg=Listening address=:9093 

Here is the prometheus.yml configuration file

 global: scrape_interval: 5s external_labels: monitor: 'my-monitor' #alerting rules file rule_files: - '/alertmanager/alert.rules' scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['localhost:9090'] - job_name: 'node-exporter' static_configs: - targets: ['node-exporter:9100'] alerting: alertmanagers: - static_configs: - targets: ["54.36.XX:9093"] #this is the alertmanager service url 

Here is my docker-compose.yml

 version: '2' volumes: grafana_data: {} services: prometheus: image: prom/prometheus privileged: true volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml - ./alertmanager/alert.rules:/alertmanager/alert.rules - ./alertmanager/alertmanager.yml:/alertmanager/alertmanager.yml command: - '--config.file=/etc/prometheus/prometheus.yml' ports: - '9090:9090' links: - "alertmanager" node-exporter: image: prom/node-exporter ports: - '9100:9100' alertmanager: image: prom/alertmanager privileged: true volumes: - ./alertmanager/alertmanager.yml:/alertmanager/alertmanager.yml command: - '--config.file=/alertmanager/alertmanager.yml' ports: - '9093:9093' 

The alert status link does not indicate that the configuration is being transferred from the volume in the docker-composer. It shows the default configuration