Grafana Alerts with Prometheus and Node Exporter Metrics - Introduction

Alerting is a critically important activity in the DevOps lifecycle. In this video I show you how to create a basic Grafana Alert using the legacy Grafana alerting system.

I am using node exporter on a cluster of Raspberry Pi's and Prometheus to scrape the metrics endpoints.

I walk you through the process of creating a Slack App that Grafana can use to send an alert message to a dedicated alerts channel.

After creating the Slack App, we create an Alert based off a Disk Usage metric (the percentage of disk space used on the Raspberry Pi).

Timestamps:
Introduction: 00:00
Slack App Creation: 00:40
Notification Channel Creation: 04:01
Alert Creation: 05:36
Alert Testing: 15:26
Grafana 8 Alerts and Conclusion: 16:50

[tech_with_moss]

Timestamps:
Introduction: 00:00
Slack App Creation: 00:40
Notification Channel Creation: 04:01
Alert Creation: 05:36
Alert Testing: 15:26
Grafana 8 Alerts and Conclusion: 16:50

[itall]

Thanks for this material! A video on Grafana 8 alerts would be great.

[elielberra]

Amazing video, your explanation was super clear!

[silverismoney]

This video was great, really simple and easy to setup.

[cosmetolog__d]

Thank you! I'm just implementing Prometheus + Grfana + Loki stack into our infrastructure, to get away from laggy Enterprise solution.

[kunlesolanke]

This is awesome.will look for how to format the % and see how it goes..thank you

[mrbr]

Thank you Moss for this help!!!! Awesome! You saved me!!!!

[harmeeshsingharora]

A video which I was looking for from a long time

[samsal]

Thank you for this video. How come only the TimeSeries graph has the alert tab and not others like guage or stat graph. How do you work around this?

[yotammedini]

It would be interesting to see (if possible) an example of Y(x) graph, which is _not_ time-based. With (x_i, y_i) sequence sent by Prometheus.

[shivamshekhar]

We can use query template function to solve the % issue.

[maximnaiman]

Hi Moss, thank for the video.
1 question, I'm trying to set an alert that will send me a message one a day with the alerting. is it possible?
I what it to monitor my dashboard for an entire day and send at the end of the day the results

[chrissmith]

Playback speed x2 is super useful here.

[moshikosiyahu]

Very helpful! Thanks!
One question, if I need to alert for multiple queries and every query has its own threshold (like the '40' you've configured) - what is the dynamic variable for it?

[aravindind]

Hi, nice content. I have a query, i am using Grafana 9. I am directly using alrert rules instead of via dashboard alert. I want all instances which is breaching the threshold as single alert message, like you showed in your video. But in my case i am getting separate alert for each individual instance. Any idea how to fix it

[piyushagarwal]

Thanks for the video, if if Rule has timing aspects why does the condition also has 5m to Now ? What role the plays or no role of that ?

[yashhirulkar]

thankyou sir ..! but i am getting 2:92: parse error: missing unit character in duration: message in alert in slack . what shud be the issue ?

[sateeshkumar]

Great video thank you
I want to include time stamp in the email of resolved alert. How can I do that. Please your response will be greatly appreciated

[parsapahlavan]

Thanks Moss, great video.
Is there a way to fully customize the message sent to the channel?
Not just the message section, but the whole template.
Is it possible on Slack or any other contact point?

[acbuddy]

can you share the queries you are using please. Thanks