How Promgen Routes Notifications

I’m Paul Traylor with LINE Fukuoka’s Development Department where I focus on tooling to monitoring the many servers used to support LINE Family Apps. One of my main tasks is to maintain Promgen which is a tool for managing Prometheus targets and routing alerts to the correct team.

How Prometheus and Alertmanager process alerts

Prometheus uses labels for many things, from writing PromQL to routing alerts, so understanding how labels are used is important to understanding how alerts are routed.