To get notified when a systemd service fails, add an OnFailure= hook to the service that calls your Hook.Notifier URL. The moment the service enters the failed state, your phone gets a native push notification with its name.
systemd already knows precisely when a unit fails. The gap is that nothing tells you. One templated hook closes it, with no monitoring stack.
Create a reusable notifier unit
Make one template unit that any service can trigger. It takes the failed service name as an argument:
# /etc/systemd/system/[email protected]
[Unit]
Description=Push a notification when %i fails
[Service]
Type=oneshot
ExecStart=/usr/bin/curl -s "https://hooknotifier.com/{IDENTIFIER}/{KEY}?object=Service%20failed&body=%i%20entered%20the%20failed%20state&priority=high&color=%23EE6767&tags=alerts"
First get your URL
Your Hook.Notifier URL is https://hooknotifier.com/{IDENTIFIER}/{KEY}. Create a free account to get yours, then drop it into the notifier unit.
Point your services at it
Add one line to any service you care about. The %n passes its own name to the template:
# in your service unit, e.g. /etc/systemd/system/myapp.service
[Unit]
OnFailure=notify-failure@%n.service
Reload systemd to apply:
systemctl daemon-reload
The next time myapp fails, systemd fires the hook and your phone buzzes with "myapp.service entered the failed state". Add the same OnFailure= line to every unit that matters.
Test it
Force a failure to confirm the whole chain works:
systemctl start [email protected]
You should get a push within a second or two.
Why it matters
The worst outages are the quiet ones: a service dies, the restart limit is hit, and it stays down until a customer notices. systemd hands you the failure event for free. Turning it into a push that reaches your pocket means you are the one who notices first.
New to this? Start with how to send yourself a native push notification, or see how to get notified when a Docker container stops.