Runbooks¶
One page per alert family in
deploy/prometheus/rabbitkit-alerts.yaml.
Every alert's runbook_url annotation points here. Every threshold in the
rules is a reasoned default with a rationale annotation; see
Tuning the alerts to fit them to
your SLOs. The metrics are listed in the observability reference.
| Alert | Runbook |
|---|---|
RabbitMQBrokerUnavailable |
Broker unavailable |
PublisherConfirmFailures, PublisherUnknownOutcomes, PublisherConfirmLatencyHigh |
Publisher confirm failures and unknown outcomes |
PublisherUnroutableMessages |
Unroutable publishes |
RabbitMQConnectionFailures, RabbitMQConnectionTerminalFailure |
Connection storm / terminal connection failure |
ConsumerBacklogGrowing, ConsumerUnackedHigh |
Consumer backlog |
ConsumerProcessingFailures, ConsumerNotConsuming |
Consumer failures |
RetryStormDetected |
Retry storm |
DeadLetterQueueGrowing |
DLQ growth |
RabbitMQQuorumMajorityLost, RabbitMQQuorumUnhealthy |
Quorum unavailable |
RabbitMQConnectionBlocked, RabbitMQDiskAlarm, RabbitMQMemoryAlarm |
Flow control and resource alarms |