OPEN
P1 Critical
INITIAL
Database Primary Node Unresponsive
Service Affected
DB-PROD-01
Opened By
J. Martinez (jmartinez)
Opened At
2026-02-24 14:31:00 UTC
Duration
Calculating…
Update Frequency
Every 1 Hour
Messages Sent
3
Acknowledgments
✓
J. Martinez
14:32 UTC · "DBA team responding"
Channels Used
📧 Email ✓
📱 SMS ✓
💬 Slack ✓
📞 Voice ▶
📨 Message Timeline
3 MESSAGES[DB-PROD-01] Database Primary — Failover in Progress
--- Thread History ---
14:31 UTC [INITIAL] J. Martinez — Database primary node db-prod-01 is unresponsive. MySQL connections timing out. DBA team engaged. NOC bridge active.
--- Current Update ---
DB failover to replica db-prod-02 is in progress. ETA for service restoration: 15 minutes. Applications will experience elevated latency during the switchover. DBA team monitoring replication lag.
Next update in 1 hour or sooner on status change.
DELIVERY LOG
sentemailNOC Team (8 recipients)
sentsmsDBA On-Call (3 recipients) via AWS SNS
sentslack#noc-incidents
[DB-PROD-01] War Room Bridge — Join Now
Bridge 1 is active for this incident.
Dial-in: +1 (888) 555-0100
PIN: 4488#
All involved teams please join immediately for coordination. DBA, NOC, and Application teams required.
DELIVERY LOG
sentemailNOC Team (8 recipients)
sentslack#noc-incidents · #dba-team
[DB-PROD-01] Database Primary Node Unresponsive — P1 Critical
Database primary node db-prod-01 (10.0.1.10) is not responding to connections. MySQL connection timeout after 30 seconds. All applications dependent on the primary database are affected.
Impact: All write operations failing. Read-only fallback to cache for some services. Estimated 100+ users affected.
DBA team has been paged. NOC monitoring.
Next update in 1 hour or on status change.
DELIVERY LOG
sentemailNOC Team (8 recipients)
sentsmsDBA On-Call (3 recipients)
sentslack#noc-incidents
failedvoiceRetry 1/3 in 60s