Labsco
github logo

incident-postmortem

✓ Official★ 36,202

by github · part of github/awesome-copilot

Use when an outage, production incident, or significant service degradation has occurred and the team needs to write a structured blameless post-mortem. Triggers on phrases like "write a post-mortem", "incident review", "what went wrong", "outage report", "root cause analysis", or "RCA". Covers timeline reconstruction, contributing factor analysis, impact quantification, and action item generation with owners.

🧩 One of 7 skills in the github/awesome-copilot package — works on its own, and pairs well with its siblings.

This is the playbook your agent receives when the skill activates — you don't need to read it to use the skill, but it's here to audit before installing.

Incident Post-Mortem

Guide a team through writing a structured, blameless post-mortem after a production incident. The output is a document that builds shared understanding, identifies root causes without blame, and produces concrete action items to prevent recurrence.

Blameless Principle

Systems fail, not people. The goal is to understand HOW the incident happened — not WHO caused it. Avoid language like "X forgot to", "Y should have known". Use "the system did not", "the process lacked", "the alert did not fire".

When to Use

  • Production outage or service degradation has been resolved
  • A significant near-miss occurred (would have been an incident if caught later)
  • User-facing errors, data loss, or SLA breach happened
  • Team wants to capture learnings before context fades

Not for: Minor bugs caught in staging, planned maintenance windows, or incidents with no learning value.

Process

Step 1 — Gather Metadata

If the user has not provided full incident details, ask for them section by section. Don't proceed to writing until you have: title, times, severity, affected services, and at least a rough timeline.

Step 2 — Reconstruct Timeline

Work with the user to build a precise chronological timeline. For each event:

  • Exact time (UTC preferred)
  • What happened (system event or human action)
  • Who observed it or took the action
  • Link to log / alert / Slack message if available

Flag gaps: "We don't know what happened between 14:32 and 14:47 — worth checking logs."

Step 3 — Root Cause Analysis

Use the 5 Whys iteratively:

Why did users see 500 errors?
→ The API pods were crash-looping.

Why were they crash-looping?
→ Memory limit was exceeded.

Why was the limit exceeded?
→ A new query was loading full result sets into memory.

Why wasn't this caught before deploy?
→ Load tests only covered the p50 case, not high-cardinality accounts.

Why did load tests only cover p50?
→ We had no test fixtures for large accounts.

Stop when you reach a system/process gap you can fix. The last "why" should point to an action item.

Distinguish:

  • Root cause — the deepest systemic gap (one or two)
  • Contributing factors — conditions that made it worse but aren't the root cause

Step 4 — Impact Quantification

Help the user be precise:

  • Duration: detection to resolution (not symptom start to resolution — separate these)
  • Error rate at peak vs. normal baseline
  • Percentage of traffic affected
  • Revenue / business impact if known

Step 5 — Action Items

For each root cause and contributing factor, generate at least one action item:

#ActionOwnerDue DatePriority
1Add load test fixtures for accounts > 10k records@eng-team2026-07-01High
2Lower memory alert threshold from 90% to 75%@platform2026-06-23High
3Add runbook for memory OOM pods@on-call-rotation2026-06-30Medium

Action items must have an owner (a person, not a team) and a due date. Vague actions like "improve monitoring" are not acceptable — break them into specific deliverables.

Step 6 — Write the Document

Produce the full post-mortem using the template below. Save to docs/postmortems/YYYY-MM-DD-<slug>.md.

Output Template

# Post-Mortem: [Incident Title]

**Date:** YYYY-MM-DD  
**Severity:** P[1-4]  
**Duration:** X hours Y minutes (HH:MM UTC – HH:MM UTC)  
**Incident Commander:** @name  
**Status:** Resolved

---

## Summary

[2–3 sentences. What happened, what was the user impact, how was it resolved. Written for someone who wasn't involved.]

## Impact

| Dimension | Value |
|-----------|-------|
| Affected services | [list] |
| User-facing impact | [errors / degraded / full outage] |
| Users affected | [estimated number or %] |
| Peak error rate | [X% vs Y% baseline] |
| Data loss | [none / describe scope] |
| SLA breach | [yes/no — by how much] |

## Timeline

All times UTC.

| Time | Event |
|------|-------|
| HH:MM | [First symptom / alert fired] |
| HH:MM | [On-call paged] |
| HH:MM | [Incident declared] |
| HH:MM | [Root cause identified] |
| HH:MM | [Mitigation applied] |
| HH:MM | [Full resolution confirmed] |
| HH:MM | [Customer communication sent] |

## Root Cause

[1–2 paragraphs. The deepest systemic gap that, if fixed, would have prevented the incident. Written in blameless language. Reference the 5 Whys chain if helpful.]

## What Went Well

- [Thing that worked — good alert, fast response, clear runbook]
- [Another positive]

## What Could Have Gone Better

- [Gap in process, tooling, or coverage — no blame language]
- [Another gap]

## Action Items

| # | Action | Owner | Due Date | Priority |
|---|--------|-------|----------|----------|
| 1 | [Specific deliverable] | @person | YYYY-MM-DD | High/Medium/Low |
| 2 | | | | |

## Lessons Learned

[Optional. 2–4 bullet points capturing non-obvious insights worth sharing with the broader team.]