# Debugging & Root Cause Analysis

Works from the observed failure to the cause that produces it — and proves the cause by making the defect appear and disappear on demand.

## Deliverable

One Markdown document, `root-cause-report.md`, in the structure set out under **Output** below. Its fix feeds **Implementation Planning** when the change is large enough to need one.

## Required inputs

- **The defect report** — what was observed, where, when and by whom. An expectation and an observation, not only a sense that something is wrong.
- **Access to the evidence** — logs, the code, a reproduction, or the system in the state that shows the fault. At least one of them is required.

If either is absent, stop and report it. A cause proposed without evidence is a guess with a paragraph of reasoning attached to it.

## Optional inputs

- A reliable reproduction, and the environment it runs in
- The change history of the area, and what shipped before the fault appeared
- Monitoring output, traces and error rates around the event
- The configuration and the data the system ran against
- Reports of the same symptom from other times or places
- The test suite, and whether it passes at the failing revision

Absent optional inputs narrow what can be proved, not what may be claimed. Every conclusion states the evidence it rests on, and one that rests on none is written as a hypothesis.

## Execution

**1 — Separate the symptom from the story.** Record what was observed and what was expected, in the words of whoever saw it, with the time, the environment and the inputs in play. Strip out the explanation already attached to the report: an early theory narrows the search before it starts.

**2 — Get a reliable reproduction.** Make the defect appear on demand, and record the exact steps, inputs and state that do it. If it cannot be reproduced, say so and continue from the evidence alone — but mark every later conclusion unproven, because nothing can be tested.

**3 — Fix the boundary.** Establish what does work: the last known-good revision, the inputs that pass, the environment where the fault does not appear. The cause lies between what works and what does not, and that is the only region worth searching.

**4 — Narrow by halving.** Bisect the revision history, the input, the data or the call path — whichever dimension separates working from failing. Record each division and its result. Bisection converges; reading code in the hope of noticing something does not.

**5 — Separate the proximate cause from the condition that allowed it.** The line that failed is rarely the whole answer. Record both: what broke, and what let the broken state exist — the missing validation, the unhandled path, the assumption that was never checked.

**6 — Prove it.** Make the defect appear and disappear on demand by introducing and removing the cause. A cause that cannot be switched on and off is a hypothesis, and is written into the report as one, with the experiment that would settle it.

**7 — Look for the same cause elsewhere.** Search the codebase for the pattern that produced it. A cause found once and fixed once usually still exists in the places it was copied to.

## Output

`root-cause-report.md`, in this order:

- **1. Symptom** — what was observed, what was expected, where, when, and who reported it
- **2. Reproduction** — the exact steps and state that produce it, or a statement that it could not be reproduced
- **3. Evidence chain** — each division made, what it showed, and what it ruled out
- **4. Proximate cause** — the failure point, with file and line
- **5. Underlying condition** — what allowed the failing state to exist
- **6. Proof** — how the defect was switched on and off, or the experiment still needed
- **7. Fix** — the change that removes the cause, and what it does not address
- **8. The test that would have caught it** — what it asserts and where it belongs
- **9. Same cause elsewhere** — the other places the pattern appears, with their paths
- **10. Unresolved** — hypotheses not settled, and what would settle each

## Validation

The report is ready when all of these hold:

- Section 2 gives a reproduction or states plainly that there is none
- Every step in section 3 records what it ruled out, not only what it looked at
- Section 4 names a file and a line, or states that the failure point was not located
- Section 6 shows the defect appearing and disappearing, or section 10 carries the hypothesis instead
- Section 8 names a test that fails before the fix and passes after it
- No claim in the report rests on evidence that was not supplied

Fail the run if a cause is stated as proven without the demonstration in section 6, or if the fix addresses the symptom while the condition in section 5 stands.

## Failure handling

- **Cannot reproduce** — deliver the report with section 2 stating so, work from the logs and the code alone, and mark every conclusion a hypothesis with its settling experiment in section 10.
- **No logs and no access** — stop. Report that there is nothing to analyse, and name the evidence required.
- **Several plausible causes** — list them all in section 10 with the experiment that distinguishes them, and fix none until one is proven. Fixing two things at once destroys the evidence.
- **The defect stops occurring on its own** — record it as unresolved with the conditions under which it appeared, and add the monitoring that would catch its return. An unexplained fault that went away is a fault waiting for load.
- **The fix is out of reach** — report the cause, the containment available now, and what the real fix needs. Containment is recorded as containment, never as a resolution.