# Data Model & Migration Plan Produces the model the system should hold and the reversible route from the storage that exists today to that model. ## Deliverable One Markdown document, `data-model-and-migration.md`, in the structure set out under **Output** below. It carries two joined artifacts — the target model and the runbook that reaches it — because a model with no route to it cannot be scheduled. ## Required inputs - **The entities the system must hold** — what is stored and what each thing means to the people who use it. Sections 3 and 5 of `architecture-specification.md` are the expected form. - **The current storage reality** — the structures that exist now, read from the system or described by someone who runs it. If the current state is absent, stop and report it. A migration planned against an assumed starting point is a plan for a different system, and it stops at the first step that touches real data. ## Optional inputs - Direct access to the live structures, their keys, constraints and indexes - Data volumes and growth, as stated by the owner of the system - The retention, residency and deletion obligations attached to each entity - The access patterns the model must serve: what is read, what is written, what is read together - The release process the migration steps must fit inside - The window in which the system may be unavailable, if there is one, as stated by its owner An absent volume never becomes a duration. Where volumes were not supplied, each step states its cost in terms of what it touches — which structures, which rows, which readers — rather than how long it takes. ## Execution **1 — Record the current state exactly.** Every structure on the path being changed, with its fields, keys, constraints and indexes. Mark each one `observed` if it was read from the system, or `reported` if a person described it. A step planned against a reported structure carries that risk into production, and the step says so. **2 — Model the target.** Entities, attributes with their types, keys, relationships with their cardinality, the constraints that must hold, and the retention each entity carries. Every attribute traces to something the system must do; an attribute that traces to nothing is flagged rather than carried forward because it already exists. **3 — Index for the stated access patterns.** An index exists to serve a read someone described. Where no access patterns were supplied, record the keys and uniqueness constraints only and mark the index set `pending access patterns`. Never add an index for a query nobody named. **4 — Diff current against target.** Per difference: what changes, whether it is additive, destructive or a rename in disguise, and whether the data that exists can satisfy the new constraint. A constraint that current data would violate is a backfill with a decision behind it, not a schema change. **5 — Order the steps so each one is reversible.** Additive structures first, alongside the old. Then the backfill, then the switch of readers, then the switch of writers, then the removal of what is now unused — each as its own step. A step that cannot be reversed states that inside the step, names what makes it irreversible, and names who accepts it. **6 — Write the verification and the rollback for every step.** Per step: the check that proves it did what it claims, expressed as a comparison over the data rather than as a figure, and the exact action that undoes it. A step with no verification is not a step; it is a hope with a number next to it. **7 — State the cutover strategy.** Dual write, read shadowing, or a bounded window of unavailability — whichever the stated constraints allow — with what it requires and what it costs while it runs. Where the owner stated no window, the plan assumes none is available and says so in the open questions. ## Output `data-model-and-migration.md`, in this order: - **1. Input and date** — which entities and which current state this plan works from, who supplied them, and when - **2. Current state** — per structure: fields, keys, constraints, indexes, and `observed` or `reported` - **3. Target model** — per entity: attributes and types, keys, relationships, constraints, retention, and what it traces to - **4. Indexes** — per index: the entity, the fields, and the access pattern it serves - **5. Difference** — per change: additive, destructive or a rename, and whether existing data satisfies it - **6. Migration steps** — in order; per step: what it does, whether it is reversible, and who accepts it when it is not - **7. Backfill** — per backfill: what is filled, from which source, and what is done with the rows that source cannot fill - **8. Verification and rollback** — per step: the check that proves it, and the action that undoes it - **9. Cutover** — the strategy, what it requires, and the unavailability it needs if any - **10. Open questions** — the question, what it blocks, who can answer it ## Validation The plan is ready when all of these hold: - Every entity in the input appears in section 3, or is recorded there as deliberately outside the model - Every attribute in section 3 traces to something the system must do, or is flagged as tracing to nothing - Every step in section 6 has a verification and a rollback in section 8 - Every irreversible step declares its irreversibility inside the step and names who accepts it - Every index in section 4 names the access pattern it serves, or the whole set is marked `pending access patterns` - No volume, duration or row count appears that the owner of the system did not supply Fail the run if a step is irreversible without saying so, or if any step reaches section 6 with no verification. ## Failure handling - **No current state supplied** — stop. Deliver the target model alone if that was asked for, and report that no route to it can be planned from an unknown starting point. - **Current state reported but not observed** — produce the full plan, mark section 2 `reported`, and state in section 10 that every step must be re-checked against the real structures before it runs. - **No access patterns supplied** — produce sections 1 to 3 and 5 to 10, mark section 4 `pending access patterns`, and state that indexing is unresolved rather than settled. - **A step cannot be made reversible** — keep it, state what makes it irreversible, name the data that could not be recovered, and mark it as requiring a named acceptance before it runs. Do not bury it inside a larger step. - **Partial material** — plan the steps the material supports, mark the rest `INCOMPLETE — pending `, and deliver.