Brainforge Knowledge

tdd-data-template

3 min read

[Client] — Technical design document — [Project / domain]

About this document (Brainforge)

Internal conventions for how this file works in the repo. Strip or export without this section when sharing with a client.

Titling and filename

Use [Client] — Technical design document — [Project name] for the document title. Examples: LMNT — Technical design document — Omnichannel revenue migration · Acme — Technical design document — Wholesale parity check.

Filename: {client}-tdd-{project}.md under knowledge/clients/{client}/resources/.

When to use this template

Use this when a complex data project needs a formal technical design before execution — migrations, parity checks, multi-system integrations, or any project where scope creep or design errors would be costly. The TDD is the engineering contract everyone builds against.

Do not use this template when:

  • designing a single data model (use the Modeling Design Doc)
  • profiling a data source (use the Discovery Memo)
  • evaluating warehouse platforms (use the Warehouse Architecture Assessment)

Document metadata

Status: [Draft / In review / Approved / Locked] Warehouse: [platform] — Account/region: [details] Project: [project name] Prepared for: [Client stakeholder names and roles] Prepared by: Brainforge Last updated: [YYYY-MM-DD]

ArtifactLink / pathNotes
Discovery Memo[path to A1 memo]Source profiling reference
Modeling Design Doc[path to C1 doc]Data model design if applicable
Migration Plan[path to E1 or E2]Migration scope if this TDD supports a migration
Linear project[Linear URL]Project issue tracking

1. Project scope

1.1 Objectives

[2–4 sentences. What is this project building or changing? Why is it needed? What will be different when it is done?]

1.2 In scope

  • [Item]
  • [Item]

1.3 Out of scope

  • [Item] — [Why excluded]
  • [Item] — [Why excluded]

2. Architecture

2.1 High-level diagram

[Source system / table]
  → [Transformation / ETL step]
    → [Intermediate / staging layer]
      → [Final model / mart]
        → [Downstream consumer]

[2–4 sentences explaining the flow above.]

2.2 Components

ComponentTechnologyPurpose
[name][tool][what it does]
[name][tool][what it does]

3. Data flow

3.1 Source to target mapping

Source tableSource columnsTarget tableTransformationsGrain change
[schema.table][col1, col2, ...][schema.table][pass-through / aggregate / join / case][none / roll-up / fan-out]
[schema.table][...][schema.table][...][...]

3.2 Parity strategy

For migration or rebuild projects: how will we verify old and new produce the same results?

Verification checkMethodPass/fail criteria
[Row count parity][COUNT(*) on old vs new][within 1%]
[Key metric parity][SUM(revenue) by month old vs new][difference < $100]
[Grain uniqueness][duplicate key check][zero duplicates]

4. Execution plan

PhaseWhatTimelineDependenciesOwner
1[e.g., Build staging models][weeks][schema audit complete][name]
2[e.g., Build core marts, validate parity][weeks][Phase 1 complete][name]
3[e.g., Cut over downstream consumers][weeks][Phase 2 complete][name]
4[e.g., Decommission old objects][weeks][Phase 3 verified][name]

5. Rollback plan

TriggerRollback actionOwner
[e.g., Parity check fails > 5%][e.g., Revert to previous version, retain old tables][name]
[e.g., Consumer reports data quality issue][e.g., Redirect consumers back to old source][name]

6. Test plan

TestWhat it validatesHow to run
[Unit / QA test][Specific behavior or check][SQL query or test command]
[Integration test][End-to-end consumer readiness][Dashboard check, report verification]

7. Risks

RiskImpactProbabilityMitigation
[Risk][High / Med / Low][High / Med / Low][How addressed]

Appendix — Pre-handoff QA checklist

  • Scope boundaries are explicit (in scope + out of scope)
  • Source-to-target mapping covers every column with transformation noted
  • Parity strategy is defined (not deferred)
  • Execution plan has phases, timelines, owners, and dependencies
  • Rollback plan states triggers and actions
  • Test plan has specific, runnable tests
  • All decisions have rationale (not assumed)

Graph View