Brainforge Knowledge

data-source-discovery-memo-update-playbook

8 min read

Playbook: Data source discovery memo (additive update to an existing memo)

Version: 0.3
Last updated: 2026-04-10
Audience: LLM agents and humans extending a published or baselined data source discovery memo when new tables or new profiling periods apply—without rewriting prior approved content.

Purpose

Keep a single canonical memo trustworthy over time: add new facts and sections in predictable places, refresh shared front-matter and executive-facing summaries, and leave frozen narrative (already reviewed text) unchanged in substance unless the engagement explicitly approves a rewrite.

When to use

Use this playbook when:

  • new tables appear in the same share/schema and must be documented alongside an existing memo
  • profiling was run on a new date and only new objects or new metrics need to be recorded
  • leadership asked for “what changed since last memo” while keeping the original body intact

Do not use this playbook when:

  • the old memo is wrong or misleading—open a correction or version 2 process with explicit stakeholder approval; do not hide substantive fixes inside “additive” edits
  • the output should be a short changelog only—produce a separate release note and link it from the memo header instead of duplicating everything

Service line / subservice

FieldValue
Service lineData Platform
Primary subserviceAnalytics and BI

Approval-before-execution pipeline

  • Additive edits to internal knowledge/clients/{client}/resources/ drafts: follow team norms.
  • Client-visible Google Doc / PDF: follow the engagement rule (often: do not edit the legacy vendor or client doc; add a new Brainforge-owned doc or a clearly dated addendum).
  • Substantive rewrites of paragraphs in frozen sections: require explicit approval—this playbook does not authorize silent rewrites.

Scope

In scope

  • Append new §3 subsections for net-new tables (3.12–3.25, next free numbers, …)
  • Add dated profiling labels (“queried YYYY-MM-DD”, “April 2026 audit snapshot”)
  • Update common sections that are designed to evolve: metadata block, §1 executive summary (additive bullets/paragraphs), §1 “What you’ll find” (new pointers)
  • Optional: add §4.x / §5.x subsections for new cross-table themes if the memo already uses that pattern

Out of scope

  • Rewriting historical narrative “for clarity” without a version decision
  • Deleting or softening metrics that were already published unless superseded with a dated note

Prerequisites

RequirementNotes
Baseline memo (link or path)Ask the user for the canonical existing memo: a link (Google Doc, Notion, Drive PDF, etc.) and/or a repo file path to markdown. Do not assume or guess which file is authoritative—confirm before editing.
Platform and auth (same as net-new playbook)Confirm Snowflake vs BigQuery (or other), discovery database/schema or project/dataset, and verified authentication before profiling new tables—see data-source-discovery-memo-new-share-playbook.md → Warehouse platform and authentication gate.
Diff in warehouseList of new tables or new columns worth documenting
Snapshot dateSingle query day for new profiling, recorded in §3 tables

Inputs

InputExampleRequired
Link or path to existing memoGoogle Doc: https://docs.google.com/document/d/<id>/edit; or repo path: knowledge/clients/{client}/resources/EMERSON_DATA_SOURCE_DISCOVERY_MEMO.mdYes
Editing surfaceIf the user only has a Doc link, decide whether work happens in Drive/Docs (export/import) or a markdown copy in repo—confirm with the userAs needed
Warehouse platform + scopeSame as memo baseline (confirm no migration to BQ/SF mid-flight)Yes
New table list14 net-new tables across 3 domainsYes
What stays frozen“December 2025 body through §3.11”Yes
Cross-linksVault schema audit pathAs available

Warehouse platform and authentication gate (before new profiling)

Treat this as Step 2 of the workflow (after Step 1 — baseline memo): same rules as the net-new discovery playbook—ask where data lives (Snowflake vs BigQuery), confirm auth method (Snow CLI, service account key, gcloud ADC, etc.), verify login and list/read access to the exact database/schema or project/dataset that contains the new tables. Do not append new §3 metrics until this passes. Full checklist and examples: data-source-discovery-memo-new-share-playbook.md → Warehouse platform and authentication gate.

Principles (non-negotiable)

  1. Do not edit frozen body text — Treat agreed ranges (for example “§1–§7 through §3.11”) as verbatim unless the ticket explicitly says “revise section X.”
  2. Append, don’t renumber — New tables get the next available §3 numbers; do not shift old 3.1–3.11 headings.
  3. Label time — Make it obvious what is older baseline vs new profiling (dates in header and in new §3 metric tables).
  4. Executive summary is additive — Add bullets/paragraphs for new coverage; do not replace or paraphrase existing bullets to “sound fresher.”
  5. Common sections to update (typical):
    • Title block / metadata — dual dates or a one-line “December 2025 …; §3.xx profiled April 2026” clause.
    • §1 — What this is — optional short paragraph after existing prose describing the extension (same share/schema).
    • §1 — Key findings — append bullets with April 2026 snapshot metrics only for new domains.
    • §1 — What you’ll find — append bullets pointing to new §3 ranges.
    • §4 — only add new subsections or paragraphs clearly marked as updates if synthesis is required; avoid rewriting prior synthesis.

Workflow

Step 1 — Ask for the existing memo (link or path)

  • Before any profiling or patching, ask the user for the canonical baseline memo:
    • Preferred: paste a link (Google Doc, Notion, Confluence, Drive PDF, etc.), or
    • Repo workflow: path to the markdown file in knowledge/ or the client repo (for example knowledge/clients/{client}/resources/*.md).
  • If both a Doc and a .md exist, ask which is source of truth for this update (or whether to sync Doc → markdown first).
  • Do not proceed to warehouse validation or §3 edits until the baseline artifact is identified.

Step 2 — Warehouse platform and authentication gate

  • Complete Warehouse platform and authentication gate (see section above and the net-new playbook). Confirm Snowflake vs BigQuery (or other), correct database/schema or project/dataset for the new tables, and verified read access before profiling.

Step 3 — Inventory and diff

  • List new tables vs the last memo (compare to §3 TOC or schema audit).
  • Confirm grain and row-count magnitude in the warehouse for each new table (same platform and connection verified in Step 2).

Step 4 — Insert new §3 blocks

  • Place new subsections after the last existing §3 subsection in that memo’s structure (or after the agreed insertion point—for example after §3.11).
  • Use the same subsection pattern as the baseline memo: Summary metrics → Business description → Interpretation → Questions.
  • Add source domain headings if the memo uses them (# **Source: …**) so readers can scan.

Step 5 — Update metadata (header)

  • Extend the Date: line or add a line explaining two time horizons (baseline narrative date vs new profiling date).
  • Do not remove the original date context.

Step 6 — Patch executive summary only by addition

  • What this is: add one short paragraph (or bullets if the memo already uses them there) describing the extension—14 tables, domains named, pointer to §3.xx.
  • Key findings: append bullets; each bullet should cite scale (orders, rows) and domain (Sage, Geodis, Walmart extension) with snapshot date where relevant.
  • What you’ll find: append bullets listing §3.12–§3.19, etc., by theme.

Step 7 — Cross-sections (optional)

  • If new tables change cross-table story materialy, add ## 4.x (or ### Update (Month YYYY)) with synthesis that references only new tables—do not rewrite old §4 paragraphs.

Step 8 — Changelog footnote (recommended)

  • At the bottom of the metadata block or in an appendix line, add: “Update YYYY-MM-DD: Added §3.12–§3.25 (…); executive summary appended.”

Step 9 — Review with guardrails

  • Confirm no accidental edits to frozen ranges (use git diff; reject whitespace-only churn in old paragraphs).
  • Run the anti-fluff rules from data-source-discovery-memo-new-share-playbook.md on new text only.

Agent guardrails (update-specific)

Do

  • Ask for a link or path to the existing memo at the start of the engagement when following this playbook (or an equivalent Cursor skill)—do not assume the file from context alone.
  • Use “Append,” “Added,” “April 2026 snapshot” language so time boundaries are obvious.
  • Keep new bullets parallel in structure to existing bullets (similar length, same tone).
  • If a metric supersedes an old one, add a dated note rather than silently deleting the old figure (unless legal/comms approves removal).

Do not

  • “Refresh” or “modernize” legacy bullets in §1 while updating—those edits are out of scope for this playbook.
  • Merge old and new memos by regenerating the whole file from an LLM—always patch the known-good file.

Failure modes / gotchas

  • No link or path provided—agent patches the wrong file or an outdated fork; stop and ask the user for the canonical memo URL or repo path first.
  • Profiling without re-verifying access—roles or datasets changed since the baseline memo; run the auth gate for the current update.
  • Silent paraphrase of old executive summary bullets—stakeholders lose trust; treat as a rewrite, not an additive update.
  • Inconsistent dates—new §3 says April, header still says only December—fix header first.
  • Renumbering §3 to “keep numbers contiguous by domain”—breaks bookmarks and prior references; append instead.
  • Huge pasted query logs in the memo—put ad hoc SQL in an appendix or repo script, not in §1.

Example implementation

  • Additive extension: knowledge/clients/lmnt/resources/EMERSON_DATA_SOURCE_DISCOVERY_MEMO.md — baseline December sections; §3.12–§3.25 and executive-summary append-only updates for April 2026 profiling (see git history for the additive patch pattern).
  • knowledge/standards/02-writing/data-source-discovery-memo-new-share-playbook.md — full memo for net-new share or first-time structure.
  • knowledge/standards/02-writing/data-source-memo-executive-style-playbook.md — one-page variant.
  • knowledge/standards/02-writing/PLAYBOOK_INDEX.md

Graph View