Brainforge Knowledge

07_qa_checklist

5 min read

Retail Dashboards — QA & Validation Checklist

Run these checks after each dashboard page is built. Cross-reference against legacy Retail Reporting Google Sheet, direct Snowflake queries, and Phil’s existing reports.

1. Data accuracy

1a. Spot-check daily POS revenue

Pick 3 specific dates across different months:

  • Query retail_rpt_daily_sales_summary in Snowflake directly for Walmart POS revenue on each date
  • Compare to the value shown on Dashboard 1, Page 1 (Executive Pulse) — latest-date table
  • Compare to the legacy Retail Reporting Google Sheet for the same date (if available)
  • Values should match to within $1 (rounding tolerance)

1b. Spot-check weekly totals

Pick 2 complete weeks:

  • Sum daily values Mon–Sun in Snowflake; confirm they match the weekly summary in retail_rpt_weekly_sales_summary
  • Confirm the week boundaries are Saturday–Friday (not Sunday–Saturday or Mon–Sun)
  • Compare to Dashboard 1, Page 1 weekly table

1c. Spot-check comparison values

For each comparison type, pick a known date and manually compute the expected value:

  • SDLW: Date X vs X-7 days — confirm same weekday, correct revenue
  • SDLM: Date X (e.g. 3rd Tuesday of March) vs 3rd Tuesday of February — confirm weekday match and month prior
  • SDLM edge case: Pick a date that is the 5th occurrence of a weekday in a month (e.g. 5th Friday of January); confirm it falls back to 1st occurrence in prior month
  • SDLY: Date X vs same Walmart week number + weekday in prior year — use WALMART_CALENDAR to verify
  • SWLY: Week ending X vs same wm_yr_wk_nbr - 100 — confirm correct week alignment

1d. Inventory snapshot

  • Query retail_fct_inventory for a specific Walmart store-SKU on the latest date; confirm on_hand_quantity, on_order_quantity, in_transit_quantity match Dashboard 2, Page 1
  • For a Target store-SKU, confirm on_order_quantity and in_transit_quantity show ”—” (not available)

1e. Store counts

  • Compare active_stores on Dashboard 1, Page 2 to a direct COUNT(DISTINCT store_id) from retail_fct_sales WHERE sale_quantity > 0 in the relevant period
  • Confirm churned_stores logic: stores present in week N-1 but absent in week N

2. Metric consistency across dashboards

  • POS Revenue on Dashboard 1 (Sales & Performance) for a given retailer/date matches POS Revenue on Dashboard 2 (Inventory & Operations) for the same retailer/date when both are filtered identically
  • total_units on Dashboard 1 matches pos_units on Dashboard 2 for overlapping grain

3. Omni Topics governance

  • Each metric in retail_sales_performance and retail_inventory_operations Topics has a description field filled in
  • No duplicate measure definitions — e.g. “POS Revenue” appears once per Topic, not multiple
  • Measures use consistent naming: total_sales_amount (not total_sales in one place and pos_revenue in another for the same calculation)
  • Default filters are set: 90 days for Sales Topic, 30 days for Inventory Topic
  • All % fields formatted as percentages (not raw decimals)
  • All k or $#,##0

4. Display and UX

  • As-of date visible on every dashboard page
  • Grain label visible (“Daily” or “Weekly Sat–Fri EST”) on Executive Pulse
  • Metric source indicator: users can see which retailers show POS vs Sales fallback
  • Conditional formatting: green for positive % changes, red for negative
  • Tables are sorted sensibly (dates DESC, revenue DESC, velocity DESC as appropriate)
  • No horizontal scroll on Daily tables (or if unavoidable, documented as expected)
  • Geography page annotated “Walmart only” until Target store geography is available
  • Dashboard footers contain: “Pilot scope. Deferred: trade spend, chargebacks, contribution margin, SPINS. Coming soon.”

5. AI exploration validation

Test Omni’s AI assistant (Blobby) with the pilot questions verbatim:

  • Q1: “Which Walmart stores have the highest velocity?” — Expect: list of stores ranked by avg_daily_units or sales_velocity_per_store
  • Q2: “Where do we see stockouts vs revenue at store level?” — Expect: stores with on_hand_quantity = 0 and recent POS revenue, or a reference to Dashboard 2 Page 2
  • Q3: “What is revenue by state for retail?” — Expect: state-level POS revenue from Walmart stores
  • Q4: “What were Walmart sales yesterday?” — Expect: POS revenue for latest date, Walmart only
  • Q5: “Show me POS by category for the last full week” — Expect: weekly POS by Drink Mix, Sparkling, etc.

For each: document whether the answer was correct, partially correct, or wrong. Note any cases where the AI returns a different metric definition than the governed one.

6. Stakeholder sign-off

  • Phil: Confirm Executive Pulse (Page 1) matches his glossary for SDLW/SDLM/SDLY/SWLY and display preferences ($k, full units)
  • Will/Russell: Confirm store health metrics (Page 2) and store-level detail (Dashboard 2, Page 3) match their operational needs
  • Jason/Brian: Confirm inventory pipeline (Dashboard 2, Page 1) and stockout logic (Page 2) are correct for supply chain decisions

7. Documentation updates

  • Update the Dashboards tab in Data Platform Documentation with:
    • Dashboard 1 name, URL, owner (Will Rossiter), pages, source tables
    • Dashboard 2 name, URL, owner (Will Rossiter), pages, source tables
  • Add new dbt models (retail_rpt_daily_comparisons, retail_rpt_weekly_comparisons, retail_rpt_sku_daily_comparisons) to dbt documentation
  • Document any known gaps in 01_topics_model_audit.md (Target geography, newly_churned definition, etc.)
  • SPINS/syndicated data gap: note that market share data is deferred; label “coming soon” in dashboard footer

Graph View