Engineering Knowledge Base
This section contains technical setup guides and engineering workflows for data engineering, infrastructure, and tooling.
Structure
setup/ - Technical Setup Guides
Step-by-step guides for installing and configuring development tools:
- 1Password CLI Setup - Secure credential management
- Snowflake CLI Setup - Snowflake command-line access
- Snowflake Key Pair Authentication - MFA bypass for automation
workflows/ - Engineering Workflows
Reusable workflows for common engineering tasks:
- dbt Dev Database Loop - Run dbt in a dev database, validate and show impact in a loop before PR (recommended for all dbt modeling)
- Snowflake Table Profiling - Profile tables and map to dbt marts
- dbt PR Data Diff - Prod vs staging impact profile after staging CI (for data PRs)
- Marts Documentation Creation - Generate marts documentation from Snowflake
- Metrics Dictionary Creation - Create metrics dictionary from dbt code
Purpose
These guides enable team members to:
- Set up development tools consistently
- Follow standardized workflows for data engineering tasks
- Reuse patterns across client engagements
- Document technical processes for knowledge sharing
Usage
- For Setup Guides: Follow step-by-step instructions to install and configure tools
- For Workflows: Use as templates for client-specific implementations
- For Troubleshooting: Reference escalation paths and common solutions
Related Documentation
- Writing Templates: See
02-writing/for document templates (PRDs, SOPs, SOWs) - Prompts: See
04-prompts/for AI prompts related to engineering tasks - Client Repos: Client-specific implementations and configurations
Contributing
When adding new guides:
- Follow existing structure and format
- Include prerequisites, step-by-step procedures, and troubleshooting
- Make guides reusable across clients (no client-specific details)
- Update this README when adding new guides