Brainforge Knowledge

TESTING_GUIDE

9 min read

Testing Guide: Playbook Content Validation

Purpose: Test playbook content and AI generation without full infrastructure
Time Required: 5 minutes per scenario
Last Updated: 2026-01-26

Overview

Test your playbooks and see what AI generates before building the full system.

What You’ll Get:

  • Generated message sequences for any scenario
  • Quick validation of playbook quality
  • Ability to iterate on templates
  • No Slack/workflow infrastructure needed

Setup (One Time - 2 minutes)

  1. Install dependencies:
pip install anthropic python-dotenv
  1. Create .env file in gtm/agents/:
ANTHROPIC_API_KEY=your_api_key_here
  1. You’re ready to test!

Usage

Run the test script:

cd gtm/agents
python test_playbook.py

Or test specific scenario:

python test_playbook.py --scenario mutual-intro-exec

Option 2: Direct Claude Testing (Fastest - No Code)

Step 1: Copy This Prompt Template

I need you to generate a personalized message sequence following this playbook.
 
# PLAYBOOK CONTEXT
[Copy content from playbook markdown file here]
 
# SCENARIO
**Prospect Information**:
- Name: [Prospect Name]
- Title: [Title]
- Company: [Company Name]
- Email: [Email]
 
**Context**:
- [Specific context - e.g., "Introduced by John Smith (investor) who mentioned they're scaling their data team"]
 
**Research Signals**:
- [Signal 1 - e.g., "Raised $20M Series B 3 weeks ago"]
- [Signal 2 - e.g., "5 open data engineering roles on LinkedIn"]
- [Signal 3 - e.g., "Using Snowflake + dbt stack"]
 
# TASK
Generate a complete multi-step sequence for this scenario:
 
1. Classify the campaign (type, segment, persona)
2. Generate the sequence (all steps with full content)
3. Include personalization notes for each step
 
Format as:
- Campaign Classification
- Step 1 (timing): Subject + Body + CTA
- Step 2 (timing): Subject + Body + CTA
- etc.

Step 2: Test With Real Scenarios

Just paste the prompt above into Claude with:

  1. Your chosen playbook content
  2. A real or hypothetical scenario
  3. Research signals about the prospect

Step 3: Evaluate Output

Check:

  • Does it match playbook structure?
  • Is personalization specific (not generic)?
  • Are CTAs clear and appropriate?
  • Does timing make sense?
  • Would you actually send this?

Test Scenarios Library

Scenario 1: Warm Intro - Executive (Mutual Intro)

prospect:
  name: "Sarah Chen"
  title: "VP of Data & Analytics"
  company: "Fintech Startup Inc"
  email: "sarah@fintechstartup.com"
 
context:
  campaign_type: "mutual-intro"
  introducer: "Michael Zhang"
  introducer_role: "investor"
  intro_message: "Sarah is scaling their data team post-Series B and needs embedded help fast"
 
research_signals:
  - "Raised $25M Series B (2 weeks ago)"
  - "6 open data engineering roles (Director, 5 ICs)"
  - "Moving from MySQL to Snowflake"
  - "Building real-time fraud detection system"
  - "Team of 3 data engineers currently"
  - "LinkedIn posts about hiring challenges"
 
expected_output:
  segment: "warm-intro-executive"
  persona: "executive"
  steps: 3
  timing: "7 days total"

Scenario 2: Cold Outbound - Practitioner (Scale-up)

prospect:
  name: "Alex Rodriguez"
  title: "Director of Data Engineering"
  company: "SaaS Platform Co"
  email: "alex@saasplatform.com"
 
context:
  campaign_type: "cold-outbound"
  trigger: "Found via LinkedIn - posted about data infrastructure challenges"
 
research_signals:
  - "Company at 150 employees, growing fast"
  - "Posted on LinkedIn: 'Struggling with data pipeline reliability'"
  - "Using Airflow + Fivetran + Looker stack"
  - "Mentioned team is underwater with requests"
  - "Previous company: Netflix (strong technical background)"
  - "Follows topics: data engineering, analytics platforms"
 
expected_output:
  segment: "scale-up"
  persona: "practitioner"
  steps: 4
  timing: "14 days total"

Scenario 3: Event Follow-Up - VIP Dinner

prospect:
  name: "Jennifer Park"
  title: "Chief Analytics Officer"
  company: "Retail Corp"
  email: "jennifer@retailcorp.com"
 
context:
  campaign_type: "event-follow-up"
  event: "Data Leaders Summit 2026"
  interaction: "VIP Dinner conversation"
  topics_discussed:
    - "Omnichannel analytics challenges"
    - "Building vs buying analytics tools"
    - "Team structure for data organizations"
 
research_signals:
  - "Enterprise company, $500M revenue"
  - "Managing team of 25 data professionals"
  - "Recently consolidated from 8 BI tools to 2"
  - "Looking to implement customer 360 view"
  - "Mentioned partner with Snowflake"
 
expected_output:
  segment: "vip-dinner"
  persona: "executive"
  steps: 3
  timing: "5 days total"

Scenario 4: Partnership Outreach - Cold Rep

prospect:
  name: "Tom Williams"
  title: "Account Executive"
  company: "Snowflake"
  email: "tom@snowflake.com"
 
context:
  campaign_type: "partnership-outreach"
  partner: "Snowflake"
  goal: "Get Tom to introduce us to his accounts"
 
research_signals:
  - "Covers fintech vertical"
  - "LinkedIn shows 3 recent wins with mid-market fintech"
  - "Posted about helping customers with data transformation"
  - "Territory: Bay Area mid-market"
  - "Been at Snowflake 2 years"
 
expected_output:
  segment: "cold-partner-rep"
  persona: "practitioner"
  steps: 4
  timing: "21 days total"

Quick Test Script

Save this as test_playbook.py:

#!/usr/bin/env python3
"""
Quick playbook testing script - generates sequences without full infrastructure
"""
 
import os
from anthropic import Anthropic
from dotenv import load_dotenv
 
load_dotenv()
 
# Initialize Claude
client = Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY"))
 
def load_playbook(playbook_name):
    """Load playbook content from markdown file"""
    playbook_path = f"playbooks/{playbook_name}-playbook.md"
    try:
        with open(playbook_path, 'r') as f:
            return f.read()
    except FileNotFoundError:
        print(f"Error: Playbook not found at {playbook_path}")
        return None
 
def generate_sequence(playbook_content, scenario):
    """Generate sequence using Claude"""
    
    prompt = f"""I need you to generate a personalized message sequence following this playbook.
 
# PLAYBOOK CONTEXT
{playbook_content}
 
# SCENARIO
**Prospect Information**:
- Name: {scenario['prospect']['name']}
- Title: {scenario['prospect']['title']}
- Company: {scenario['prospect']['company']}
- Email: {scenario['prospect']['email']}
 
**Context**:
{scenario['context_description']}
 
**Research Signals**:
{chr(10).join(f"- {signal}" for signal in scenario['research_signals'])}
 
# TASK
Generate a complete multi-step sequence for this scenario following the playbook exactly.
 
Provide:
1. **Campaign Classification**
   - Campaign Type
   - Segment
   - Persona
   - Sequence Length
   - Reasoning
 
2. **Complete Sequence** (all steps with full content)
 
For each step, include:
- Step number and timing
- Channel (email/linkedin/etc)
- Subject line (if email)
- Full message body (ready to send)
- Call-to-action
- Personalization notes
 
Make it production-ready - I should be able to copy-paste and send these messages.
"""
 
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4000,
        messages=[{
            "role": "user",
            "content": prompt
        }]
    )
    
    return message.content[0].text
 
def print_divider():
    print("\n" + "="*80 + "\n")
 
def test_scenario(playbook_name, scenario_name, scenario):
    """Test a specific scenario"""
    
    print(f"\n🧪 Testing: {scenario_name}")
    print(f"📘 Playbook: {playbook_name}")
    print_divider()
    
    # Load playbook
    playbook_content = load_playbook(playbook_name)
    if not playbook_content:
        return
    
    print("📊 Scenario Details:")
    print(f"  Prospect: {scenario['prospect']['name']} ({scenario['prospect']['title']})")
    print(f"  Company: {scenario['prospect']['company']}")
    print(f"  Context: {scenario['context_description'][:100]}...")
    print_divider()
    
    print("🤖 Generating sequence with Claude...")
    
    # Generate sequence
    sequence = generate_sequence(playbook_content, scenario)
    
    print_divider()
    print("📧 GENERATED SEQUENCE")
    print_divider()
    print(sequence)
    print_divider()
    
    # Ask for feedback
    print("\n✅ Review the sequence above:")
    print("  - Does it follow the playbook?")
    print("  - Is personalization specific?")
    print("  - Would you send this?")
    print("\n")
 
# Test Scenarios
SCENARIOS = {
    "mutual-intro-exec": {
        "playbook": "mutual-intro",
        "scenario": {
            "prospect": {
                "name": "Sarah Chen",
                "title": "VP of Data & Analytics",
                "company": "Fintech Startup Inc",
                "email": "sarah@fintechstartup.com"
            },
            "context_description": "Warm introduction from Michael Zhang (investor) who mentioned Sarah is scaling their data team post-Series B and needs embedded help fast.",
            "research_signals": [
                "Raised $25M Series B (2 weeks ago)",
                "6 open data engineering roles (Director, 5 ICs)",
                "Moving from MySQL to Snowflake",
                "Building real-time fraud detection system",
                "Team of 3 data engineers currently",
                "LinkedIn posts about hiring challenges"
            ]
        }
    },
    "cold-outbound-practitioner": {
        "playbook": "cold-outbound",
        "scenario": {
            "prospect": {
                "name": "Alex Rodriguez",
                "title": "Director of Data Engineering",
                "company": "SaaS Platform Co",
                "email": "alex@saasplatform.com"
            },
            "context_description": "Found via LinkedIn - posted about data infrastructure challenges. No prior relationship.",
            "research_signals": [
                "Company at 150 employees, growing fast",
                "Posted on LinkedIn: 'Struggling with data pipeline reliability'",
                "Using Airflow + Fivetran + Looker stack",
                "Mentioned team is underwater with requests",
                "Previous company: Netflix (strong technical background)",
                "Follows topics: data engineering, analytics platforms"
            ]
        }
    },
    "event-vip-dinner": {
        "playbook": "event-follow-up",
        "scenario": {
            "prospect": {
                "name": "Jennifer Park",
                "title": "Chief Analytics Officer",
                "company": "Retail Corp",
                "email": "jennifer@retailcorp.com"
            },
            "context_description": "Met at Data Leaders Summit 2026 VIP dinner. Discussed omnichannel analytics challenges, build vs buy decisions, and team structure.",
            "research_signals": [
                "Enterprise company, $500M revenue",
                "Managing team of 25 data professionals",
                "Recently consolidated from 8 BI tools to 2",
                "Looking to implement customer 360 view",
                "Mentioned partnership with Snowflake"
            ]
        }
    }
}
 
if __name__ == "__main__":
    import sys
    
    # Check for API key
    if not os.environ.get("ANTHROPIC_API_KEY"):
        print("❌ Error: ANTHROPIC_API_KEY not found in environment")
        print("Add it to .env file or export it:")
        print("  export ANTHROPIC_API_KEY='your_key_here'")
        sys.exit(1)
    
    # Parse arguments
    if len(sys.argv) > 1:
        scenario_name = sys.argv[1]
        if scenario_name not in SCENARIOS:
            print(f"❌ Unknown scenario: {scenario_name}")
            print(f"Available scenarios: {', '.join(SCENARIOS.keys())}")
            sys.exit(1)
        
        scenario_data = SCENARIOS[scenario_name]
        test_scenario(scenario_data["playbook"], scenario_name, scenario_data["scenario"])
    else:
        # Run all scenarios
        print("\n🚀 Testing All Scenarios\n")
        for scenario_name, scenario_data in SCENARIOS.items():
            test_scenario(scenario_data["playbook"], scenario_name, scenario_data["scenario"])
            input("\nPress Enter to continue to next scenario...")

Usage Examples

Test Single Scenario

# Test mutual intro
python test_playbook.py mutual-intro-exec
 
# Test cold outbound
python test_playbook.py cold-outbound-practitioner
 
# Test event follow-up
python test_playbook.py event-vip-dinner

Test All Scenarios

python test_playbook.py

Test Custom Scenario

Edit the SCENARIOS dict in the script or create a new scenario YAML file.

Evaluation Checklist

For each generated sequence, check:

✅ Playbook Adherence

  • Matches expected segment
  • Uses correct persona tone
  • Follows sequence structure from playbook
  • Includes all required steps

✅ Personalization Quality

  • Uses specific research signals (not generic)
  • References actual context (intro, event, etc.)
  • Mentions relevant details (funding, roles, tech stack)
  • Feels custom (not templated)

✅ Message Quality

  • Subject lines are compelling
  • Body is concise and clear
  • CTAs are specific and easy to act on
  • Timing makes sense
  • Professional but not stuffy

✅ Production Ready

  • You would actually send this
  • No placeholder text
  • No obvious errors
  • Appropriate length for persona

Iterating on Playbooks

If output isn’t good:

  1. Check playbook examples: Are they specific enough?
  2. Add more templates: Include more variations in playbook
  3. Improve scenarios: Add more research signals
  4. Refine prompts: Make instructions clearer
  5. Test again: Run same scenario after changes

Adding Your Own Scenarios

Create a new scenario:

"your-scenario-name": {
    "playbook": "campaign-type",
    "scenario": {
        "prospect": {
            "name": "...",
            "title": "...",
            "company": "...",
            "email": "..."
        },
        "context_description": "...",
        "research_signals": [
            "...",
            "..."
        ]
    }
}

Add to SCENARIOS dict in test_playbook.py.

Next Steps

Once you’re happy with the generated sequences:

  1. ✅ Validate playbook quality - Run multiple scenarios
  2. ✅ Iterate on templates - Improve based on output
  3. ✅ Test with real prospects - Use actual data
  4. ✅ Collect feedback - Would you send these?
  5. → Then: Build full system with Slack integration

Tips

Get Better Results

  • More specific research signals = better personalization
  • Real prospect data = more realistic output
  • Multiple test runs = find edge cases
  • Compare to manual = see if AI matches your quality

Common Issues

Issue: Output is too generic
Fix: Add more specific examples to playbook, provide more research signals

Issue: Wrong tone/style
Fix: Update persona descriptions in playbook, add more example language

Issue: Sequences too long/short
Fix: Adjust sequence length guidance in playbook, specify in scenario

Issue: Not following playbook structure
Fix: Make playbook structure more explicit, add section headers clearly

Remember: The goal is to validate content quality before building infrastructure. Test, iterate, and refine until you’re confident the AI generates sequences you’d actually send.

Graph View