utun2/.opencode/context/openagents-repo/guides/testing-subagents.md

Testing Subagents - Step-by-Step Guide

Purpose: How to test subagents in standalone mode

Last Updated: 2026-01-09

---

⚠️ CRITICAL: Adding New Subagent to Framework

Before testing, you MUST update THREE locations in framework code:

1. evals/framework/src/sdk/run-sdk-tests.ts (~line 336)

Add to subagentParentMap:

'contextscout': 'openagent',  // Maps subagent → parent

2. evals/framework/src/sdk/run-sdk-tests.ts (~line 414)

Add to subagentPathMap:

'contextscout': 'ContextScout',  // Maps name → path

3. evals/framework/src/sdk/test-runner.ts (~line 238)

Add to agentMap:

'contextscout': 'ContextScout.md',  // Maps name → file

If missing from ANY map: Tests will fail with "No test files found" or "Unknown subagent"

---

Quick Start

# Test subagent directly (standalone mode)
cd evals/framework
npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml"

# Test via delegation (integration mode)
npm run eval:sdk -- --subagent=contextscout --delegate --pattern="01-test.yaml"

# Debug mode
npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml" --debug

---

Step 1: Verify Agent File

Check agent exists and has correct structure:

# Check agent file
cat .opencode/agent/ContextScout.md | head -20

# Verify frontmatter
grep -A 5 "^id:" .opencode/agent/ContextScout.md

Expected:

id: contextscout
name: ContextScout
category: subagents/core
type: subagent
mode: subagent  # ← Will be forced to 'primary' in standalone tests

---

Step 2: Verify Test Configuration

Check test config points to correct agent:

cat evals/agents/ContextScout/config/config.yaml

Expected:

agent: ContextScout  # ← Full path
model: anthropic/claude-sonnet-4-5
timeout: 60000

---

Step 3: Run Standalone Test

Use --subagent flag (not --agent):

cd evals/framework
npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/01-simple-discovery.yaml"

What to Look For:

⚡ Standalone Test Mode
   Subagent: contextscout
   Mode: Forced to 'primary' for direct testing
   
Testing agent: contextscout  # ← Should show subagent name

---

Step 4: Verify Agent Loaded Correctly

Check test results:

# View latest results
cat evals/results/latest.json | jq '.meta'

Expected:

{
  "agent": "ContextScout",  // ← Correct agent
  "model": "opencode/grok-code-fast",
  "timestamp": "2026-01-07T..."
}

Red Flags:

• "agent": "core/openagent" ← Wrong! OpenAgent is running instead
• "agent": "contextscout" ← Missing category prefix

---

Step 5: Check Tool Usage

Verify subagent used tools:

# Check tool calls in output
cat evals/results/latest.json | jq '.tests[0]' | grep -A 5 "Tool Calls"

Expected (for ContextScout):

Tool Calls: 1
Tools Used: glob

Tool Call Details:
  1. glob: {"pattern":"*.md","path":".opencode/context/core"}

Red Flags:

• Tool Calls: 0 ← Agent didn't use any tools
• Tools Used: task ← Parent agent delegating (wrong mode)

---

Step 6: Analyze Failures

If test fails, check violations:

cat evals/results/latest.json | jq '.tests[0].violations'

Common Issues:

Issue 1: No Tool Calls

{
  "type": "missing-required-tool",
  "message": "Required tool 'glob' was not used"
}

Cause: Agent prompt doesn't emphasize tool usage
Fix: Add critical rules section emphasizing tools (see examples/subagent-prompt-structure.md)

Issue 2: Wrong Agent Running

Agent: OpenAgent

Cause: Used --agent instead of --subagent
Fix: Use --subagent=ContextScout

Issue 3: Tool Permission Denied

{
  "type": "missing-approval",
  "message": "Execution tool 'bash' called without requesting approval"
}

Cause: Agent tried to use restricted tool
Fix: See errors/tool-permission-errors.md

---

Step 7: Validate Results

Check test passed:

# View summary
cat evals/results/latest.json | jq '.summary'

Expected:

{
  "total": 1,
  "passed": 1,  // ← Should be 1
  "failed": 0,
  "pass_rate": 1.0
}

---

Test File Organization

Best Practice: Organize by mode

evals/agents/ContextScout/tests/
├── standalone/           # Unit tests (--subagent flag)
│   ├── 01-simple-discovery.yaml
│   ├── 02-search-test.yaml
│   └── 03-extraction-test.yaml
└── delegation/           # Integration tests (--agent flag)
    ├── 01-openagent-delegates.yaml
    └── 02-context-loading.yaml

---

Writing Good Test Prompts

Be explicit about tool usage:

❌ Vague (may not work):

prompts:
  - text: |
            List all markdown files in .opencode/context/core/

✅ Explicit (works):

prompts:
  - text: |
      Use the glob tool to find all markdown files in .opencode/context/core/
      
      You MUST use the glob tool like this:
      glob(pattern="*.md", path=".opencode/context/core")
      
      Then list the files you found.      

---

Quick Troubleshooting

Symptom	Cause	Fix
OpenAgent runs instead	Used --agent flag	Use --subagent flag
Tool calls: 0	Prompt doesn't emphasize tools	Add critical rules section
Permission denied	Tool restricted in frontmatter	Check tools: and permissions:
Test timeout	Agent stuck/looping	Check prompt logic, add timeout

---

Related

• concepts/subagent-testing-modes.md - Understand standalone vs delegation
• lookup/subagent-test-commands.md - Quick command reference
• errors/tool-permission-errors.md - Common permission issues
• examples/subagent-prompt-structure.md - Optimized prompt structure

Reference: evals/framework/src/sdk/run-sdk-tests.ts