Ai Agents Test

You are an expert in multi-agent system testing and observability.

Mission

Test a multi-agent orchestration system by:

• Running a sample task through the agent network
• Showing real-time agent handoffs and routing
• Displaying performance metrics (time, handoff count)
• Validating agent coordination and output quality
• Identifying bottlenecks or issues

Usage

User invokes: /ai-agents-test "Task description"

Examples:

• /ai-agents-test "Build a REST API with authentication"
• /ai-agents-test "Research best practices for React performance"
• /ai-agents-test "Debug this authentication error"

Test Process

1. Validate Setup

First check if the multi-agent project exists:

# Check for required files
if [ -f "index.ts" ] && [ -d "agents" ]; then
  echo "✅ Multi-agent project found"
else
  echo "❌ Multi-agent project not found"
  echo "💡 Run /ai-agents-setup first to create the project"
  exit 1
fi

2. Parse Test Query

Extract the task from user input:

• If provided: Use their task
• If empty: Use default test task

Default tasks by category:

• Code generation: "Build a TODO API with CRUD operations"
• Research: "Research microservices best practices"
• Debug: "Why is my JWT authentication failing?"
• Review: "Review this code for security issues"

3. Start Test Execution

Create a test runner script:

test-runner.ts

import { runMultiAgentTask } from './index';

interface TestMetrics {
  startTime: number;
  endTime?: number;
  handoffs: Array<{
    from: string;
    to: string;
    reason: string;
    timestamp: number;
  }>;
  agentsInvolved: Set<string>;
  totalDuration?: number;
}

async function testMultiAgentSystem(task: string) {
  console.log('🚀 Multi-Agent System Test\n');
  console.log('━'.repeat(60));
…

4. Enhanced Orchestration with Metrics

Update index.ts to emit events for testing:

export async function runMultiAgentTask(task: string, options?: {
  onHandoff?: (event: HandoffEvent) => void;
  onComplete?: (result: any) => void;
  verbose?: boolean;
}) {
  const verbose = options?.verbose ?? true;

  if (verbose) {
    console.log(`\n🤖 Starting multi-agent task: ${task}\n`);
  }

  const handoffs: Array<{
    from: string;
    to: string;
    reason: string;
    timestamp: number;
  }> = [];
…

5. Execute Test

Run the test:

# Using ts-node
ts-node test-runner.ts "Build a REST API with authentication"

# Or using npm script
npm run test:agents "Build a REST API with authentication"

6. Display Real-Time Progress

Show live updates during execution:

🚀 Multi-Agent System Test

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📋 Task: Build a REST API with authentication
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🔄 Handoff: coordinator → researcher
   Reason: Need to research authentication best practices

🔄 Handoff: researcher → coder
   Reason: Research complete, ready to implement

🔄 Handoff: coder → reviewer
   Reason: Implementation complete, needs review

🔄 Handoff: reviewer → coordinator
   Reason: Review complete, all checks passed
…

7. Add Test Script to package.json

{
  "scripts": {
    "test:agents": "ts-node test-runner.ts"
  }
}

8. Create Pre-defined Test Scenarios

Create tests/scenarios.json:

{
  "scenarios": [
    {
      "name": "Code Generation",
      "task": "Build a REST API with authentication and CRUD operations",
      "expectedAgents": ["coordinator", "researcher", "coder", "reviewer"],
      "expectedHandoffs": 4,
      "maxDuration": 60000
    },
    {
      "name": "Research Task",
      "task": "Research best practices for microservices architecture",
      "expectedAgents": ["coordinator", "researcher"],
      "expectedHandoffs": 2,
      "maxDuration": 20000
    },
    {
      "name": "Debug Task",
…

9. Troubleshooting

If test fails, check:

# 1. Environment variables
if [ -z "$ANTHROPIC_API_KEY" ]; then
  echo "❌ Error: ANTHROPIC_API_KEY not set"
  echo "💡 Add your API key to .env file"
  exit 1
fi

# 2. Dependencies installed
if [ ! -d "node_modules/@ai-sdk-tools/agents" ]; then
  echo "❌ Error: Dependencies not installed"
  echo "💡 Run: npm install"
  exit 1
fi

# 3. Agents registered
if ! grep -q "researcher" index.ts; then
  echo "⚠️  Warning: Not all agents registered in index.ts"
fi