Ai Experiment Logger

Track, analyze, and optimize your AI experiments with a comprehensive logging system featuring both MCP tools and a beautiful web dashboard.

Overview

The AI Experiment Logger helps you systematically track experiments across different AI tools (ChatGPT, Claude, Gemini, etc.), analyze effectiveness patterns, and make data-driven decisions about which tools and prompting strategies work best for your use cases.

Key Features:

• 🎯 Structured Experiment Logging - Capture tool, prompt, result, rating, and tags
• 📊 Rich Statistics Dashboard - Visualize tool performance, rating distribution, and trends
• 🔍 Advanced Search & Filtering - Find experiments by any field or search term
• 📈 Performance Analytics - Track which AI tools perform best for your needs
• 💾 Data Persistence - Local JSON storage with CSV export capability
• 🌐 Dual Interface - MCP tools for Claude Code + web UI for visual exploration
• 🎨 Modern UI - Clean, responsive interface built with Tailwind CSS

What's Included

This plugin provides 7 MCP tools for experiment management:

Installation

1. Install Dependencies

cd plugins/mcp/ai-experiment-logger
pnpm install

2. Build the Plugin

pnpm build

3. Configure MCP Server

Add to your Claude Code MCP configuration file (~/.claude/mcp_config.json):

{
  "mcpServers": {
    "ai-experiment-logger": {
      "command": "node",
      "args": [
        "/absolute/path/to/plugins/mcp/ai-experiment-logger/dist/index.js"
      ]
    }
  }
}

Important: Replace /absolute/path/to with your actual installation path.

4. Restart Claude Code

Restart Claude Code to load the MCP server.

Usage

Using MCP Tools (in Claude Code)

Log an Experiment

Use the log_experiment tool to record:
- AI Tool: "ChatGPT o1-preview"
- Prompt: "Write a Python function to calculate Fibonacci numbers recursively"
- Result: "Provided clean recursive implementation with base cases. Included time complexity analysis (O(2^n))."
- Rating: 4
- Tags: ["code-generation", "python", "algorithms"]

List Recent Experiments

Use the list_experiments tool to show my last 10 experiments

Search Experiments

Use the list_experiments tool with searchQuery="code-generation" to find all coding experiments

Get Statistics

Use the get_statistics tool to show me which AI tools perform best

Export Data

Use the export_experiments tool to generate a CSV file of all my experiments

Using the Web Dashboard

Start the Web Server

cd plugins/mcp/ai-experiment-logger
pnpm web

The web UI will be available at http://localhost:3000

Web Dashboard Features

1. Dashboard Tab

• View all experiments in a sortable table
• Search across all fields in real-time
• Filter by AI tool, rating, or date range
• Delete experiments with confirmation

1. Statistics Tab

• Total experiments and average rating
• AI tool performance comparison
• Rating distribution visualization
• Top tags analysis
• Recent activity chart (30 days)

1. Log Experiment Button

• Easy form with all fields
• Star rating selector (1-5)
• Tag management (comma-separated)
• Date/time picker (defaults to now)

1. Export CSV Button

• Download all experiments as CSV
• Compatible with Excel, Google Sheets, etc.
• Includes all metadata

Data Storage

Experiments are stored locally in JSON format:

• Location: ~/.ai-experiment-logger/experiments.json
• Format: Structured JSON with full experiment history
• Backup: Recommended to periodically back up this file
• Privacy: All data stays on your local machine

Example Workflows

Workflow 1: Comparing AI Tools for Code Generation

# Log experiments with different tools
1. ChatGPT o1-preview - Rating: 5 - Tag: "code-generation"
2. Claude Sonnet 3.5 - Rating: 5 - Tag: "code-generation"
3. Gemini Pro - Rating: 3 - Tag: "code-generation"

# View statistics
Use get_statistics tool to see average ratings by tool

# Result: ChatGPT o1-preview and Claude tied at 5.0 avg rating