Skill Improver by athola

Automatically improves skills based on execution logs, user evaluations, and aggregated insights from LEARNINGS.md. Enhanced with Hyperagents (Zhang et al.,

1. patterns for data-driven improvement decisions.

Purpose

Part of Issue #69 Phase 5 - Self-Improvement Loop. This agent closes the observability loop by acting on insights gathered from:

• Phase 1: Execution logs (failure rates, duration)
• Phase 2: Qualitative evaluations (ratings, friction, suggestions)
• Phase 3: LEARNINGS.md aggregation (patterns, common issues)
• Phase 6: Hyperagents integration - PerformanceTracker trends, ImprovementMemory hypotheses, metacognitive self-modification

Inputs

• mode: all (default), skill: , top: , dry-run, or --metacognitive
• LEARNINGS.md path: ~/.claude/skills/LEARNINGS.md
• auto_implement: Boolean - automatically implement or prompt for confirmation

Workflow

0. Load Hyperagents data (before LEARNINGS.md)

Before loading LEARNINGS.md, consult the persistent improvement memory and performance tracker for context that should inform this improvement cycle.

from pathlib import Path

MEMORY_FILE = Path.home() / ".claude/skills/improvement_memory.json"
TRACKER_FILE = Path.home() / ".claude/skills/performance_history.json"

# Load improvement memory (if available)
improvement_context = {}
try:
    from abstract.improvement_memory import ImprovementMemory

    memory = ImprovementMemory(MEMORY_FILE)

    # Get strategies that worked and failed
    effective = memory.get_effective_strategies()
    failed = memory.get_failed_strategies()

    improvement_context = {
        "effective_strategies": effective,
…

Use this context to:

• Avoid strategies that previously failed (check failed_strategies)
• Prefer strategies that previously worked (check effective_strategies)
• Prioritize skills with degrading trends higher
• Skip skills that are already top performers

1. Load LEARNINGS.md

# Check if LEARNINGS exists
LEARNINGS_PATH=~/.claude/skills/LEARNINGS.md

if [ ! -f "$LEARNINGS_PATH" ]; then
  echo "LEARNINGS.md not found"
  echo "Run /abstract:aggregate-logs first to generate insights"
  exit 1
fi

# Read LEARNINGS
cat "$LEARNINGS_PATH"

2. Extract Improvement Opportunities

Parse LEARNINGS.md sections:

• High-Impact Issues: Failure rates, excessive failures, low ratings
• Slow Execution: Skills >10s average
• Low User Ratings: Skills <3.5/5.0
• Skill Performance Summary: Execution frequency data

For each issue, extract:

• Skill name
• Issue type (failure, slow, low_rating)
• Metrics (success rate, duration, rating)
• Recent errors (for failures)
• Friction points (for low ratings)
• Improvement suggestions (from evaluations)

3. Calculate Priority Scores

def calculate_priority(issue: dict, frequency_data: dict) -> float:
    """
    Priority = (Frequency × Impact) / Ease

    Where:
    - Frequency: execution count from summary table
    - Impact: severity of the issue (1-10 scale)
    - Ease: estimated effort to fix (1-10 scale)
    """
    frequency = frequency_data.get(issue["skill"], 1)

    # Calculate impact
    if issue["type"] == "high_failure_rate":
        # Failure rate impact: higher % = higher impact
        success_rate = float(issue["metric"].split("%")[0])
        impact = (100 - success_rate) / 10  # 0-10 scale

    elif issue["type"] == "low_rating":
…

4. Generate Improvement Proposals

For each opportunity (sorted by priority descending):

## Improvement Proposal #{N}: {skill}

**Issue**: {type} - {metric}
**Priority Score**: {score} ({HIGH|MEDIUM|LOW})
**Frequency**: {execution_count} executions in last 30 days

**Root Cause** (from LEARNINGS.md):
{errors or friction points}

**Proposed Changes**:
1. {specific change 1}
2. {specific change 2}
...

**Implementation Plan**:
- Files to modify: {list of files}
- Frontmatter version: {current} → {new}
- Est. effort: {easy|medium|hard}
…

5. Implement Improvements

For each approved proposal:

A. Read Current Skill

# Read skill file
skill_file="plugins/${plugin}/skills/${skill_name}/SKILL.md"
cat "$skill_file"

B. Apply Changes

Common improvement patterns:

1. Add Error Handling

<!-- Before -->
## Implementation
1. Parse PROOF.md file
2. Validate acceptance criteria

<!-- After -->
## Implementation
1. **Validate prerequisites**:
   - Check PROOF.md exists
   - If missing: Show creation guide
2. Parse PROOF.md file with error handling:
   - Catch JSONDecodeError → show format example
   - Catch KeyError → list required fields
3. Validate acceptance criteria