1. Token Reduction Mechanisms

2. The Agentic Loop Architecture

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Planner   │────▶│   Executor   │────▶│   Observer  │
│  (LLM)      │     │  (Tools/Env) │     │  (LLM/Logs) │
└─────────────┘     └──────────────┘     └──────┬──────┘
       ▲────────────────────────────────────────┘
                    (Feedback / Reflection)

• Deterministic shell vs. LLM shell: The agent decides whether to use grep (deterministic, no tokens) or ask the LLM to analyze (expensive, fuzzy).
• Error taxonomy: The observer classifies failures (syntax, test, logic, permission) and routes them—some go to the LLM for reflection, others trigger automatic retries.
• Context window management: The loop prunes history aggressively. Old file reads are summarized; only the current working set is kept in the LLM context.

3. Tasks Offloaded from the LLM

A. File System & Search

• Grep / Ripgrep: Searching for symbols, strings, patterns across the codebase.
• AST Parsing: Finding function definitions, imports, call graphs (tree-sitter, ctags).
• File I/O: Reading, writing, appending, deleting files.
• Directory Traversal: Listing structures, filtering by extension.

B. Execution & Validation

• Shell Command Execution: Running build scripts, package managers, dev servers.
• Test Runners: Executing pytest, jest, cargo test and capturing exit codes.
• Linters / Formatters: Running eslint, black, clippy and parsing structured output.
• Git Operations: git diff, git log, branching, committing, staging.

C. Environment & State

• Dependency Installation: npm install, pip install, resolving version conflicts.
• Process Management: Starting/stopping background services, port checking.
• Environment Variable Handling: Reading .env, injecting secrets.

D. Planning & Memory (Hybrid)

• Task Decomposition: Breaking "add OAuth" into: 1) find auth module, 2) add middleware, 3) write tests.
• Checkpointing: Saving working states before risky edits (enables rollback without LLM involvement).
• Long-term Memory: Storing user preferences (e.g., "I use tabs, not spaces") across sessions.

E. Communication & UI

• Progress Reporting: Streaming status updates to the user ("Running tests...").
• Permission Gates: Asking user approval before destructive ops (rm -rf, git push).
• Diff Rendering: Computing and displaying human-readable diffs.

4. Practical Impact: LLM vs. Agent Work Split

5. The "Great Agentic Loop" Thesis

1. Minimize LLM turns: Every tool call that succeeds reduces the need for another LLM inference.
2. Maximize signal-to-noise: Only exceptions and ambiguities reach the LLM; happy paths execute deterministically.
3. Stateful, not stateless: The agent maintains a working memory of file states, not just a chat log. This avoids the "re-read the file to confirm" token waste.

1. Token Reduction: Brutal Minimalism

2. Agentic Loop Engineering: Primitives, Not Opinions

The Loop Design

Deterministic Hooks

3. What Pi Offloads (and What It Refuses To)

What Pi DOES handle (the irreducible core)

• File I/O: read, write, edit
• Shell execution: bash (the universal escape hatch)
• LLM abstraction: 15+ providers, transport normalization, auth
• Rendering: A ~600-line TUI using direct DOM manipulation (no React re-layout lag)
• Cost tracking: Real-time token burn dashboards, provider-specific pricing

What Pi does NOT handle (intentional anti-features)

• Linting / type-checking: You run eslint or tsc via bash, not via a built-in tool
• Test runners: You invoke pytest yourself; the agent sees only the exit code + stderr
• Web search: Not built-in; add an extension or use curl
• Git operations: No built-in git tool; use bash or extensions
• Permission gates: No popups; add an extension like permission-gate.ts if you want them
• MCP: Not built-in; load via extensions if needed
• Context compaction: Not implemented; the minimal prompt is the compaction strategy

4. The Self-Modification Escape Valve

• Need a custom linter integration? Ask Pi to write an extension.
• Need audit logging? Ask Pi to write an extension that cryptographically signs session histories.
• Need visualization of the decision tree? Ask Pi to write a TUI component.

5. The Verdict: When Does It Win?

• You run local models (especially on Mac/MLX) and need every token to count
• You want to embed a coding agent as a subprocess in a larger automation system (RPC mode)
• You prefer to compose with Unix tools rather than learn a harness's opinions
• You want to inspect, audit, and reshape the agent loop in real time

• You want everything to work out of the box with zero configuration
• You need AST-aware patch verification (OpenCode's strength)
• You are on a Claude Max subscription and don't want to pay API rates on top (Anthropic blocks third-party harnesses from drawing on subscription credits)

1. No Hard Minimum, But a Practical Floor

2. Context Engineering: Where the Real Minimum Lives

• Tool result summarization: A 3,000-token file read is replaced with "✓ Read file (142 lines, 4,831 chars)" — under 100 tokens.
• Large output offloading: Outputs >8,000 chars are written to scratch files and referenced by path, replacing the full output with a 500-char preview.
• Staged compaction: When context hits ~95% usage, an emergency LLM-based summarizer prunes history.
• Lazy MCP discovery: External tool schemas are not loaded until searched for, reducing baseline overhead from ~40% to <5%.

3. What Users Report for Local/Small Models

"For coding agents 60k is my max. Increasing the context too much will only reduce accuracy and speed. System prompts are around 5k tokens."

"Local models are just slow enough that the subscription models are easier to use. For most of my tasks these days, the model's capability is sufficient; it is just not as snappy."

"I use it with Qwen 3.5 running locally when my daily limits run out... The harness is great."

4. The "Small Model" Fallback Trap

5. Bottom Line

• Disable MCP servers
• Disable subagents
• Strip down the system prompt (requires forking)
• Use the names-only tool detail level (minimal token mode)

1. What You Can Configure Now

Per-Agent maxTokens (Output Limit)

{
  "$schema": "https://opencode.ai/config.json",
  "agents": {
    "coder": {
      "model": "claude-4-sonnet",
      "maxTokens": 50000,
      "reasoningEffort": "medium"
    },
    "task": {
      "model": "gpt-4.1-mini",
      "maxTokens": 5000
    },
    "title": {
      "model": "gpt-4o-mini",
      "maxTokens": 80
    }
  }
}

Custom Provider limit.output

{
  "provider": {
    "myprovider": {
      "models": {
        "my-local-model": {
          "limit": {
            "context": 65536,
            "output": 32768
          }
        }
      }
    }
  }
}

Experimental Environment Variable

OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX=65536 opencode

Compaction Buffer

{
  "compaction": {
    "auto": true,
    "prune": false,
    "reserved": 10000
  }
}

2. What You Cannot Configure (Yet)

Per-Session Total Token Budget

Max Input Tokens Per Request

3. The 32K Trap for Local Models

1. Undershooting: If your model only supports 16K output, the API rejects the request.
2. Overshooting context: On a 128K model, max_tokens: 32000 means once your input hits ~96K tokens, the remaining window is exhausted and the conversation truncates.

4. Summary Table