perf: trim history by token budget, not message count

[aryansk]

Fixes #50

Problem

trim_history keeps at most MAX_HISTORY = 30 messages (plus system). Message count is a poor proxy for context: a 2000-line read_file result or huge grep output can fill the window in one turn, while long conversations of short messages get cut prematurely. Issue #50 proposes estimating token usage per message and trimming to a budget, and capping oversized tool results at the source.

Change

• Add gcode/agent.py:MAX_HISTORY_TOKENS = 12000 and _estimate_tokens(msg) (len//4 heuristic, tiktoken cl100k_base if available, includes tool_calls)
• Rewrite trim_history to keep history within both MAX_HISTORY (30) and MAX_HISTORY_TOKENS (12000): preserve system message, keep most recent within count, then drop oldest while over token budget (keep at least one recent turn, drop orphaned ToolMessages)
• Cap gcode/tools.py:grep output at 8000 chars / 200 lines (both external grep and pure-Python fallback) with truncation note, matching read_file line truncation, so a single tool call can't blow the window
• Keep unrelated cleanup out of this PR

Why this approach

Token budget (≈30 * 400 tokens) matches the existing message-count window but adapts to large tool outputs. Heuristic len//4 is the standard estimate when tiktoken is absent; tiktoken is used when installed for accuracy. Capping at the source prevents one huge grep from dominating the budget before trimming even runs.

Testing

command: python3 -m py_compile gcode/agent.py gcode/tools.py
result: ok

command: git diff --check
result: clean

command: grep -n "MAX_HISTORY_TOKENS\|_estimate_tokens" gcode/agent.py
result: constants and helper present, called in trim_history

command: manual
result: large grep output truncated at 200 lines; long history trimmed to budget while preserving system message and avoiding orphaned ToolMessages

Documentation and release impact

• User-facing behavior updated (history trimming)
• Changelog/release note needed: performance
• Migration or compatibility note needed
• No documentation impact

Review notes

• Known limitations: heuristic estimate; tiktoken improves accuracy when installed
• Follow-up issue, if any: none
• Security/licensing considerations: none