Skip to content

edimuj/tokenlean

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

177 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Tokenlean mascot - a squirrel collecting tokens

tokenlean

57 CLI tools + MCP server that let AI agents understand codebases without burning tokens

npm version npm downloads license

WhyInstallQuick ReferenceAll ToolsAI IntegrationWorkflowsLanguages


Zero confignpm i -g tokenlean and you're done · Token-conscious — every tool outputs only what's needed · Fast — ripgrep-powered with disk caching · Multi-language — JS/TS, Python, Go, Rust, Ruby · MCP native — structured tool access, no shell overhead · Minimal deps — installs in seconds


The Problem

If you've ever checked your usage quota at 2 PM and decided whether to debug now or wait for the reset — this is for you. Enterprise teams with unlimited API keys don't feel the burn. Solo devs, small teams, anyone on a Pro subscription does. Every cat of a 2000-line file when you needed one function, every 300-line test run when only 3 lines failed, every full directory read to find a signature — that's your working day getting shorter.

tokenlean fixes this:

Instead of... Use Savings
Reading a 500-line file for signatures tl symbols ~90% fewer tokens
Reading all files to find exports tl exports ~95% fewer tokens
Guessing what a change might break tl impact Know for sure
Reading a file to extract one function tl snippet ~85% fewer tokens
Running npm test and reading all output tl run "npm test" Errors only
Scanning long logs for real failures tl tail app.log Errors/warns + dedupe

How much are you wasting?

Find out in one command, no install needed:

npx tokenlean audit --all --savings --claudecode
Summary (348 Claude Code sessions)
  Opportunities:
  Category                Count  Actual     Saveable   Suggestion
  ----------------------------------------------------------------------------
  read-large-file          75x    253.4k      202.7k   -> tl symbols + tl snippet
  build-test-output        34x     28.2k       18.3k   -> tl run
  grep-command            115x     59.6k       11.9k   -> Grep tool
  tail-command              7x      4.8k        3.3k   -> tl tail
  find-command             25x      8.6k        2.6k   -> Glob tool
  curl-command             13x      3.2k        2.3k   -> tl browse
  cat-large-file            1x      1.1k         902   -> tl symbols + tl snippet
  webfetch                  4x      1.2k         823   -> tl browse
  head-command             11x      3.8k         759   -> Read tool (with limit)

  Still saveable:     243.6k of 363.9k (67%)

  Already saved by tokenlean:
  Tool              Count  Compressed   Raw estimate   Saved
  ------------------------------------------------------------------
  tl snippet          233x      215.7k            2.2M   1.9M
  tl symbols           93x       59.0k          295.0k   236.0k
  tl run               98x       28.7k           82.0k   53.3k

  Tokens saved:       2.2M (424 uses)
  Capture rate:       90% of potential savings realized

tokenlean demo — tl structure, tl symbols, and tl exports in action

Install

npm install -g tokenlean

Requires Node.js >= 18, ripgrep for search tools, and git for history tools.

Use tl as the global entry point — one command, many subcommands:

tl          # List all available commands
tl doctor   # Verify Node.js, ripgrep, git, hooks, and config
tl update   # Update the global tokenlean install
Development setup
git clone https://github.com/edimuj/tokenlean.git
cd tokenlean
npm link

Quick Reference

# What's in this file?           tl symbols src/auth.ts
# Functions only (dir/multi-file) tl symbols src/ --filter function
# Extract just one function      tl snippet handleSubmit
# What does this module export?  tl exports src/lib/
# How many tokens will this cost? tl context src/api/
# What's the project shape?      tl structure

# What depends on this file?     tl impact src/auth.ts
# How complex is this code?      tl complexity src/auth.ts
# Where are the tests?           tl related src/Button.tsx

# What changed recently?         tl diff --staged
# Is it safe to commit?          tl guard
# Find real usage examples       tl example useAuth
# Summarize noisy logs            tl tail logs/app.log
# What's the tech stack?         tl stack

Every tool supports -l N (limit lines), -t N (limit tokens), -j (JSON output), -q (quiet), and -h (help).

See all 57 tools for the complete reference.

AI Agent Integration

Add tokenlean instructions to your AI tool's config with a single command:

AI Tool Command
Claude Code tl prompt >> CLAUDE.md
Codex tl prompt --codex >> AGENTS.md
Cursor tl prompt --minimal >> .cursorrules
GitHub Copilot tl prompt >> .github/copilot-instructions.md
Windsurf tl prompt --minimal >> .windsurfrules

MCP Server — expose tokenlean as native MCP tools for structured, faster access with no CLI overhead:

# Add to any project's .mcp.json (Claude Code, Codex, etc.)
{
  "mcpServers": {
    "tokenlean": { "command": "tl mcp" }
  }
}

# Or pick just the tools you need
{ "command": "tl mcp", "args": ["--tools", "symbols,snippet,run"] }

8 core tools available via MCP: tl_symbols, tl_snippet, tl_run, tl_impact, tl_browse, tl_tail, tl_guard, tl_diff. Structured JSON in/out — no bash command construction, no argument parsing, no stdout scraping.

Hooks — automatically nudge agents toward token-efficient tool usage:

tl hook install claude-code    # Gentle reminders when agents waste tokens
tl audit --all --savings       # Measure actual savings across sessions

See measuring token savings for full audit and hook setup details.

Agent Skills

Ready-made workflows following the Agent Skills open format, organized for both Claude Code and Codex runtimes.

Claude Code skills

Skill What it does
code-review Review PRs efficiently — scope, blast radius, complexity, then targeted code reading
explore-codebase Understand an unfamiliar project in minutes without reading everything
safe-refactor Rename, move, or extract code with impact analysis and verification at each step
add-feature Add functionality by studying existing patterns first — locate, learn conventions, implement, verify
debug-bug Systematic bug investigation — reproduce, localize with blame/history, trace call path, verify fix
debug-performance Measure before optimizing — establish baselines, identify bottlenecks, confirm improvements with numbers
write-tests Write tests by studying existing patterns and code under test before writing assertions
upgrade-deps Upgrade dependencies safely — audit usage, research breaking changes, scale effort to version jump
migrate-framework Incremental framework/API migration with verification at each step, batched by dependency order

Codex skills

Skill What it does
code-review Risk-first code review workflow for Codex using git diff + targeted validation
explore-codebase Build a fast architecture map in Codex with targeted reads and dependency tracing
safe-refactor Refactor safely in Codex using blast-radius checks and incremental verification
add-feature Add features in Codex by mapping precedent first, then implementing minimal safe changes
debug-bug Repro-first bug fixing workflow in Codex with root-cause tracing and regression checks
debug-performance Performance debugging in Codex with baseline metrics and before/after proof
write-tests Write behavior-focused tests in Codex that match project conventions
upgrade-deps Dependency upgrade workflow in Codex with changelog-driven risk control
migrate-framework Incremental framework/API migrations in Codex with batch-level verification
# Claude Code — copy a skill
cp -r node_modules/tokenlean/skills/claude/code-review ~/.claude/skills/

# Claude Code — copy all skills
cp -r node_modules/tokenlean/skills/claude/* ~/.claude/skills/

# Codex — copy a skill
cp -r node_modules/tokenlean/skills/codex/code-review ~/.codex/skills/

# Codex — copy all skills
cp -r node_modules/tokenlean/skills/codex/* ~/.codex/skills/

# Or clone and pick what you need
git clone https://github.com/edimuj/tokenlean.git
cp -r tokenlean/skills/claude/code-review ~/.claude/skills/
cp -r tokenlean/skills/codex/code-review ~/.codex/skills/

Design Principles

  1. Single purpose — one tool, one job
  2. Minimal output — show what's needed, nothing more
  3. Composable — every tool supports -j for JSON piping
  4. Fast — no heavy parsing, no external services, aggressive caching
  5. Multi-language — JS/TS first, expanding to Python, Go, Rust, Ruby

More

Also by Exelerus

Project Description
claude-rig Run multiple Claude Code profiles side-by-side — isolate or share settings, plugins, MCPs per rig
agent-awareness Modular awareness plugins for AI coding agents
claude-mneme Persistent memory for Claude Code — context across sessions
claude-simple-status Minimal statusline — model, context usage, quota at a glance
vexscan-claude-code Security scanner for untrusted plugins, skills, MCPs, and hooks

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines on adding tools, code style, and submitting PRs.

License

MIT © Edin Mujkanovic

About

Lean CLI tools for AI agents and developers - reduce context, save tokens

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Contributors