DEV Community

Tanishq Agarwal
Tanishq Agarwal

Posted on Edited on

Cut my always-loaded Claude Code context by 57.5% (7,337 -> 3,121 tokens) and built a model-compatibility lint + skill factory (tq-forge, MIT)

After building dozens of custom skills and subagents in Claude Code, two problems kept biting me:

  1. Always-loaded skill definitions were burning 7,300+ tokens before I even typed a prompt.
  2. A skill that worked great on Opus would quietly degrade on Sonnet or Haiku—and pinned model IDs in templates kept going stale.

So I built tq-forge (MIT) — an open-source CLI and Claude Code plugin that scaffolds, learns, scores, compresses, and promotes skills and 5-file subagents.

What it actually does under the hood

  • Lossless Context Optimizer (/savings): Compresses always-loaded skill context without losing MCP server availability. On our test-suite benchmark fixture, it cut always-loaded tokens from 7,337 down to 3,121 (-57.5%), and is 100% reversible with restore.
  • 0–100 Structural Rubric + Model-Compatibility Lint: Runs 320 deterministic scorer assertions and checks model compatibility before promoting any artifact from sandbox to production (/tq-forge-test, /tq-forge-promote).
  • Forge from Intent or Auto-Learn (/tq-forge-skill, /tq-forge-agent, /tq-forge-learn): Generate a SKILL.md or 5-file subagent from a one-line prompt—or point /tq-forge-learn at a doc page, browser workflow, or technical thread to deduplicate links (linkdedupe) and distill repeatable steps into a scored skill.
  • Weekly Model Sync: Automated GitHub workflow that detects new Claude model IDs, updates templates, runs 165 unit tests, and opens a PR so agents don't drift onto retired models.

Install & Try

Happy to share the scoring rubric or optimizer diff if anyone wants to inspect how the token compression works!


If tq-forge saves tokens or catches model drift in your Claude Code setup, drop a star on GitHub (https://github.com/tanishq286/tq-forge) — it helps other Claude Code builders discover the project and directly supports our open-source cross-model eval roadmap!

Top comments (10)

Collapse
 
laogedev2 profile image
laogedev2 •

Great numbers — and one related gotcha worth adding to the lint: the description doesn't just decide whether a skill triggers, it also decides in what language the skill answers. If the output language is only stated in the skill body, Claude can still reply in English, because the routing decision happens on the description alone. We hit this with Chinese-output skills: putting e.g. 'code review in Chinese' in the description fixed it, body-only didn't. So when you compress descriptions for token savings, keep the language + trigger keywords in there. For a working example, the Chinese-language collection laolaoshiren/claude-code-skills-zh does exactly this — its zh-code-reviewer / zh-readme skills declare Chinese output in the description: github.com/laolaoshiren/claude-cod...

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

This is an incredible insight, thank you for sharing! You're completely right: because Claude Code's skill selector only passes the YAML frontmatter (name and description) during the initial system prompt phase, any language constraint buried in the markdown body is invisible until after the skill has already been invoked.

In tq-forge, our /savings optimizer strips non-essential prose but preserves primary trigger verbs and keywords. Following your feedback, I'm adding an explicit rule to our 0–100 quality scorer / model-compatibility lint that flags when a skill body defines target output languages (e.g., Chinese, Japanese, Spanish) that are missing from the frontmatter description.

I'd love your feedback on the rule formulation or PR once we push the lint assertion—feel free to check out the repo at github.com/tanishq286/tq-forge!

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

Quick update: v1.3.0 is now live on GitHub with the cross-model compatibility matrix (docs/MODEL_COMPATIBILITY.md) and 971 scorer assertions, and your frontmatter output-language lint rule is queued up next for v1.3.1: github.com/tanishq286/tq-forge

Collapse
 
halfcycle profile image
Halfcycle •

Nice result. One thing to watch when compressing: the description line is also what the agent reads to decide whether a skill applies, so a shorter one can quietly stop matching requests it used to catch. Does the rubric test that a compressed skill still triggers on the same prompts as before? We wrote up how loading and picking work in Claude Code and Codex, if useful: halfcycle.ai/writing/agent-skills

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

Great callout, and thank you for linking your write-up on agent skill picking!

In tq-forge, the 0–100 rubric doesn't just do static length checks; it validates semantic trigger coverage. When /savings compresses a skill or when /tq-forge-skill scaffolds one, the scorer checks:

  1. Frontmatter description contains explicit intent triggers (action verbs + target noun phrases).
  2. The description length stays within the optimal routing window (preventing over-truncation that causes under-triggering or over-verbosity that dilutes attention).
  3. In our upcoming v1.3.0 release, we're expanding this with an automated trigger regression test suite comparing prompt routing pass/fail rates across Opus, Sonnet, and Haiku.

We'd love your thoughts on our trigger assertion suite—feel free to explore the codebase at github.com/tanishq286/tq-forge or share any edge cases you've encountered!

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

Quick update: v1.3.0 just shipped on GitHub with the cross-model compatibility matrix (docs/MODEL_COMPATIBILITY.md) and 971 scorer assertions across Opus, Sonnet, Haiku, and Fable 5.2, and we're adding post-compression trigger retention checks next in v1.3.1: github.com/tanishq286/tq-forge

Collapse
 
halfcycle profile image
Halfcycle •

Trigger retention is the right next check. One suggestion for it: keep a few prompts the skill should NOT catch alongside the ones it should. Compression tends to make descriptions more generic, and the failure that shows up then is a skill firing on work that belongs to a neighbour.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Always-loaded context is paid on every call, so the split between start-up and on-demand loading is the number that matters. Does that start-up part load every session, or only when a skill fires?

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

The start-up context loads into Claude Code on every single interaction and turn in the session.

Claude Code injects all skill frontmatter (name and description) plus MCP tool definitions directly into the system prompt upon launch. If you have 15–20 detailed skills and several MCP servers installed, that overhead easily eats 7k–10k tokens before you even type your first prompt.

tq-forge's /savings optimizer tackles this exact split: it aggressively strips extraneous bloat from the always-loaded frontmatter down to lean, high-signal trigger lines (cutting context by 57.5% from 7,337 to 3,121 tokens on our benchmark fixture), while keeping the full skill instructions in the body to be loaded on-demand only when triggered. And it's completely lossless and reversible with restore.

You can check out the implementation and run it on your own setup: github.com/tanishq286/tq-forge. Let me know if you test it on your setup!

Collapse
 
tanishq286 profile image
Tanishq Agarwal •

Quick update: v1.3.0 is now live on GitHub with the full cross-model compatibility matrix (docs/MODEL_COMPATIBILITY.md) alongside the lossless /savings optimizer (-57.5%): github.com/tanishq286/tq-forge