Capital & Compute

Does Ponytail Actually Cut Your Claude Code Bill?

Ponytail cut Claude Code costs by 10% to 26% in two serious tests, not the half its code cut implies. One Cursor test found it cost more. What to expect.

· ai· coding-agents· economics· By Capital & Compute
Ponytail cut Claude Code cost 26% and 10% in two tests but cost 55% more on one model

Yes, but modestly: Ponytail cut Claude Code’s cost per task by 10.3% in an independent 80-task test and by 26% in its author’s own benchmark, far less than the code it removes. On some models it costs more.

Ponytail is the MIT-licensed skill that tells a coding agent to act like “the laziest senior dev in the room.” It had 177,158 GitHub stars on October 27, 2026, four and a half months after the repo was created, which makes it one of the most installed add-ons for Claude Code. Most of the people installing it want the same thing: a smaller bill. The code cut is real. The bill cut is smaller, and the reason is the same one that sank the last wave of token savers.

What is Ponytail?

Ponytail is a single prompt. The full version lives in skills/ponytail/SKILL.md, a compact version in AGENTS.md, and everything else in the repo exists to load that prompt into different agents: Claude Code, Codex, Cursor, Copilot, Gemini and, by the README’s count, 20 agents in total.

The prompt is a ladder. Before writing code, the agent stops at the first rung that holds:

  1. Does this need to exist at all?
  2. Is it already in the codebase?
  3. Does the standard library do it?
  4. Is there a native platform feature?
  5. Does an installed dependency do it?
  6. Can it be one line?
  7. Only then, write the minimum that works, plus one small test if it has logic.

The README’s own example is a date picker. Without the skill, the agent in its benchmark wrote 335 lines (a hand-built calendar). With it, the agent noticed the repo already had an Input component and that every browser ships a date picker, and wrote 10 lines. The README also states what the ladder never cuts: validation at trust boundaries, data-loss handling, security and accessibility.

The latest release is v5.3.2 (October 16, 2026). Version 5 added a rebuilt /ponytail-review and /ponytail-audit, and 5.3 added “the band”, a strip above the Claude Code prompt that shows what the current task cost.

How much does Ponytail save in each test?

Three published tests measured cost with and without the skill. They do not agree, and the disagreement is more useful than any one number.

Test Cost change Code change Model and setup Sample
Author’s benchmark, Ponytail 5 (Oct 2026) -26% per session -53% lines Opus 5.5, headless Claude Code, no shell 39 tasks x 5 runs (585 sessions)
JetBrains, Ponytail 4.8.4 (Jul 2026) -10.3% per task -15.4% lines Sonnet 5 medium, Claude Code 2.1.201 80 paired tasks
Community test, issue #121 (Jun 2026) +55% to -13% by model fewer lines on most models Cursor SDK, five models one task, about two runs per model

The independent number is the one to plan around. In Ponytail Skill for Claude Code: Does It Really Cut Agent Code by 54%? (JetBrains AI blog, Denis Shiryaev, July 28, 2026), JetBrains ran 80 paired tasks from the SkillsBench suite through Claude Code on Sonnet 5. A typical task cost 10.3% less with Ponytail (p=0.004, cheaper on 46 tasks and dearer on 34). That was part three of a series. Caveman, part one, measured 8.5% against an advertised 65%. rtk, part two, raised cost by 7.6%. Ponytail was the first tool in the series to save money with a statistically solid result.

JetBrains was careful about what that number means. The bootstrap interval on the median “just touches zero,” so in its words “‘Roughly 10% cheaper on this workload’ is defensible; ‘ponytail saves you 10%’ is not.” Its ten-task smoke run, before the full 80, had shown Ponytail 9.6% more expensive. Small samples of this tool swing both ways.

Ponytail's effect on cost depends on the test and the modelCost change with Ponytail installed, by test. The author's Ponytail 5 benchmark on Opus 5.5 found 26 percent cheaper. JetBrains on Sonnet 5 found 10.3 percent cheaper. A community Cursor test found GPT-5.5 High 13 percent cheaper, Opus 4.8 2 percent cheaper, Sonnet 4.6 5 percent dearer, Gemini 3.5 Flash 6 percent dearer and Composer 2.5 Fast 55 percent dearer.CheaperCosts moreAuthor, Opus 5.539 tasks x 5 runs, shell off-26%JetBrains, Sonnet 580 paired tasks, median-10.3%#121, GPT-5.5 HighCursor, one task, estimated-13%#121, Opus 4.8Cursor, one task, estimated-2%#121, Sonnet 4.6Cursor, one task, estimated+5%#121, Gemini 3.5 FlashCursor, one task, estimated+6%#121, Composer 2.5 FastCursor, one task, estimated+55%-60%-40%-20%0%+20%+40%+60%Cost change vs no skill. Mixed setups: see each note.
Ponytail's effect on cost depends on the test and the model
StudyMeasured effectMetric and sample
Author, Opus 5.5-26%39 tasks x 5 runs, shell off
JetBrains, Sonnet 5-10.3%80 paired tasks, median
#121, GPT-5.5 High-13%Cursor, one task, estimated
#121, Opus 4.8-2%Cursor, one task, estimated
#121, Sonnet 4.6+5%Cursor, one task, estimated
#121, Gemini 3.5 Flash+6%Cursor, one task, estimated
#121, Composer 2.5 Fast+55%Cursor, one task, estimated
Cost change with Ponytail against the same agent without it, by test and model. Rows differ in harness, model and sample size, so read the spread, not a ranking. The issue #121 rows are a single task with estimated cost.Source: Ponytail benchmark (2026-10-07), JetBrains AI blog (2026-07-28), GitHub issue #121 (2026-06-16); checked 27 October 2026

Why does half the code not mean half the bill?

Because output is the smaller part of what you pay for.

Ponytail works on what the model writes. Every Claude Code turn also re-sends the growing conversation, and that input (fresh, cached, and written to cache) is billed whether the agent writes 10 lines or 335. Ponytail’s own benchmark shows the gap: output tokens fell 45%, cost fell 26%. JetBrains saw the same split from the other side. The input side barely moved: re-reading its own history fell 8.4% and fresh input 3.9%, neither statistically significant. The saving came from “what the model writes, and on this benchmark that is the side the money moved on.”

This site’s earlier replay of 500 Claude Code sessions put numbers on that split for one real $926 bill: cache writes were 42% of it and model output 29%. Compression tools like rtk and headroom attack tool results, which mostly get re-read at the cheapest cache-read rate, so they barely dented it. Ponytail goes after the 29% output slice instead, plus the turns and cache writes that a shorter answer avoids. That is why it is the first add-on in this lane with a measurable saving, and why that saving is a tenth to a quarter of the bill rather than half.

It also explains JetBrains’ best and worst cases. On large builds, where the plain agent had room to over-build, code fell 31%. On tasks where the plain agent already wrote almost nothing, the typical task did not move. Ponytail can only cut code that would have been written.

Why is the author’s 26% higher than JetBrains’ 10%?

Three differences in setup, all disclosed in the author’s benchmark write-up (October 7, 2026), and none of them dishonest:

  • No shell. Bash is disabled for every arm. The agents write code and tests, and the harness runs them afterwards. Real Claude Code sessions run tests, grep and retry, and those turns re-send context that Ponytail does not shrink. The write-up says so: “This measures what the agent writes, not what it can debug.”
  • Clean rooms. Each session runs headless (claude -p) in a fresh folder outside any git repo, with CLAUDE.md and auto memory switched off. A real project’s context is larger, so the same output saving is a smaller share of each turn.
  • Tasks with room to over-build. Twelve of the 39 tasks are feature tickets in a real FastAPI and React repo, the kind of ticket where a plain agent reaches for a library. JetBrains’ SkillsBench tasks were mostly data, analysis and repair work, which it calls “a conservative test of the code claim.”

The author’s benchmark has also been corrected once in public. In June, issue #126 showed that the original baseline was a chatty model with no system prompt: one line asking for “just one example” cut the baseline from 108 lines to 16, against 8.25 for Ponytail. The author rebuilt the benchmark around real headless Claude Code sessions, and JetBrains credited the rebuild as “more methodological candour than most tools in this space manage.” The October numbers come from that rebuilt harness, on Opus 5.5, with a mean cost of $0.200 per session without the skill and $0.142 with Ponytail 5.

Read the 26% as a ceiling for clean, feature-shaped work. Read JetBrains’ 10% as a floor for mixed work.

Can Ponytail make Claude Code cost more?

Yes, on some models, and the reason is in the ladder itself.

Rungs two through five tell the agent to look before it writes: check the codebase, the standard library, the platform, the installed dependencies. Looking costs input tokens. In issue #121 (June 16, 2026), a developer ran Ponytail on and off through the Cursor SDK on one test-design task across five models. On most of them, Ponytail meant more tool calls and more tokens for fewer lines. Composer 2.5 Fast used about 308,000 tokens with it against 187,000 without, an estimated $1.07 against $0.69. GPT-5.5 High went the other way, 13% cheaper, and Opus 4.8 was roughly flat.

Treat that as a mechanism, not a measurement. It is one task, about ten finished runs in total, with cost estimated from list prices, and it predates Ponytail 5. But it names the trade honestly: the skill swaps output tokens for reading tokens. When reading is cheap relative to writing (cached input on a long session, an expensive output rate), you come out ahead. When the model reads a lot and writes little anyway, you can lose.

Is Ponytail better than caveman for saving tokens?

They work on different streams, so it is less a contest than a choice of target.

Caveman shortens the model’s prose: terse replies, fewer words around the code. Ponytail shortens the code and the work. In JetBrains’ series, caveman measured 8.5% against an advertised 65%, and its cost result was fragile (about 10% cheaper only after removing one outlier). Ponytail’s 10.3% held up on the full run. In Ponytail’s own October benchmark, Ponytail 5’s replies were also 30% shorter than the no-skill agent’s, which covers most of what caveman is for.

If you only install one, install Ponytail. If you already run caveman, the two do not conflict, but do not expect the savings to add.

Does writing less code cut corners on safety?

Not measurably, in either test, though neither proves it is safe.

In the author’s benchmark, the 18 tasks with hidden checks passed 87 of 90 runs with Ponytail 5 against 86 of 90 without, and all three arms passed all 30 runs of the six security tasks (SQL injection, path traversal, forged tokens, rate limiting, malformed CSV rows and caching). Ponytail also left a test behind where the logic needed one 98% of the time, against 68% for the plain agent. JetBrains found 65 tasks scored identically, 9 slightly worse and 6 slightly better: “a null result, not a clean bill of health,” since 80 pairs cannot rule out a small drop.

One small tell from the JetBrains run: the ruleset asks the agent to mark deliberate shortcuts with a comment, and across 80 trials that happened once. “The ladder gets followed; the paperwork does not.”

How do you install Ponytail and check the savings yourself?

  1. Install it as a plugin. In Claude Code, run these as two separate prompts:

    /plugin marketplace add DietrichGebert/ponytail
    /plugin install ponytail@ponytail

    The plugin’s SessionStart hook injects the ruleset every session. JetBrains copied the bare SKILL.md into a skills folder first, and Claude Code chose to use it zero times in ten sessions. A loose skill file will most likely do nothing.

  2. Install only from the real source. The README warns that the only genuine copies are DietrichGebert/ponytail on GitHub and @dietrichgebert/ponytail on npm, and that Ponytail never ships .exe or .dll files. A repo with 177,000 stars attracts lookalikes.

  3. Measure a week, not a task. Note your spend with Claude Code’s /cost command or your plan’s usage page for a normal week, then a week with Ponytail on. JetBrains’ smoke run shows how far ten tasks can mislead. The band (Ponytail 5.3, Claude Code 2.1.290 or later) shows cost per task above the prompt, and /ponytail band import loads past sessions into a local board for comparison. It does not appear in the VS Code chat panel, in cloud sessions or under claude -p.

  4. Turn it down where it hurts. /ponytail lite softens it, and /ponytail off disables it. If your model reads heavily and writes little, run a week at each level before deciding.

The bigger levers on a Claude Code bill are still model choice, effort level and session length (this site’s breakdown of AI coding agent token usage shows where the tokens go). Ponytail is the rare add-on that moves the bill at all. Expect about a tenth off, more if your agent likes to build things nobody asked for.

Frequently asked questions

Does Ponytail reduce Claude Code costs?
Modestly. An independent JetBrains test of 80 paired tasks on Sonnet 5 found Ponytail cut the typical task cost by 10.3%, and the author benchmark of Ponytail 5 on Opus 5.5 found 26% per session. Both are much smaller than the 45% to 53% cut in code and output tokens.
Why does Ponytail save less money than it saves code?
Ponytail shrinks what the model writes, but most of a Claude Code bill is input: the conversation re-sent each turn, cache reads and cache writes. In the author benchmark, output tokens fell 45% while cost fell 26%.
Can Ponytail increase token usage?
Yes. Its ladder tells the agent to check the codebase and standard library before writing, which costs input tokens. A small Cursor SDK test in June 2026 found higher estimated cost on three of five models, up to 55% on Composer 2.5 Fast.
Is Ponytail free?
Yes. Ponytail is open source under the MIT license and installs as a Claude Code or Codex plugin, or as an AGENTS.md rules file for other agents. You still pay your normal model usage.

Sources

Get each breakdown before it makes the rounds

You get one email when a new source-backed analysis goes live: what AI agents actually cost, which models are worth running, and what the benchmarks really mean. No hype.

No spam. Unsubscribe anytime.

← Back to Coding agents