How Context Mode cuts the cost of AI agents in CI
An AI agent in CI does the same job again and again: review a pull request, fix a failing test, label an issue. Each run sends a large prompt, and most of that prompt is the same every time. Context Mode sits between the agent and the model and cuts what you pay for that repeat. On five CI jobs the measured saving is 20% to 23% per run, with the same quality. This post names the five things it does, and the one thing it cannot change.
TL;DR
- Smaller first request: request 1 is 30% to 42% smaller.
- Cache across runs: run 2 reads the whole prefix from cache.
- Cache lifetime that fits the repo: one hour or five minutes, picked from how often the job reruns.
- Repeated tool output sent once.
- Large output folded to head, tail and a docid the agent can read back.
Why CI saves more than a chat
A CI job is short and it repeats. We measured how often real repos rerun the same job over seven days. Issue triage in anthropics/claude-code ran 985 times with a median gap of 1.9 minutes. PyTorch's PR review ran 270 times, also 1.9 minutes apart. LobeHub's ran 466 times, 12 minutes apart.
The prompt barely changes between those runs. If the model provider can read it from cache, each run is cheap. If every run writes it again, each run is not. Everything below is about making the first case happen.
1. A smaller fixed prefix
Every request starts with the same block: the tool list, the system text, the skill and agent lists. Context Mode trims it. On the gateway arm, request 1 was 30% to 42% smaller in every area we measured (lab notes). In the @claude fix job, request 1 went from 20,155 to 12,358 tokens.
A smaller prefix pays twice. The first cache write is smaller, and so is every cache read after it.
2. A cache that survives to the next run
Claude Code adds a git-status block near the top of the prompt. It changes on every run, so every run wrote the whole prefix to cache again: run 1 wrote what run 0 had already written. Context Mode moves that block behind the stable part. The text does not change. Only its place does.
Result in a back-to-back test: run 2 read 10,543 tokens from cache and wrote 0.
3. A cache lifetime picked from the rerun gap
The provider sells two lifetimes. On Sonnet 5.5 a five-minute write costs $2.50 per million tokens, a one-hour write costs $4, and a read costs $0.10. The long write costs $1.50 more, so it only pays if the next run comes within the hour often enough.
Context Mode counts how often each repo reruns the job and picks per repo. Claude Code's own issue triage came back inside five minutes 79% of the time and inside an hour every time, so it gets one hour. A review job that came back inside five minutes 1% of the time, and inside an hour 25%, stays on five minutes.
4. Repeated tool output is sent once
Agents read the same file or log twice. The second copy in one request becomes a one-line pointer to the first. The first copy stays whole, so the model loses nothing.
5. Large output is folded, and readable
A 60 KB test log rarely needs 60 KB. Context Mode keeps the head and the tail and adds a docid. On CI turns the agent can read the middle back by that docid when it needs it.
What it cannot change
The model's answer and its thinking. In our split of a direct run, the answer is 27% of the cost and thinking is 7%. Context Mode does not touch either, and that is why the saving is about a fifth and not more. Where the money goes shows the full split.
The measured result
Each row is a paired test: the same tasks, the same model, one arm direct and one through Context Mode. Ranges are 95%. The plan was pushed before the first paid run.
| CI job | Saved | Direct | Context Mode | Per 1,000 runs | Cheaper | Quality | Evidence |
|---|---|---|---|---|---|---|---|
| CI failure fix | 22.8%Sonnet 5.5, 95% range 11.6 to 32.2% | $0.0502 | $0.0420 | $8 | 14 of 16 | Tests pass 16 of 16 direct, 15 of 16 with Context Mode (McNemar p = 1) | Workflow · Tasks |
| @claude fix (issue to PR) | 21.8%Sonnet 5.5, 95% range 10.3 to 32.1% | $0.1113 | $0.0872 | $24 | 16 of 19 | Tests pass 9 of 19 direct, 10 of 19 with Context Mode (McNemar p = 1) | Workflow · Tasks |
| Issue triage and labels | 22.4%Sonnet 5.5, 95% range 15.5 to 27.7% | $0.0281 | $0.0223 | $6 | 24 of 26 | Label F1 0.499 direct, 0.478 with Context Mode, inside the 0.05 margin | Workflow · Tasks |
| CI failure fix | 19.7%Opus 5.5, 95% range 4.1 to 31.5% | $0.1691 | $0.1320 | $37 | 6 of 8 | Tests pass 8 of 8 direct, 8 of 8 with Context Mode (McNemar p = 1) | Workflow · Tasks |
| @claude fix (issue to PR) | 21.4%Opus 5.5, 95% range 7.1 to 35.4% | $0.3350 | $0.2759 | $59 | 7 of 8 | Tests pass 3 of 8 direct, 3 of 8 with Context Mode (McNemar p = 1) | Workflow · Tasks |
Claude Code headless (claude -p), Sonnet 5.5 and Opus 5.5, 2026-11-01, the live gateway. Cost per run at list price, warm runs. Saved: each task gives a cost ratio, Context Mode over direct; the saving is 1 minus the geometric mean of those ratios, so it can differ from the two mean costs. Per 1,000 runs: the direct cost minus the Context Mode cost, times 1,000. Proof: the round · raw results (in the lab) · data file · the plan, written 2026-11-01 23:39:27 UTC, before the first run at 2026-11-01 23:40:03 UTC · reproduce it.
Quality is equal: tests pass, and label F1 is within range. Plan: PLAN-r2. Results: RESULTS-r2. Every run row: final.json.
Turn it on
Install your agent its own way, then run two Context Mode commands. The first writes the workflow lines. The second makes a CI key and stores it as a repo secret, with no key on screen.
curl -fsSL https://claude.ai/install.sh | bash
npx @context-mode/cli login
npx @context-mode/cli ci init
npx @context-mode/cli ci key create --repo owner/name --ghPick a job and see your own saving on the try page, or run the same tests in your own repo with the lab template and your own keys.