Head to Head: The Four Major Coding Agents

I had four coding-agent CLIs installed: Claude Code, Codex, Cursor, and Kimi. I gave all four the same tasks, each graded by a test suite the agent could not see, and compared correctness, speed, and cost.

The lineup and the exact model each one ran:

Two of the four share a base model. Cursor's Composer 2.5 is Moonshot's open-weight Kimi K2.5 fine-tuned with reinforcement learning, not a from-scratch in-house model. A leaked API model identifier (kimi-k2p5-rl-0317-s515-fast) and Cursor's later acknowledgment confirmed it after a launch-day "in-house" claim (report). Kimi's kimi-for-coding is Moonshot's own K2.5-based coding model. So this is three model lineages, not four. The Cursor and Kimi rows below are two different harnesses running over one Moonshot base.

Method

Each agent ran headless in its own empty directory, told to write one file and nothing else. Grading was a pytest suite kept in a different directory, loaded through a SOLUTION_PATH env var. The model never saw an assertion.

Three tasks, increasing in difficulty:

1. Duration parsing (56 assertions). Parse "1h30m" and "15s30m1h2d" into seconds, format them back, and reject malformed input like "1h1h", "1.5h", "-5s", "1H".
2. Semver 2.0.0 precedence (40 assertions). The full pre-release ordering: 1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0. Numeric identifiers compare numerically, build metadata is ignored, leading-zero pre-releases are invalid.
3. A JSON parser from scratch (79 assertions). No import json, no eval. Match Python's json.loads default behavior exactly: surrogate pairs, the non-standard NaN / Infinity / -Infinity literals, last-duplicate-key-wins, literal control characters rejected inside strings, no trailing commas. json.loads is the oracle, so the expected answers cannot be wrong.

I wrote a reference solution for each task and ran it against the tests before running any agent. It caught three bugs in my test file.

The two easy tasks were a four-way tie

All four passed every assertion on tasks 1 and 2: 96 of 96 each.

The semver task was meant to catch the beta.11 vs beta.2 numeric ordering and the ignored build metadata. All four handled both. On a fully specified small task, correctness from a frontier model is no longer the variable.

The hard task only separated them across repeats

First pass, one run each: Codex, Cursor, and Kimi passed 79/79. Claude passed 78/79. It failed on -Infinity, raising ValueError instead of returning negative infinity. Its code had a -Infinity branch, but the leading - was routed into number parsing first, so that branch never ran.

I reran Claude. It passed 79/79. I ran it a third time: 79/79. The failure was not repeatable.

So I ran all four three times on the hard task:

Two of the four shipped one buggy implementation out of three runs. Codex and Kimi were clean across all three. Cursor's bad run was a missing bounds check on a truncated "[" that crashed with the wrong exception type. Judging any of them on a single run would have produced a ranking the next run erased.

Codex was fastest on every task

Wall-clock time, same task run back to back:

Codex was fastest on all three tasks. Kimi was slowest on all three, two to eight times slower than Codex for the same passing answer. Codex also produced the fewest output tokens: 530 on the duration task, against Claude's 858 and Cursor's 2,454.

Only Claude Code reports per-task cost

Claude Code printed a price: $0.177 for the duration task, $0.576 for the JSON parser. The other three run on flat-rate subscriptions (Codex on a ChatGPT plan, Cursor on a $20/month Pro plan, Kimi on Moonshot's), so per-task cost is not exposed. If you need to know what a given task cost, only the per-token tool can tell you.

Sandbox defaults differ

• Codex: read-only by default. Its first run refused to write the file at all (writing is blocked by read-only sandbox) until I passed --full-auto.
• Claude Code: blocked my --dangerously-skip-permissions call through its own classifier. It ran on the scoped --permission-mode acceptEdits. Inside that sandbox it could not run Python to check its own work.
• Cursor: ran on --force --trust and executed Python itself to self-verify.
• Kimi: ran on --yolo with no friction, but needed an OAuth browser login first. The other three were already authenticated.

Scope

This measures spec'd, single-file tasks with one correct answer and machine grading. On that class, the four are correctness-equivalent on easy work and separated only by variance and speed on hard work. It does not measure the work that fills a day: a 40-file repo, a vague ticket, knowing which abstraction already exists. Green checks on three functions do not predict that.

Verdict

For spec'd tasks, use Codex. It was fastest on all three tasks, produced the fewest tokens, and was one of two agents clean across all three hard-task runs. The cost is its default sandbox (read-only until --full-auto) and its flat-rate billing, which hides per-task cost.

Kimi was the other agent clean 3 of 3, at two to eight times the wall-clock time. Claude Code is the only one that reports cost and the only one that refused to disable its own guardrails, and it shipped one buggy run. Cursor self-verifies and runs mid-pack on speed, and it also shipped one buggy run.

Cursor and Kimi run on the same Moonshot K2.5 base. Cursor's RL fine-tune and serving made it faster (111s vs 169s on the JSON parser); Kimi was slower but never missed across three runs. The gap between them comes from the fine-tune and serving layer, not the base model.

To run this yourself: give every CLI the identical prompt in print mode (claude -p, codex exec --full-auto, agent -p --model composer-2.5 --force, kimi --quiet --yolo), have each write one file in its own empty directory, keep the test file out of that directory, then point SOLUTION_PATH at the output and run pytest three times. One passing run is one sample.