DEV Community

Hamza Ahmad Aslam
Hamza Ahmad Aslam

Posted on Originally published at hamzaahmadaslam.com

Gemini 4 Carbon review: Google's claims, independent results and what testers found

Originally published at hamzaahmadaslam.com.

How this article was made: drafted with AI assistance from a researched brief, then checked against primary documentation before publishing.

Gemini 4 Carbon, often searched as Google Carbon, is Google DeepMind's newest top model. Google announced it on 28 October 2026 for a small group of security teams, then made it generally available on 10 November 2026. It is built on Gemini 4 Argon, which stays on sale beside it.

The short version, as of 11 November 2026: Carbon is a strong model at a low launch price, and it leads several independent leaderboards. It does not lead most of them. Artificial Analysis now ranks it fourth, behind three settings of Claude Fable 5.2. It is also slow, it writes a lot of tokens, and it makes up answers more often than Argon does.

This review keeps three kinds of evidence apart: what Google says, what outside labs measured, and what the first users wrote after a day with it.

What is Gemini 4 Carbon, and who can use it?

Carbon is Google's model for the hardest tasks: long agent runs, multi-step coding and debugging, and scientific research. The model card says it uses more thinking tokens than Argon to check its own work. Google's own advice in its launch post is that Argon remains the better choice for most software engineering and knowledge work.

Item Gemini 4 Carbon
Announced 28 October 2026, for cyber defenders in Google's Fairwind Program only
Generally available 10 November 2026
Developers Paid tier of the Gemini API in Google AI Studio, and Gemini for business
Consumers Google AI Ultra subscribers, in the Gemini app, AI Mode and Antigravity
Price until 30 Nov $2 per million input tokens, $10 per million output tokens
Price from 1 Dec $4 input, $20 output
Context window Up to 1 million tokens, input and output combined
Knowledge cutoff March 2026

Gemini AI Pro subscribers keep Argon and do not get Carbon in the app. Google has not published regions, rate limits or a Vertex AI date for Carbon.

Security teams can apply to the Fairwind Program for a version with safeguards tailored to defensive work, such as checking whether a vulnerability can be exploited and analysing malware.

What does Google claim?

Most of Google's numbers come from its own runs, and its strongest claims are in cybersecurity. VentureBeat reported that Google ran these cyber tests without guardrails, through the Fairwind access path.

Benchmark (Google's run) Gemini 4 Argon Gemini 4 Carbon
SEC-bench Pro (finding vulnerabilities) 83.4% 88.2%
ExploitGym 42.0% 54.8%
CWE-bench v1 68.0% 71.0%
Agentic honesty, 375 tasks 70% 92%

Google also says Carbon ranks first on CursorBench 4.0, Perplexity's WANDR and Zapier's AutomationBench at less than half the cost per task of the next best outside model. It published no scores for CursorBench or WANDR, so those two claims cannot be checked yet. Zapier did publish its AutomationBench report: Carbon scored 53.9% at High thinking, for about $2.12 per task, in runs made before the public release.

On safety, Google reports that external red-teamers at Trajectory Labs found no new universal jailbreaks in 132 hours and more than 13,000 requests. The same post says resistance to prompt injection "remains an area for continued improvement."

What did independent labs measure?

Five outside groups had published Carbon results by 11 November. Their numbers mostly agree: Carbon is near the top, and Claude Fable 5.2, released a day before Carbon's general availability, is ahead on the broad indexes.

Evaluator Carbon result For comparison
Artificial Analysis, Intelligence Index 60, fourth Fable 5.2 (max) 65, Opus 5.5 (max) 59, GPT-6 Astra 57, Argon 50
Artificial Analysis, Coding Agent Index 59 Claude Code with Opus 5.5 (max) 62
Vals AI, Vals Index 70.45%, third of 36 Fable 5.2 71.31%, first
Arena, text First, 1,526 (preliminary, 4,282 votes) Argon 1,522, Opus 5.5 (High) 1,506
Arena, web development Fourth, 1,768 (preliminary) Fable 5.2 (xHigh) 1,836, first
ARC Prize, ARC-AGI-2 94.2% Fable 5.2 93.8%
ARC Prize, ARC-AGI-3 65.1% Fable 5.2 81.2%

Artificial Analysis ranked Carbon first at launch on 28 October, with 59 points. That ranking is out of date. Fable 5.2 has since scored 65, and Carbon's own score moved to 60. Articles that still call Carbon the number one model are quoting the October result.

Vals AI puts Carbon first on its Finance Index, at 67.05%, and fifth on Terminal-Bench 4.0, at 63.13%. One test of the Vals Index costs $10.51 with Carbon.

The Arena text leaderboard has Carbon in first place, but the score is marked preliminary and rests on about 4,300 votes. Its 4-point lead over Argon is smaller than the plus or minus 10 points shown beside Carbon's score.

The ARC Prize leaderboard shows the widest split. Carbon edges Fable 5.2 on ARC-AGI-2, then trails it by 16 points on the newer ARC-AGI-3. It was cheaper to run there: $7.2K against $21.9K.

No METR time-horizon result and no public SWE-bench Verified score existed for Carbon on 11 November.

Does Carbon hallucinate more than Argon?

Yes, on the one independent test that measures it. In its launch analysis, Artificial Analysis found that Carbon answers more knowledge questions correctly than Argon, but that its hallucination rate rose from 4% to 11%. Artificial Analysis notes that 11% is still the second lowest of any model it has tested, behind Argon.

In practice, Carbon knows more and guesses more. If your use case punishes a confident wrong answer, such as legal, medical or financial text, test that rate on your own questions before you switch.

How fast is Carbon, and what does a task cost?

Carbon is slow and wordy. Artificial Analysis measured about 80 output tokens per second and ranks it 70th of 73 models for verbosity: it produced 205 million output tokens to complete the index. At High thinking, its leaderboard lists 67 seconds before the first answer token.

Per-token prices are low until 30 November, but the bill depends on how many tokens a task uses. Artificial Analysis lists these costs per task on its index, at Carbon's standard price of $4 and $20:

Model and setting Index score Cost per task
Gemini 4 Carbon (high) 60 $7.76
Claude Fable 5.2 (max) 65 $26.54
Claude Fable 5.2 (high) 62 $9.08
Claude Opus 5.5 (max) 59 $10.04
GPT-6 Astra (max) 57 $6.24

Two things follow. Carbon is cheaper per task than Fable 5.2 at a similar level, but Fable 5.2 at High scores 2 points more for about $1.30 more per task. And the launch price halves every figure in Carbon's row until 30 November, so a December budget built on November invoices will be wrong by a factor of two.

What are early users saying?

Few people had used Carbon for more than a day when this was written, so treat these as first impressions.

Simon Willison tested it on 10 November with his usual SVG drawing prompt at three thinking levels, and built a 3D scene in Antigravity that needed a second prompt to become a standalone HTML file. He gave no ranking, and noted the price is "scheduled to double in price to $4/$20 on December 1st."

The Hacker News launch thread was mixed:

  • One developer ran Carbon on a personal benchmark and found it matched Fable 5.2 on quality, but called it "still slower and more costly than Fable 5.2."
  • A user on the Ultra 5x plan wrote that they were "barely able to complete a single project" with Carbon before hitting the plan's limit.
  • Another commenter measured a time to first token of about 60 seconds at the 75th percentile and around 10 tokens per second, and called the model "completely unusable at the moment."
  • On the positive side, one user gave Carbon an unfinished research paper and found it raised points that other models had missed.

Bito, a company that sells an AI code review tool, published its own comparison inside Claude Code: Carbon scored 79.5% against 68.2% for Argon and 85.0% for Opus 5.5, while taking about twice as long as Argon per session. It is one vendor's test, but it matches the pattern above: better than Argon, behind Anthropic's top models on coding, slower.

What does Google's safety report say?

The Safety & Security Report has one detail most launch coverage left out. Google judged that Carbon was not a big enough step up from Argon to need a new full frontier-safety assessment, and it did not run a new round of outside capability testing for this release. It carried Argon's findings forward instead.

Those findings matter. Google says it cannot rule out that Carbon, without its outside safeguards, reaches the danger thresholds in its Frontier Safety Framework for cyberattack uplift and for biological information. It says its safeguards bring the remaining risk below those thresholds, and that it strengthened its cyber protections for Carbon because the model's exploit scores went up.

That is a defensible process, and Google published it. It also means the safety evidence for Carbon is mostly Argon's evidence plus targeted red-teaming of the safeguards. If you deploy Carbon in a product that browses the web or reads customer documents, treat prompt injection as an open risk, as Google itself does.

Should you switch to Gemini 4 Carbon?

It depends on what you run now and what the task is.

  • If you use Gemini 4 Argon and hit its limits on long agent runs or research, test Carbon on those tasks. Google itself still recommends Argon for the rest.
  • If you use Claude Fable 5.2 or Opus 5.5 for coding, the independent results above give no reason to move. Carbon trails on the coding indexes and is slower.
  • If your work is finance or tool-heavy business automation, Carbon's Vals Finance Index and AutomationBench results are worth a test, and the launch price makes November a cheap month to run one.
  • If wrong answers are costly, measure the hallucination rate on your own questions first.

Copy this checklist before you switch

Twenty of your own tasks will tell you more than any table on this page.

MODEL SWITCH CHECKLIST: Gemini 4 Carbon

Current model:
Candidate: Gemini 4 Carbon
Thinking level tested: Low / Medium / High

1. TASK SET
   20 real tasks from the last month
   Include 5 the current model got wrong
   Include 5 where a wrong answer would be costly

2. FOR EACH TASK, RECORD
   Correct? Yes / Partly / No
   Invented a fact? Yes / No
   Input tokens:          Output tokens:
   Cost at the December price ($4 / $20):
   Time to first token:   Time to finish:

3. COMPARE
   Tasks fixed by Carbon:
   Tasks broken by Carbon:
   Cost per correct answer, old vs new:

4. DECIDE
   Switch all / switch hard tasks only / stay
   Re-test date (after 1 December):
Enter fullscreen mode Exit fullscreen mode

Price the test at the December rate. A model that looks cheap in November can cost twice as much a month later.

Where Carbon fits in a business workflow

The model matters less than the limits around it. An agent running on Carbon still needs fixed permissions, a person's approval for anything that moves money or data, and a log you can audit.

For how the main rival compares, see the Claude Fable 5.2 benchmarks and price. For running costs, see what AI automation costs a small business. If you want a model like Carbon connected to WordPress, WooCommerce or your own systems with those limits in code, that is the work behind AI integration for business workflows.

Top comments (0)