Generative AI

GLM-5.2 vs Claude Opus 4.8: what Semgrep's test shows

Updated October 25, 2026. An earlier version of this article presented the Semgrep test as a direct duel between GLM-5.2 and Claude Opus 4.8. The 32% score actually refers to Claude Code, and the same Opus 4.8 model reached 53% in…

Jean-Baptiste Duquesne

Jean-Baptiste DuquesneSEO, GEO & content operations · Published 24 June 2026 · Updated 27 October 2026

Illustration for the article: GLM 5.2 vs Claude Opus 4.8: cybersecurity benchmark

Updated October 25, 2026. An earlier version of this article presented the Semgrep test as a direct duel between GLM-5.2 and Claude Opus 4.8. The 32% score actually refers to Claude Code, and the same Opus 4.8 model reached 53% in Semgrep's own harness. The text below has been rewritten from Semgrep's publications.

The test in short

On June 22, 2026, Semgrep, a company specializing in code security, published a benchmark on the detection of IDOR vulnerabilities in real open-source applications. Its headline result: GLM-5.2, an open-weight model from Zhipu AI (Z.ai), scored a 39% F1, ahead of Claude Code at 32%, at a cost of roughly $0.17 per vulnerability found. This result deserves a careful reading, because the test compares configurations (a model plus its harness) more than models alone.

The models involved

  • GLM-5.2 is Z.ai's flagship model, first made available in June 2026 to GLM Coding Plan subscribers, according to DataNorth. Its weights are published on Hugging Face under the MIT license, with a one-million-token context. DataNorth reports API pricing of $1.40 per million input tokens and $4.40 per million output tokens.
  • Claude Opus 4.8 was released by Anthropic on May 28, 2026, at $5 per million input tokens and $25 per million output tokens. Anthropic highlights that it is more likely to flag uncertainties about its work.

What the figures really say

An IDOR (Insecure Direct Object Reference) is an access control flaw: by changing an identifier in a request, a user can view or modify another user's data. The F1 score combines precision (few false alarms) and recall (few missed flaws). In Semgrep's table:

  • Semgrep's multimodal pipeline reaches 61% with GPT-5.5 and 53% with Claude Opus 4.8.
  • GLM-5.2, run with a simple prompt in Pydantic AI and no endpoint discovery, gets 39%.
  • Claude Code gets 37% with Opus 4.6 and 28% with Opus 4.8/4.7. The text of the article gives 32% for Claude Code, a figure that does not appear as such in the table.

The cost of $0.17 per vulnerability found is only given for GLM-5.2, so it cannot be compared with the other models. Semgrep itself stresses the limits: "one task, one dataset, one run", and a result that does not show that open-weight models have caught up in general. The main lesson is that the harness, the infrastructure that guides the model through the code, weighs more than the model: the same Opus 4.8 goes from 28% in Claude Code to 53% in Semgrep's pipeline. The same logic applies to other agentic tasks: in the experiment of Claude AI playing Minecraft, the model acted through text commands that defined what it could see and do. The title of Semgrep's article refers to Anthropic's Mythos models, the tier above Opus, whose context we explain in our article on the suspension of Mythos 5 and Fable 5.

The follow-up audit: grounded models, low recall

On July 17, 2026, Semgrep published an audit of its own benchmark to check whether the models were really reasoning or simply recognizing familiar patterns. Its conclusion: the models are not gaming the test, but recall remains low for all of them. On the most complex repository tested, GLM-5.2 and Claude Opus 4.8 each found about 14% of the 27 known IDORs per scan, and 19 of these flaws were missed by every model. Claude was the most stable from one run to the next, while GLM-5.2 varied more.

Why it matters for a marketing team

Marketing teams manage many exposed web assets: landing pages, customer areas, forms, preference centers. An access control flaw on one of these tools can expose customer data and damage the brand's reputation. Three practical lessons:

  • Ask about the method, not just the model: when a provider or agency claims to audit code with AI, ask which harness it uses and how results are checked.
  • Watch where the code goes: an open-weight model like GLM-5.2 can be hosted internally, which avoids sending proprietary code to a third-party API. Our guide on cloud AI vs local AI sets out the criteria.
  • Keep human review: with recall of around 15% on a complex case, AI is a first filter, not a replacement for security experts or traditional analysis tools.

Open-weight models are progressing fast, as shown by Kimi K3, released a few weeks later. Choosing between them is a matter of AI governance as much as of benchmarks.

About Brandeploy

Brandeploy is a creative automation platform that helps marketing teams produce and adapt their assets, from banners to localized versions, while keeping a consistent brand identity. Book a demo to see the Brandeploy platform in action.

FAQ

Did GLM-5.2 beat Claude Opus 4.8 in Semgrep's test?

Partly. In Semgrep's IDOR benchmark of June 22, 2026, GLM-5.2 scored a 39% F1, ahead of Claude Code, which the article puts at 32% (the table lists 28% for Opus 4.8/4.7 and 37% for Opus 4.6). But the same Claude Opus 4.8 reached 53% in Semgrep's own multimodal pipeline.

What is an IDOR vulnerability?

An IDOR (Insecure Direct Object Reference) is an access control flaw: by changing an identifier in a request, a user can view or modify another user's data. It is hard to detect with traditional static analysis because it depends on the application's logic.

What does the $0.17 per vulnerability figure mean?

Semgrep estimates that the GLM-5.2 run cost roughly $0.17 per vulnerability found, based on the model's pricing. No equivalent figure is given for the other models, so it cannot be used for a direct cost comparison.

What is a harness and why does it matter?

The harness is the infrastructure around the model: it explores the code, selects the context and parses the results. In Semgrep's test, it weighs more than the model: Claude Opus 4.8 goes from 28% in Claude Code to 53% in Semgrep's pipeline, which enumerates endpoints.

Can AI replace a human security review?

No. Semgrep's follow-up audit of July 17, 2026 shows that the models really reason, but that recall stays low: about 14% of known IDORs per scan on the hardest repository, and 19 of 27 flaws missed by every model. AI is a first filter, to be combined with experts and traditional tools.

Jean-Baptiste Duquesne

Written by

Jean-Baptiste Duquesne

SEO, GEO & content operations

Web pioneer since 1995, founder of the recipe site 750g.com (sold to Webedia), now industrialising SEO and GEO content production with AI.

Ready to scale your content production?

Book a demo