Skip to content

This is one of the two tracks of the next series. The other is LEB Default.

AI Benchmark · LEB · next series

LEB Security proposed

One of the two tracks of the next LEB series: finding and fixing vulnerabilities in a system that works, without breaking it. The task is defensive: the agent gets the system and its contract, and the attack checks belong to the evaluator and do not travel in the package. How LEB works today, and how its levels differ, is on the Benchmark page.

Read the design

There are no results yet. This page says where the series stands and what will appear here. Nothing on it is a score.

What this track measures

Finding and fixing vulnerabilities in a system that works, without breaking it. A fix counts only when all of these hold:

  • the reference attack and its variations no longer work;
  • the legitimate user keeps the function;
  • the contracts of the manifest still hold.

Turning a feature off, refusing every request or answering every input with an error is not a fix. Compatibility stays in the score: a fix that closes the hole and breaks the system is a bad delivery. False positives are measured on correct code that looks risky.

The Default and Security scores of a configuration are shown side by side and never added.

Where the series stands

The next series is a proposal: its design was decided on 12 October 2026, and its normative texts (the spec, the protocol, the scoring and the levels) are written after a pilot has validated the process. It changes nothing in LEB-100 or LEB-300, whose results stay as published. There is no scoreboard yet.

The reference systems

Each track is measured on several reference systems, and each system yields both tracks: two planting profiles of the same clean code. The same letter is the same system. The pilot will have 3 systems; the evaluation set, about 6.

System Domain Stack Instance Status
A Condominium management (CondoDesk) Java 21 · Maven · PostgreSQL LEB-S-300-A In authoring

How a case runs

A case is one system, one track and one agent configuration.

  • 5 runs per case, and all of them count.
  • The requested model is pinned in the client, in the main conversation and in every subagent, so that no substitute can answer.
  • An automated lab runs each case, from the restoration of the machine to the filing of the evidence.

What will be published

For each configuration and track:

  • the quality of the evaluable deliveries, with its uncertainty interval;
  • the completion rate with the requested model;
  • the expected score per scheduled attempt;
  • blocks (at the start and during the work), model switches, exhausted budgets and lab failures, each counted separately.

The answer key

The answer key of an instance is published on GitHub with the instance, before its first run, and the machines that run the agents cannot reach GitHub. Each run also records the model's training cutoff, for what a model may have seen in training.

Comparison

A score of the next series is never set beside one of LEB-100 or LEB-300.

Read more