← Building Glasstrace

The Scarcity of Admissible Bugs in Agent-Debugging Research

Building Glasstrace, Part 6 · August 27, 2026

The most binding supply constraint in this company is bugs: specifically, bugs that are demonstrably resistant to source-only repair by current frontier agents. This article covers why they are rare and what their scarcity has changed in both the research program and the business strategy.

The admission bar is the product thesis

For a bug to enter the validation roster it must pass gates that reduce to two properties. The decisive fact — whatever separates a correct repair from a plausible one — must reside in what the application did at runtime rather than in what the code says. And a strong agent reading the source, unaided, must not be able to reconstruct that fact reliably.

This bar is the product claim, operationalized. Glasstrace matters exactly where those two properties hold, so every admission decision doubles as a market-sizing measurement.

What rejection looks like

The early value-fidelity screening produced the first form of the finding: bugs that are both runtime-required and not inferable are rare. Most real defects, including severe ones, leave enough evidence in source code for a frontier model to repair them without runtime information. The models are simply too capable at static inference for the middle of the difficulty distribution to need this product.

The most instructive rejection eliminated a theory I liked. For candidate MFG-RLY-018, difficulty was deliberately engineered by dispersing a defect across multiple files, on the assumption that cross-file dispersion defeats static reasoning. The IF-1b assessment failed the candidate anyway. Dispersed or not, the bug remained inferable from source. Difficulty for human readers and difficulty for static inference are different axes, and only the second defines this product’s niche.

July’s dedicated expansion cycle made the finding quantitative: 8 fresh candidates were screened, and 0 were admitted. The preregistered study roster stood at 2 bugs, MFG-RLY-009 and RPORT-004-obf, the subjects of Part 5, while the screening waves concentrated on a third hard case, MFG-RLY-021, the stale-cache defect of Part 4. By late August, 89.2 percent of all canonical result rows in the screening corpus concerned that single bug.

Second-order effects

On the research side, exposure is irreversible. Every diagnostic, pilot, and study executed against MFG-RLY-021 — hundreds of trials by now — makes it permanently unusable as a confirmation holdout. Set aside any question of model training data; our own harness, prompts, and oracles have co-evolved with the bug. The August 25 prelaunch planning is explicit on the consequence: previously inspected or rejected bugs are not confirmation holdouts, and genuine confirmation requires fresh, prospectively admitted incidents with no prior efficacy outcomes. Such incidents do not currently exist. The same planning documents order restraint in the meantime — do not search for fresh incidents now — so that the known product friction is repaired first and the scarce budget of unexposed incidents is spent on a study capable of carrying a claim.

On the statistical side, the roster size dictates depth over breadth. Work under these conditions means one deep bug and a prominent scope statement (“one seeded hard bug, one behavioral oracle”) rather than implied breadth. The temptation to suggest a six-repository, many-bug evidence base is real. The six repositories exist, but they come from the installation audit, not from the outcome studies, and conflating the two is the kind of error the evidence policy exists to block.

On the business side, the constraint is disclosed rather than argued away. The board materials carry it as a named risk with a quantified consequence: if the runtime-required class cannot be expanded, the low-revenue scenario becomes the base case.

A small niche or a high bar

The optimistic reading, which I hold tentatively, is that the screening process under-samples the true class. Screening candidates come from seeded or reconstructed defects in a single codebase, which selects for the kinds of defects one can manufacture. Production systems generate categories that are difficult to seed deliberately: race conditions under real concurrency, data-shape drift, third-party interface surprises, cache coherence across real deployment topologies. The defect in Part 4 belongs to exactly this genus and was found in the wild by our own instrumentation.

The strategy that follows is to stop manufacturing candidates and to build an intake path for organic incidents — which is also the design-partner motion on the commercial side, in which each design partner brings one real defect. The commercial funnel and the scientific holdout supply are the same channel. Whether that coupling is elegant or fragile, I do not yet know.

For anyone constructing evaluations: admission criteria are a market thesis, so instrument the rejection rate, not only the pass rate. In this program, a screen of 8 candidates with 0 admissions set the evidence budget, the holdout policy, the go-to-market design, and the register of every claim shipped since.

Supporting data and commit references

Evidence availability: references to glasstrace-sdk are public — the repository, commits, and files are linked in the series evidence manifest. All other repositories cited above are private; artifacts may be shared as needed, and the manifest publishes each cited artifact’s SHA-256 digest so that any shared copy can be verified against the version these articles describe.