A companion to Escape's Duck Store benchmark. RedPick ran the same article-parity target — a FastAPI + React e-commerce app — with only a URL, the OpenAPI pointer, and the default credentials, no source code: 20 / 20 known vulnerabilities found (100% strict key score) over a single ~7h run, plus 88 validated extras beyond the answer key and 95.6% extended precision.
Published 2026-06-06 by the RedPick team. Reference: Escape, "Benchmarking AI Pentesting Tools: A Practical Comparison" (2026-04-30, Gwendal Mognier). RedPick's run was source-free grey-box — URL + /openapi.json pointer + the article-default credentials only, no source code, no answer-key access during testing. RedPick was not part of Escape's study and Escape did not validate our results; the numbers below are our own scorer's output. Full finding tables and scoring methodology are on GitHub.
The short version. Escape's Duck Store post did something the agentic-pentesting market needs: it put tools on a real vulnerable app, published concrete counts, and separated confirmed findings from claims. Its leaders were Escape's own multi-model tool at 15/20 and a raw Claude Code (Opus 4.6) baseline at 14/20. We added Duck Store to RedPick's benchmark suite and ran it under the same article-parity inputs: 20/20. The interesting part isn't the model — it's what closed the gap from a strong 14/20 raw baseline to a full 20/20.
This is a companion data point, not an externally controlled bakeoff. Escape's numbers were produced and validated by Escape; RedPick's were produced by RedPick's own post-run scoring and evidence review. Our single run also took longer (~7h) than Escape's 4h and far longer than the 10-minute raw Claude Code baseline — the trade was breadth and verification depth, not speed.
TL;DR
| Metric | Escape benchmark | RedPick run |
|---|---|---|
| Target | Duck Store (FastAPI + React) | Duck Store |
| Known vulnerabilities scored | 20 | 20 |
| Best tools in Escape's study | Escape 15/20 · Claude Code 14/20 | — |
| Found | — | 20 / 20 |
| Strict score | 75.0% (Escape) | 100.0% |
| Runtime | 4h (Escape) · 10 min (Claude Code) | ~7h |
| Source code | None (most tools) | None |
| Context given | URL, OpenAPI, credentials | URL, OpenAPI, credentials |
| Missed key items | 5 (Escape) | 0 |
| Validated extras beyond key | not a reported metric | 88 |
| Precision signal | 6.25% FP (Escape) | 95.6% extended precision (5 FP) |
The result isolates a point Escape's own table already hints at: the raw model is not the product. Shannon and Strix both ran the same DeepSeek v3.2 family and landed at 6/20 vs 1/20 — same model, very different outcome. The variable that matters is the system around the model.
What we ran
The target was Escape's hosted Duck Store (https://duck-store.escape.tech/), with the article-parity setup: the /openapi.json documentation pointer, the /api/ surface, admin credentials admin/admin, and a low-privilege user/password account — no source code, and no answer-key access during testing. (The low-privilege account wasn't consistently present on the hosted instance, so RedPick registered and verified it before the run, then used it as the article intended.)
We skipped only the broad, slow recon that would have been redundant — the article already gives the target, the API doc location, and the credentials. Everything else was tested: access control, authentication, API surface, business logic, injection, SSRF, and misconfiguration. The run used RedPick's standard dual-engine workflow — a primary engine driving the testing, a second independent engine re-checking high-impact findings as a peer, automatic re-runs before closure when a pass didn't complete cleanly, and scoring performed separately, after the evidence was frozen. The testing agent never reads the answer key.
Score
| Result | Count |
|---|---|
| Known vulnerabilities in the article-parity key | 20 |
| Found / Partial / Missed | 20 / 0 / 0 |
| Strict score | 20.0 / 20 = 100.0% |
| Runtime | ~7h |
The 20 known items break down as 7 Critical, 9 High, 3 Medium, 1 Low, spanning broken access control (5), authentication failures (4), business logic (4), injection (2), SSRF (2), open redirect / misc (1), and information disclosure (2). RedPick confirmed each with a working, reproducible proof of concept — including the one finding Escape highlighted as separating the top two tools, the unauthenticated testimonial modification that every other tool in the study missed.
The misses in Escape's table clustered exactly where single-pass agents thin out: multi-role access control, business-logic validation, authentication edge cases, and API routes that are reachable but not obvious from the browser flow. RedPick's methodology forces those areas to be tested rather than relying on one agent to "remember" them — it carries every identity (unauthenticated, registered user, admin) through discovery and testing, reasons from the API spec and validates with live requests, and re-checks high-impact findings with the second engine.
Full 20-item table with CVSS, CWE, OWASP, and endpoints →
The extras are the product signal
The 20-item key is the fair head-to-head measurement — it matches the Escape article, and RedPick hit all 20. But RedPick didn't stop at the finite key.
After scorer hygiene, the run produced 108 extended true positives (the 20 key items + 88 credited extras), 5 false positives, 1 near-duplicate, and 95.6% extended precision. The honest read of the 88, though, is not "88 new bugs": they're finding files, and multiple specialist agents re-reported the same issues. Deduplicated, the 88 collapse to 26 distinct vulnerability classes — roughly half are deeper instances of the 20 scored items (the app exposes the same weakness across many objects and routes), and 12 classes are genuinely new beyond the article's key (e.g. a shadow product API allowing non-admin create/modify, checkout/inventory race conditions, DOM-based open redirect across SPA routes, missing JWT revocation, and HTTPS→HTTP credential exposure).
That's why we report both: 20/20 for comparability, and the extended view for real-pentest signal — whether the system keeps finding useful attack surface after the answer key is exhausted. Full deduplicated extras table (26 classes) → · scoring methodology →
Why the result changed
The jump from a strong 14/20 raw Claude Code baseline to 20/20 isn't a model story — it's the engineering around the model. A free-form "test this app" prompt thins out exactly where this benchmark is hardest: the last 30% lives in multi-role access control, business-logic validation, authentication edge cases, and API routes that are reachable but not obvious from the browser flow. Rather than asking one agent to stay comprehensive in a single monologue, RedPick carries every identity through discovery, testing, and evidence; reasons from the API specification and validates with live requests; re-checks high-impact findings with a second independent engine; and re-runs affected work when a pass hits operational friction instead of accepting a partial result. Scoring then happens separately, after the evidence is frozen — so the result reflects what was discovered and proven through the target, not benchmark leakage.
Limitations
This is not a universal leaderboard. RedPick was not part of Escape's original evaluation and Escape did not validate our results. We ran a later public hosted Duck Store instance, not necessarily the exact container state from the article; the current public target also exposes a broader vulnerability page than the article's 20-item score, and our key intentionally excludes those extra items for article parity. We ran one session, not a multi-run average. And RedPick is a full orchestration system, not a raw Claude Code baseline — comparing it to Escape's raw Claude Code row answers a different question.
The fair conclusion is narrow:
On Escape's Duck Store benchmark, under article-parity source-free grey-box conditions and internal RedPick scoring, RedPick found all 20 known vulnerabilities for a 100% strict score, and kept finding validated attack surface well beyond the key.
Escape deserves credit for publishing a benchmark concrete enough to reproduce — small enough to score, realistic enough to expose the gap between "an agent can find bugs" and "a system can finish the pentest." On that benchmark, RedPick finished it.
Want to see what RedPick finds in your applications? Request a demo and we'll run a proof-of-concept scan, or explore our managed pentest. For the full program, see all benchmark results.
Related: Doyensec: Aikido vs XBOW, black-box · 104/104 on XBOW · Vibe-Coding: 152 Findings, 0 FP
— the RedPick team, 2026-06-06