Proof
Five benchmarks. Five perfect scores.
RedPick was submitted to leading public benchmarks for autonomous penetration testing and scored 100% on all of them. No cherry-picking, no partial results, no asterisks. Every score below was achieved black-box, or source-free grey-box.
XBOW validation benchmark
What it tests. An open-source benchmark maintained by the XBOW project: 104 challenges spanning 18 vulnerability categories, from XSS (23 challenges) and IDOR (15) to HTTP request smuggling, race conditions, and Phar deserialization. Challenges are designed to be realistic and representative of production web applications.
RedPick’s result. RedPick solved 104 out of 104 challenges in pure black-box, no-hint mode, the hardest testing configuration. The benchmark’s own creators score 85%. The next best black-box result is 91% (with hints). Shannon Lite reaches 96.15%, but with full source code access.
Next best was white-box, with full source code. RedPick was black-box.
PortSwigger Web Security Academy
What it tests. The industry-standard training platform for web application security: 274 labs across 31 vulnerability categories and 3 difficulty tiers (Apprentice, Practitioner, Expert). Categories include SQL injection, XSS, CSRF, SSRF, XXE, deserialization, access control, HTTP request smuggling, web cache poisoning, prototype pollution, and DOM-based vulnerabilities.
RedPick’s result. RedPick completed all 274 labs plus 5 mystery labs (unreleased challenges provided directly by PortSwigger) with a 100% success rate, and achieved the EXPERT proficiency level, the highest tier in PortSwigger’s classification.
EXPERT proficiency, PortSwigger's highest available tier.
HackBench
What it tests. A benchmark for evaluating autonomous cybersecurity agents on real-world software: 16 CVE-based challenges across 3 difficulty tiers (Easy, Medium, Hard) running full application stacks, including WordPress with LearnPress, Chatwoot, XWiki, Lucee Server, and CKEditor, with a maximum score of 4,000 points.
RedPick’s result. RedPick scored 16 out of 16 challenges with a perfect 4,000 out of 4,000 points in black-box mode. No other publicly reported autonomous tool has achieved a perfect score on HackBench.
First autonomous tool with a perfect score on HackBench.
ProjectDiscovery Vibe-Coding Benchmark
What it tests. Three full-stack web applications (Vaultbank, Medportal, Claimflow) built by an AI coding agent and seeded with a curated 74-vulnerability answer key. It spans critical business-logic flaws, authentication and authorization failures, IDOR, mass assignment, CSRF, weak crypto, and a long tail of low and informational findings.
RedPick’s result. RedPick reached 74/74 ground-truth coverage with 0 false positives, and verified 78 additional code-backed vulnerabilities outside the answer key: 152 total confirmed findings and 10 end-to-end attack chains (8 Critical, 2 High). The Neo baseline published by ProjectDiscovery scored 66/74 with 5 false positives.
Neo baseline: 66/74 with 5 false positives. RedPick: 74/74, zero false positives.
Escape Duck Store
What it tests. A deliberately vulnerable FastAPI and React e-commerce app with a documented REST API, used in Escape’s April 2026 comparison of agentic pentesting tools. Escape scored five tools against 20 known vulnerabilities: its own multi-model tool led at 15/20, with a raw Claude Code (Opus 4.6) baseline at 14/20.
RedPick’s result. RedPick found all 20 known vulnerabilities in a single source-free grey-box run (URL, the OpenAPI pointer, and default credentials only, no source code), and kept going: 88 validated extra findings beyond the answer key at 95.6% precision. Caveat: RedPick was not part of Escape’s study and these counts are self-scored, not validated by Escape.
Escape's own multi-model tool: 15/20. RedPick: 20/20.
XBOW, full breakdown
104 challenges. 18 categories. 100% in pure black-box.
All 18 categories at 100%
Benchmark methodology
Run under the strictest available conditions.
Black-box mode
No source code, no internal documentation, no hints provided to RedPick.
Fully autonomous
No human intervention during test execution. RedPick planned, executed, and verified each exploit independently.
Reproducible
These benchmarks are publicly available. Results can be independently verified by running the same suites.
No cherry-picking
Every challenge in every benchmark was attempted. Scores reflect the complete suite, not a selected subset.
FAQ
Common questions.
Can AI actually find real vulnerabilities?
Yes, and the results are public. RedPick scored 104/104 on XBOW, 274/274 on the PortSwigger Web Security Academy, 16/16 on HackBench, and 74/74 on ProjectDiscovery, all in black-box mode. Those cover the vulnerability classes that appear in real engagements: injection, broken access control, business logic, SSRF, and multi-stage exploitation chains.
Are the benchmark results independently verifiable?
Yes. These are public, third-party benchmarks — anyone can reproduce them. RedPick also publishes per-challenge walkthroughs and Tier-1 evidence on GitHub, so the findings can be reviewed and re-run, not just taken on trust.
How does RedPick compare to other autonomous pentest agents?
On XBOW's own benchmark RedPick is the only system to reach 104/104 (100%) in pure black-box, no-hint mode — ahead of the next published results, and without the source-code access that some higher-scoring white-box agents rely on. The full per-agent comparison is in the XBOW write-up.