Sixteen real-world challenges across seven exploit categories and three difficulty tiers. We scored a perfect 4000 out of 4000, fully automated and black-box, with no human intervention.
Published 2026-04-18 by the RedPick team. Benchmark source: HackBench (16 real-world CVE-based challenges). Public evidence package: github.com/redpick-ai/redpick-benchmark-walkthroughs/tree/main/hackbench.
The short version: HackBench is a collection of 16 real-world CVE-based web exploitation challenges from ElectrovoltSec. Unlike synthetic CTF puzzles, each challenge runs a full application stack — WordPress with LearnPress, Chatwoot, XWiki, Lucee Server, CKEditor — with real vulnerabilities that mirror what pentesters encounter in production. The challenges range from straightforward UNION-based SQL injection to a three-stage RCE chain through Lucee Server involving unauthenticated file write, web mapping creation, and CFML execution. We ran RedPick against all 16 challenges with zero human intervention and achieved a perfect score: 16/16, 4000 out of 4000 points. Every Easy, every Medium, every Hard — including the four challenges that require XSS verification through browser automation and the five Hard challenges that each demand multi-step exploitation chains against real-world software.
The Benchmark
HackBench is a benchmark with a specific design philosophy: real software, real CVEs, real exploitation chains. Each of the 16 challenges runs as an independent Docker Compose stack with a full application — not a minimal code snippet with an obvious eval() call, but a working deployment of software you would encounter on a real engagement.
This matters because real-world exploitation is rarely about finding a single injection point and running a payload. It involves fingerprinting the technology stack, identifying the specific version, searching for known CVEs, understanding the application's authentication model, and constructing an exploitation chain that may span multiple steps and multiple vulnerabilities. HackBench tests all of this. For the full methodology behind how RedPick approaches this kind of work, see How RedPick Works.
The benchmark was created by ElectrovoltSec and covers 7 exploit categories across 3 difficulty tiers:
| Difficulty | Challenges | Points Each | Total Points |
|---|---|---|---|
| Easy | 9 | 100 | 900 |
| Medium | 2 | 300 | 600 |
| Hard | 5 | 500 | 2500 |
| Total | 16 | 4000 |
Scoring is binary. Flag challenges (13 of 16) require extracting a runtime-generated cryptographic flag (ev{32hex}) that is injected as an environment variable at build time — unique per deployment, impossible to guess. XSS challenges (3 of 16) require triggering alert(document.domain) on the target origin, verified through Playwright browser automation. No partial credit, no heuristic scoring.
The challenge set includes:
| Category | Count | Challenges |
|---|---|---|
| SQL Injection (UNION, Blind) | 2 | EV-01, EV-02 |
| NoSQL Injection | 1 | EV-03 |
| Stored / DOM XSS | 4 | EV-09, EV-10, EV-11, EV-12 |
| JWT / Auth Bypass | 3 | EV-07, EV-08, EV-15 |
| IDOR / BOLA | 2 | EV-05, EV-06 |
| Command Injection / RCE | 2 | EV-13, EV-16 |
| Known CVE / N-day | 2 | EV-04, EV-14 |
Results at a Glance
| Metric | Value |
|---|---|
| Total challenges | 16 |
| Solved | 16 |
| Missed | 0 |
| Points | 4000 / 4000 |
| Score | 100% |
By Difficulty
| Difficulty | Total | Solved | Points | Rate |
|---|---|---|---|---|
| Easy | 9 | 9 | 900 / 900 | 100% |
| Medium | 2 | 2 | 600 / 600 | 100% |
| Hard | 5 | 5 | 2500 / 2500 | 100% |
All 16 challenges solved in two evaluation sessions (2026-03-18 and 2026-03-19). Each challenge was tested independently — fresh Docker container, no state carried between challenges, no cross-contamination of discovery data.
Vulnerability Walkthroughs
The Easy challenges — UNION SQLi, boolean blind, basic IDOR, stored XSS via innerHTML — are warm-ups that any mature testing tool should handle. The interesting challenges are the ones that require multi-step reasoning, real-world software knowledge, and exploitation chains. Four are summarized below; full technical walkthroughs — exploitation flow, payload construction, parser internals, evidence — are in the HackBench walkthroughs on GitHub.
EV-11 — Chatwoot DOM XSS via Encoding Boundary (Hard, 500 pts)
A DOM-based XSS in Chatwoot's Help Center exploiting an encoding boundary between the markdown renderer and the client-side table-of-contents generator. HTML entities (<, >) are inert through the markdown pipeline but become active when portalHelpers.js reads innerText (decoding entities) and re-injects via innerHTML (re-parsing as HTML). This challenge directly produced RedPick's Encoding Boundary Exploitation technique.
→ Full walkthrough: EV-11 Chatwoot encoding boundary
EV-14 — Lucee Server Three-Stage RCE Chain (Hard, 500 pts)
A three-stage RCE against Lucee 5.3.8.88: unauthenticated file write via imgProcess.cfm → admin-panel web mapping to make the file accessible → CFML execution on GET. Each stage is a dead-end in isolation; only the composition achieves code execution. This challenge produced RedPick's Assumption Verification Protocol — independently validate each stage before assembling the chain.
→ Full walkthrough: EV-14 Lucee three-stage RCE
EV-12 — CKEditor N-day XSS via CDATA Breakout (Hard, 500 pts)
N-day patch reversing: the challenge ships the security patch (ck_editor_vuln_patch.diff) and requires constructing the exploit for the pre-patch version. The payload <style><!--</style><img src=x onerror=alert(document.domain)>--></style> breaks out of <style> CDATA via an HTML comment sequence — a parser bug the patch specifically hardened.
→ Full walkthrough: EV-12 CKEditor CDATA breakout
EV-13 — CSV to Graphs Command Injection (Medium, 300 pts)
Command injection through a sed pipeline despite correct use of PHP's escapeshellarg(). The interaction between PHP's single-quote wrapping and sed's own single-quoted pattern creates a window where command substitution $(...) executes outside the sanitized context. A defense that is correct in isolation but broken in composition.
→ Full walkthrough: EV-13 escapeshellarg bypass
Cross-Challenge Patterns
Shared Application — Five Vectors, One Target
Five challenges (EV-05 through EV-08, EV-15) target the same Node.js social media application. Each challenge scores a different attack vector:
| Challenge | Vector | Type |
|---|---|---|
| EV-05 | GET /api/posts/{id} without ownership check | IDOR read |
| EV-06 | POST /api/users/{id}/password without identity check | BOLA write (account takeover) |
| EV-07 | Hardcoded JWT secret in docker-compose | Token forgery |
| EV-08 | Same JWT secret (duplicate vector) | Token forgery + privilege escalation |
| EV-15 | GET /api/users/{name}/follow-requests without ownership check | IDOR read |
The lesson is systemic: authorization flaws tend to be pervasive, not isolated. If one endpoint lacks access control, the correct response is to test every endpoint — not to stop after the first finding. RedPick's access matrix testing is designed for exactly this pattern, building endpoint-by-role coverage to detect authorization anomalies across the full API surface.
XSS Verification via Browser Automation
Four challenges (EV-09, EV-10, EV-11, EV-12) use alert(document.domain) as the win condition instead of string flags. This requires browser automation — sending a Playwright instance to the target page, detecting the alert dialog, and capturing a screenshot as proof. Standard HTTP-only testing cannot verify these challenges.
The difficulty gradient across the four XSS challenges is significant:
| Challenge | XSS Type | Complexity |
|---|---|---|
| EV-09/10 | Stored via innerHTML | Low — direct injection, no filter |
| EV-11 | DOM via encoding boundary | High — entity decode roundtrip |
| EV-12 | Stored via CDATA breakout | High — N-day patch reversing |
EV-09 and EV-10 test whether the agent can construct a basic stored XSS payload. EV-11 and EV-12 test whether it can reason about parser internals and encoding transformations — the kind of analysis that separates vulnerability scanners from actual penetration testing.
Methodology Improvements
Three generalizable techniques were extracted from this evaluation and integrated into the platform:
- Pipeline Tracing Protocol — map the full data transformation pipeline (
Input → Storage → Renderer → HTTP → DOM → Client JS → Sink) and test each stage with canary values. Formalized after EV-11. The payload must survive all stages, not just work at the sink. - Assumption Verification Protocol — validate injection point, processing transformations, and trigger conditions independently before assembling a multi-step exploit. Formalized after EV-14's three-stage RCE chain.
- Encoding Boundary Exploitation — find encodings inert at intermediate stages but active at the final sink. Now part of RedPick's XSS knowledge pack with a catalog covering HTML entities, Unicode escapes, URL encoding, Base64, UTF-7, homoglyphs, and double encoding.
→ Full methodology write-up on GitHub — observable impact, pattern catalogs, when each protocol fires.
What We Missed
We found every flag and triggered every XSS alert — 16 out of 16 challenges solved.
A perfect score does not mean the testing was effortless. The five Hard challenges each required sustained, multi-step exploitation efforts. EV-11 (Chatwoot DOM XSS) was the most demanding: the encoding boundary exploit required understanding the interaction between five processing stages — markdown rendering, HTML entity preservation, DOM parsing, innerText extraction, and innerHTML re-injection — and constructing a payload that survived all five while being active at the final one. Conventional XSS payloads, even from comprehensive lists, do not contain this class of exploit because the vulnerability depends on parser-specific behavior unique to this application.
EV-14 (Lucee three-stage RCE) required identifying three independent issues and recognizing that only their composition achieves code execution. The first two stages — unauthenticated file write and web mapping creation — were each dead ends in isolation. The exploitation chain only emerged after mapping the relationships between the temp directory, the web mapping mechanism, and the CFML execution model.
It is also worth noting: a perfect score on HackBench means the benchmark was within our current capability range. The 16 challenges, while grounded in real-world software, represent a fixed set of known vulnerability classes. Real-world applications present challenges that fixed benchmarks do not fully capture: WAF rules trained on real traffic, rate limiting that punishes automated probing, custom frameworks with proprietary internals, and multi-service architectures where the exploitation path spans multiple applications. A benchmark is a necessary validation, not a sufficient one.
We encourage benchmark authors to expand HackBench with additional challenges — particularly in vulnerability classes that are underrepresented in the current set: race conditions, deserialization chains, and protocol-level attacks (HTTP smuggling, cache poisoning). These are the classes that most consistently challenge automated testing tools.
Where This Places HackBench — Cross-Benchmark Context
HackBench is part of a benchmark program that now spans five independent evaluations — each testing a different dimension of automated security testing:
| Benchmark | Score | Challenges | Categories | What It Tests |
|---|---|---|---|---|
| XBOW CTF | 104 / 104 (100%) | 104 | 18 | Multi-category exploitation, black-box |
| PortSwigger Academy | 270 / 270 (100%) | 270 | 31 | Breadth across vulnerability categories |
| Vibe-Coding Benchmark | 74/74 + 78 extra | 74 + 78 | Full-stack apps | Real-world coverage beyond the answer key |
| HackBench | 16 / 16 (100%) | 16 | 7 | Real-world CVE exploitation chains |
| HackMerlin | 7 / 7 (100%) | 7 | LLM defense bypass | Progressive LLM defense bypass |
| Combined | 471/471 + 78 extra | 471 + 78 | 50+ | — |
HackBench's unique contribution to this picture is realism. While XBOW and PortSwigger test breadth across vulnerability categories with purpose-built challenges, HackBench tests depth against real-world software — WordPress, Chatwoot, XWiki, Lucee, CKEditor. The challenges are not designed to be solvable; they are real applications with real vulnerabilities. The five Hard challenges each require reasoning about production software internals, and the multi-step exploitation chains mirror what an actual engagement looks like.
Methodology and Proof
Testing conditions:
- Black-box: no answer keys consulted, no
docker exec, no container inspection - Automated: no human intervention during testing
- Per-challenge isolation: fresh Docker container per challenge, independent pentest session, no cross-challenge state
- Flag-based verification: 13 challenges verified by cryptographic flag extraction (
ev{32hex}format, runtime-generated per deployment) - XSS verification: 3 challenges verified by
alert(document.domain)through Playwright browser automation with screenshot capture - Anti-cheat compliance: no target source code reads during testing, no answer key lookups, no internet searches for solutions
Scoring mechanism:
- Flag challenges: automated scorer matches captured flag against runtime-generated flag values (unique per Docker build)
- XSS challenges: automated scorer validates
alert(document.domain)trigger through Playwright browser automation with screenshot evidence
Testing infrastructure:
- RedPick automated testing platform
- Per-challenge automated invocation with Docker management and scoring
The complete public proof artifact — including sanitized findings, anti-cheat audit results, benchmark version verification, and integrity checksums — is available at github.com/redpick-ai/redpick-benchmark-walkthroughs/tree/main/hackbench.
We encourage independent verification. HackBench is open source at github.com/ElectrovoltSec/HackBench. Each challenge can be launched independently via Docker Compose.
What This Means for Real-World Testing
HackBench's value is not in the score — any benchmark can be solved given enough time and human intervention. Its value is in what the challenges test: the ability to reason about real-world software, identify specific software versions, search for known CVEs, understand multi-step exploitation chains, and construct working exploits against production-grade applications.
The five Hard challenges are particularly instructive:
- EV-04 (WordPress + LearnPress): Tests whether the agent checks infrastructure configuration AND known CVEs for identified software versions — both the hardcoded credentials path and the CVE-2024-6589 SQLi path lead to the flag.
- EV-11 (Chatwoot): Tests understanding of encoding boundaries across multi-stage processing pipelines — a vulnerability class that no scanner can detect because it requires reasoning about parser interactions.
- EV-12 (CKEditor): Tests N-day patch reversing — given a security fix, construct the exploit for the unpatched version. A core penetration testing skill.
- EV-14 (Lucee): Tests multi-stage chain construction — combining three individually low-impact issues into a critical RCE chain.
- EV-16 (XWiki): Tests recognition of setup-phase vulnerabilities — fresh installations with uncompleted wizards that grant admin rights to the first user.
These are not theoretical exercises. They are the exact patterns that appear in real-world penetration tests. An agent that can solve all five — automated, with no human guidance — demonstrates capability that directly translates to production security testing.
For organizations evaluating automated penetration testing: HackBench tests the dimension that most benchmarks do not — whether the agent can handle the messy reality of real software with real CVEs, rather than clean purpose-built challenges. A perfect score on HackBench, combined with 100% on XBOW (104/104), PortSwigger (270/270), HackMerlin (7/7), and Vibe-Coding (74/74 + 78 extra), covers the full spectrum from synthetic challenges to real-world applications. See all benchmark results for the full picture.
Ready to see what RedPick finds?
If you are evaluating automated security testing solutions, the techniques validated on HackBench — CVE exploitation chains, encoding boundary analysis, N-day patch reversing, multi-stage RCE construction — are the same techniques RedPick applies in production engagements.
Request a demo to run a proof-of-concept assessment against your application, or explore RedPick's managed pentest where senior pentesters deliver complete engagements with the same engine that produced this benchmark run.