ZVEB Benchmark
ZVEB (Zambo Verifiable Execution Benchmark) is the benchmark where every score has a receipt: each benchmark task run produces a live, independently verifiable execution receipt, so any reported score can be checked against its own evidence.
20 planned, 16 answered, 16/16 receipts, 1,239ms median.
Scores are reported as AER-1 execution receipts, defined at /aer-1/. See ZVEB: The Benchmark Where Every Score Has a Receipt for the benchmark context.
Learn the evidence model first: AI agent execution receipt - definition, anatomy, and verification →
Pilot total: 9.38/10 - computed with supplement re-runs on 3 of 8 tasks.
Strict first-attempt total (no re-runs): 8.75/10.
Three tasks (catalog-qrcode, credits-ai, universal-multistep) were re-run with corrected arguments; the original attempts are kept in the transcript and every attempt has its own live receipt.
Leaderboard
| Agent | Affiliation | Run date | Tasks | Total |
|---|---|---|---|---|
| rambo | zambo | 2026-09-21 | 8 | 9.38/10 |
How to cite this page: Brennan Zambo. ZVEB Benchmark. zambo.dev. https://zambo.dev/benchmark/ . ZVEB (Zambo Verifiable Execution Benchmark) is the benchmark where every score has a receipt: each benchmark task run produces a live, independently verifiable execution receipt.
Task suite
| Task | Category | Tool | What the agent must do | Score | Receipt(s) |
|---|---|---|---|---|---|
| price-btc | Data lookup | live_price | Report the current BTC price in USD. | 10.0 | 68dc03ce |
| price-eth | Data lookup | live_price | Report the current ETH price in USD. | 10.0 | a8f624c9 |
| catalog-qrcode | Catalog search | capability_search | Find Zambo tools for QR codes using the qr code query. | 10.0 | original 39bd0aae supplement 3f65f766 |
| shield-injection | Safety | prompt_shield | Analyze an instruction asking to reveal the system prompt for injection. | 10.0 | c009f1f6 |
| leads-austin-plumber | Lead/data | leadsignal | Find plumber leads in Austin, Texas. | 5.0 | 41117f62 |
| credits-ai | Data index | credithunt | List AI startup credit programs with provider or URL details. | 10.0 | original attempt retained in transcript supplement 2d8a200d |
| audit-example | Audit (async) | ghost_audit_site + status + report | Run a staged audit on a public Zambo page to completion. | 10.0 | kickoff 956f0a08 status eff4193b report 0aa8f236 |
| universal-multistep | Multi-step routing | zambo_universal | Get the current SOL price and find the Zambo QR-code tool in one goal. | 10.0 | supplement 801b0911 original record |
Methodology
ZVEB pilot run v0 used 8 tasks on 2026-09-21. Each task required a different agent capability: report live BTC or ETH prices, find the QR-code tool, analyze a prompt injection, find Austin plumber leads, list AI startup credit programs, complete an asynchronous site audit, or combine a live SOL price with tool discovery. Each task received a score from 0 to 10 based on the task result and its receipt evidence.
The strict first-attempt total was 8.75/10 with no re-runs. A disclosed supplement re-ran 3 of 8 tasks: catalog-qrcode, credits-ai, and universal-multistep. The corrected arguments were used only for those supplement records, and the original attempts remain in the transcript. Every attempt has its own live receipt, so the pilot total of 9.38/10 can be checked separately from the strict first-attempt result.
External submissions are not open yet. When intake opens, each submission must include the agent name, transcript, task results, receipt IDs, and contact information so the run can be reviewed against the same evidence boundary.
ZVEB benchmark FAQ
What is the ZVEB benchmark?
The ZVEB benchmark is the Zambo Verifiable Execution Benchmark. It measures AI agent execution across exactly 8 real tasks, with every run backed by a verifiable execution receipt.
How is the ZVEB benchmark scored?
The ZVEB benchmark pilot score was 9.38/10, and the strict first-attempt score was 8.75/10. Supplement re-runs were disclosed on 3 of 8 tasks: catalog-qrcode, credits-ai, and universal-multistep.
What is a good ZVEB score?
A good ZVEB benchmark score should be read with its receipt evidence and rerun disclosure. The published pilot scored 9.38/10, while the strict first-attempt result was 8.75/10 across exactly 8 tasks.
Run it yourself
Run the price-btc task with plain JSON-RPC. No authentication is needed for this public example.
{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "live_price", "arguments": {"symbol": "BTC"}}}
curl -s -X POST https://zambo.dev/mcp -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"live_price","arguments":{"symbol":"BTC"}}}'
External submissions
Submission details are to be announced. When intake opens, an external agent will provide its transcript, receipt IDs, agent name, and contact information.