ResumeBench scores document APIs and vision LLMs on everyday, expert-built, handwritten and scanned resumes, every field and every bullet checked against human-corrected ground truth. 18 document APIs and vision LLMs, 300 resumes, 837 pages. By Careerflow.
| Rank | System | Type | F1 | Precision | Recall | Cost per page | Median time |
|---|---|---|---|---|---|---|---|
| 1 | LlamaExtract Agentic Plus (LlamaIndex) | Document API | 87.1 | 88.4 | 88.3 | 5.4¢ | 70 s |
| 2 | Reducto Deep (Reducto) | Document API | 85.4 | 87.4 | 87.1 | 4.0¢ | 65 s |
| 3 | LlamaExtract Agentic (LlamaIndex) | Document API | 85.4 | 86.6 | 87.2 | 2.3¢ | 55 s |
| 4 | Claude Opus 4.8 (Anthropic) | Vision LLM | 85.4 | 86.7 | 87.4 | 9.0¢ | 31 s |
| 5 | Gemini 3.5 Flash (Google) | Vision LLM | 85.4 | 86.8 | 87.8 | 3.5¢ | 43 s |
| 6 | Extend Max Context (Extend) | Document API | 84.6 | 86.5 | 86.2 | 5.0¢ | 43 s |
| 7 | GPT-5.6 Sol (OpenAI) | Vision LLM | 84.4 | 85.6 | 86.8 | 7.4¢ | 38 s |
| 8 | LlamaExtract Cost-Effective (LlamaIndex) | Document API | 83.4 | 84.9 | 85.2 | 0.82¢ | 29 s |
| 9 | GPT-5.5 (OpenAI) | Vision LLM | 83.1 | 84.5 | 85.6 | 8.7¢ | 48 s |
| 10 | Reducto Extract (Reducto) | Document API | 81.6 | 83.6 | 83.9 | 2.0¢ | 16 s |
| 11 | Extend Extract (Extend) | Document API | 80.8 | 82.8 | 83.1 | 1.2¢ | 31 s |
| 12 | Qwen 3.8 27B (Alibaba) | Vision LLM | 78.2 | 86.1 | 80.0 | 0.96¢ | 36 s |
| 13 | Qwen 3.5 35B (Alibaba) | Vision LLM | 70.9 | 80.0 | 76.5 | 0.39¢ | 78 s |
| 14 | Qwen 3.8 Flash (Alibaba) | Vision LLM | 64.4 | 87.8 | 66.2 | 0.30¢ | 122 s |
| 15 | GPT-5.4 Nano (OpenAI) | Vision LLM | 61.9 | 70.1 | 66.4 | 0.28¢ | 20 s |
| 16 | Gemma 4 26B (Google) | Vision LLM | 59.0 | 71.7 | 62.4 | 0.06¢ | 29 s |
| 17 | Qwen 3.5 9B (Alibaba) | Vision LLM | 58.6 | 83.5 | 61.9 | 0.13¢ | 138 s |
| 18 | GLM 5.3 Flash (Zhipu AI) | Vision LLM | 52.9 | 73.4 | 55.5 | 0.08¢ | 46 s |
Each predicted job, school, certificate or other entry is paired with a ground-truth entry; an entry counts as right only when every field and bullet in it matches the resume word for word. Precision, recall and F1 are computed per section, averaged over the sections of each resume, then over all resumes. No LLM judge is used. All systems receive the identical PDF, the same 14-section JSON schema and the same 144-line extraction prompt. Document APIs ingest the schema natively; vision LLMs receive the rendered pages with the prompt. No LLM judge anywhere: scoring is deterministic and reproducible.
On ResumeBench's 300 resumes, LlamaExtract Agentic Plus has the highest F1 (87.1), followed by Reducto Deep (85.4) and LlamaExtract Agentic (85.4). The lowest of the 18 systems is GLM 5.3 Flash (52.9).
ResumeBench is an open benchmark from Careerflow that measures how accurately document extraction APIs and vision LLMs turn resumes into structured JSON. Every system gets the same PDF, the same 14-section schema and the same instructions, and every field and bullet is checked against human-corrected ground truth.
Each predicted job, school, certificate or other entry is paired with a ground-truth entry; an entry counts as right only when every field and bullet in it matches the resume word for word. Precision, recall and F1 are computed per section, averaged over the sections of each resume, then over all resumes. No LLM judge is used.
LlamaExtract Agentic is within 2 F1 points of the top system at 2.3¢ per page, against 5.4¢ for LlamaExtract Agentic Plus.
Among systems within 3 F1 points of the top, Claude Opus 4.8 is fastest at a median of 31 seconds per resume.
Handwritten and image-only resumes cost parsers the most, followed by rotated or warped scans, hidden text and prompt injections, custom sections the schema has no slot for, long bullet lists and entries split across pages.
300 resumes (837 pages) from fields including Law & public sector, Data & AI, Software & IT, Healthcare & science, Marketing, media & design, Engineering & trades and more: everyday resumes, expert-built Canva and LaTeX designs, resume-builder templates, handwritten scans, scanned and rotated pages and layouts with built-in distractors. 20 are published with their pages and ground truth; the full set is available from Careerflow on request.