How often the AI grade matches College Board scores
Studiorum grades College Board's published, officially scored AP® sample answers with the same AI grader teachers use, then compares the two scores. Each result comes from an exam year set aside to check the grader rather than to build it.
352
Held-out answers
Official sample answers in 11 AP® subject areas, from exam years set aside to check the grader.
Gemini 3.8 Flash
Grading model
The model that drafts AI grades for teachers to review, with the same settings. An AI grade stays a draft until a teacher approves it.
October 12–13, 2026 and November 1, 2026
Measured
Each row shows the model and the date of its own measurement.
Agreement with the official score
Exact means the AI's score equals College Board's score. Within 1 point means the two scores are no more than 1 point apart. When an answer was graded more than once, every grading counts.
Whole-question scores
The AI's score for the whole question, compared with College Board's score for the same answer.
| Task | Answers | Exact | Within 1 point |
|---|---|---|---|
| AP® United States History, AP® World History: Modern, and AP® European History | |||
| Long essay question (LEQ) (also read while writing the rules) | 54 | 56% | 89% |
| 6 points · 2024 exam · Gemini 3.8 Flash · November 1, 2026 | |||
| Document-based question (DBQ) (fewer than 20 answers) (also read while writing the rules) | 18 | 44% | 100% |
| 7 points · 2024 exam · Gemini 3.8 Flash · November 1, 2026The grader reads each document as text, up to 1,500 characters; an image document arrives as a written description. | |||
| AP® United States Government and Politics | |||
| Concept application (fewer than 20 answers) (quarter-point partial credit) | 6 | 33% | 100% |
| 3 points · 2025 exam · Gemini 3.8 Flash · November 1, 2026 | |||
| Quantitative analysis (fewer than 20 answers) (quarter-point partial credit) | 6 | 50% | 100% |
| 4 points · 2025 exam · Gemini 3.8 Flash · November 1, 2026 | |||
| AP® African American Studies | |||
| Short-answer questions (fewer than 20 answers) (quarter-point partial credit) (also read while writing the rules) | 9 | 17% | 100% |
| 3–4 points · 2025 exam · Gemini 3.8 Flash · November 1, 2026For the question with an image source, the grader sees only the image's title, as it does in Studiorum. | |||
| AP® English Language and AP® English Literature | |||
| Essays (all six question types) (also read while writing the rules) | 72 | 56% | 93% |
| 6 points · 2023–2024 exams · Gemini 3.8 Flash · October 12, 2026Half of these essays were handwritten and transcribed by AI before grading. | |||
| AP® Human Geography | |||
| Free-response question with a stimulus (Q2–Q3) (fewer than 20 answers) (also read while writing the rules) | 12 | 88% | 100% |
| 7 points · 2024 exam · Gemini 3.8 Flash · October 13, 2026The grader reads only the text of a map or image. | |||
| Free-response question without a stimulus (Q1) (fewer than 20 answers) (also read while writing the rules) | 6 | 33% | 83% |
| 7 points · 2025 exam · Gemini 3.8 Flash · October 13, 2026 | |||
| AP® Latin | |||
| Translation (fewer than 20 answers) (also read while writing the rules) | 6 | 50% | 83% |
| 15 points · 2025 exam · Gemini 3.8 Flash · October 12–13, 2026 | |||
| AP® Computer Science A | |||
| Free-response questions (fewer than 20 answers) | 10 | 100% | 100% |
| 9 points · 2025 exam · Gemini 3.8 Flash · October 12–13, 2026The 2026 questions are worth 5 to 7 points. The grader had the official solution and scoring criteria, as questions Studiorum writes do. | |||
Part of each question
For these tasks only part of each question can be compared, or College Board scores each rubric row or point on its own. Exact counts matches on what each row lists under Compared. Within 1 point compares totals for the same answers.
| Task | Answers | Exact | Within 1 point |
|---|---|---|---|
| AP® African American Studies | |||
| Short-answer questions, pilot exam (fewer than 20 answers) (quarter-point partial credit) (also read while writing the rules) | 12 | 73% | Not compared |
| 4 points · 2024 exam · Gemini 3.8 Flash · November 1, 2026Compared: each part, on the pilot exam's rubric (its totals don't follow the current rubric). | |||
| AP® Biology | |||
| Long free-response question, with a scoring key (fewer than 20 answers) | 12 | 96% | 97% |
| 9 points · 2023–2024 exams · Gemini 3.8 Flash · October 12, 2026Compared: each part, leaving out the graphing part.Handwritten answers, transcribed by AI before grading. | |||
| Short free-response question, with a scoring key | 24 | 92% | 100% |
| 4 points · 2023–2024 exams · Gemini 3.8 Flash · October 12, 2026Compared: each part. | |||
| Short free-response question, no scoring key | 24 | 84% | 100% |
| 4 points · 2023–2024 exams · Gemini 3.8 Flash · October 12, 2026Compared: each part. | |||
| AP® Chemistry | |||
| Free-response questions, with a scoring key | 21 | 100% | 90% |
| 4 or 10 points · 2023 exam · Gemini 3.8 Flash · October 12–13, 2026Compared: each explanation or justification point; calculation and drawing points count only toward Within 1. | |||
| Free-response questions, no scoring key | 21 | 97% | 90% |
| 4 or 10 points · 2023 exam · Gemini 3.8 Flash · October 12–13, 2026Compared: each explanation or justification point; calculation and drawing points count only toward Within 1. | |||
| AP® Macroeconomics and AP® Microeconomics | |||
| Free-response questions (also read while writing the rules) | 72 | 88% | 100% |
| 5 or 10 points · 2024–2025 exams · Gemini 3.8 Flash · October 12, 2026Compared: the written rows together; drawn-graph rows are left out. | |||
| AP® Art History | |||
| Short essays (Q3–Q6) (fewer than 20 answers) | 12 | 88% | 100% |
| 5 points · 2023 exam · Gemini 3.8 Flash · October 12, 2026Compared: each rubric row.Text only: the grader got the artwork's name and a short description, not the image. | |||
Fewer than 20 answers. One answer moves the percentage by 5 points or more.
Studiorum gives partial credit on short-answer parts in quarter points; College Board scores whole points. A partly credited answer can be close without matching exactly.
Some of this year's answers were also read while the grading rules were written, so the result may run high.
How we measure
Official samples
For each free-response question, College Board publishes a few real student answers with their official scores and the readers' reasoning. The samples are chosen to show the range of scores, so they aren't a random set of students.
The grader teachers use
Each sample goes through the same grading code, model (Gemini 3.8 Flash), and settings as a teacher's AI grade. Feedback on students' own practice runs on a different model and isn't part of these results.
Held-out years
Each grader's rules were written while reading one or two exam years. A different year was set aside to check each rule change, and a change stayed only if it didn't hurt that year. That makes these results somewhat optimistic, and rows marked § more so.
The 2026 rubric
The grader always applies the current College Board rubric. Where a rubric row changed after the sample's exam year, only the unchanged rows are compared.
Text answers
Handwritten samples from older exams were transcribed by AI before grading. Parts that need a drawing, a graph, handwritten math, or audio are left out.
Repeat gradings
Grading the same answer twice changes its score about 1 time in 5. Treat differences of a few points between rows as noise.
What these numbers don't cover
AP® tasks not on this page
- AP® U.S., World, and European History short-answer questions. Every exam year with scored samples (2023–2025) was read while the grading rules were written, so none is held out.
- AP® U.S. Government SCOTUS comparison and argument essay. Their samples were read while rules were tested or written.
- AP® Comparative Government and Politics. The 2023–2024 samples were used for tuning, and some 2025 samples were read before the rules were final.
- AP® Psychology, AP® Environmental Science, and AP® Physics. Every exam year in the current format was used for tuning, apart from a few AP® Physics C answers per question type: too few to report, and also read during tuning.
- AP® Seminar, AP® Spanish Literature, and AP® World Language email replies. Studiorum shows these on a different point scale than College Board uses, so the score a teacher sees can't be compared directly.
- AP® Research. College Board scores the whole academic paper, while Studiorum grades it one section at a time, so the comparison is indirect.
- AP® Latin short answers and AP® Art History comparison question. Latin short answers are laid out differently in Studiorum, so samples had to be matched by hand. The Art History comparison question has only 3 set-aside samples, and they shaped one of its rules.
- AP® African American Studies document-based question. It is graded with the AP® History DBQ grader, whose evidence thresholds differ. An October 2026 check on 6 samples found it scored about 1 point too low on average.
- AP® Statistics and AP® Computer Science Principles written responses. College Board changed how these are scored for 2026, so older samples can only be compared against a retired scale.
- AP® World Language argumentative essays, Chinese and Japanese email responses, and AP® Latin essays. The current rubrics use new point scales, so older samples can be ranked against each other but not scored the same way.
- Handwritten math, drawings, graphs, audio, and artwork. AP® Calculus, Precalculus, Music Theory, and Art and Design, drawn-graph points in Economics and Biology, calculation and drawing points in Chemistry, and World Language speaking tasks need more than typed text, which this check doesn't score.
- AP® Business with Personal Finance and AP® Cybersecurity. College Board hasn't published scored samples for these new courses yet.
Changes since a row was measured
- October 13, 2026: score fields began accepting only the point values a rubric row allows. No scoring rule changed. Most rows dated October 12–13 were measured just before this change; the AP® History essay rows, re-measured after it, didn't move.
Everything else
- Non-AP® grading. Nothing outside AP® has a comparison with human graders. There, the teacher's approval of each grade is the safeguard, and teachers can steer the AI with instructions for a question or rules in their profile.
- Your students' answers. The official samples are picked to illustrate scores, not to represent a class. Agreement on your students' work can differ.
- Fairness across groups. A pilot on 30 AP® History answers in October 2026 found no score penalty for features of English-learner writing that the rubric doesn't grade. A full run across subjects isn't scheduled yet.
When we measure again
Every change of grading model is checked against a sample of these answers. A change of model family brings a full re-measure, and so does each new year. Every row names its model and the date it was measured.
When College Board posts scored samples from the 2026 exams, they become the next held-out set, and the first that no grading rule was written from.