AI Can Build the Test. It Can’t Prove the Test Works.

AI-generated assessments can be built in seconds. A unit quiz, a reading check, a “growth” survey with a clean chart at the end. It looks finished. It looks official. It looks like evidence.

That is exactly the problem. Looking like evidence and being evidence are not the same thing, and this week two pieces in Education Week made that gap impossible to ignore, one about the tests we create and one about the research we lean on.

Different tools. Same leadership work. Someone still has to ask whether the thing actually measures what we say it measures.

What’s happening

On November 2, three assessment researchers, Christina Cipriano of UMass Amherst, Corinne Huggins-Manley of the University of Florida and Max Lu of the Harvard Graduate School of Education, published an opinion piece in Education Week titled You Can Use AI to Create Assessments in a Snap. You Shouldn’t. They wrote it as part of a National Council on Measurement in Education taskforce studying validity, fairness and quality in AI-integrated assessments.

Their argument comes down to five cautions. Scores are not automatically correct, so humans should independently score a sample of AI-scored work and check whether they agree. AI can carry bias from its training data; the authors write that AI platforms can, and currently do, systematically misjudge scoring for multilingual learners and students with disabilities. AI can produce convincing growth charts from data that meet none of the conditions for measuring growth. AI summaries and dashboards add measurement error you cannot see, and educators who hand off interpretation weaken their own data skills. And schools often do not need AI to build an assessment at all; good, vetted instruments already exist. They also note that the Standards for Educational and Psychological Testing are being revised specifically to address AI-integrated assessment.

One line from their piece stuck with me: “If an AI-generated artifact only looks like a valid assessment, your educational decisions will be based on incorrect data.”

The same day, Education Week’s Stephen Sawchuk reported that the research on AI in education is a mess. Patrick O’Neill, an associate professor at Ivy Tech Community College, audited 14 meta-analyses, drawn from nearly 40, that all concluded AI had positive effects on learning. He found that all 14 failed basic standards. They pooled very different populations and outcomes, often did not define what kind of AI use they studied, and in some cases included studies that did not use AI at all. Those papers have been cited more than 3,000 times, according to Google Scholar, and one has since been retracted. The article also notes that a Stanford review of more than 800 studies found just 20 with a strong cause-and-effect design, and not one involved K-12 students.

Meanwhile, use keeps climbing. Bett’s State of AI in Education 2026 report, a YouGov survey of 1,033 UK teachers released this week, found that eight in ten teachers use AI for teaching tasks and 76% use it to create lesson plans, worksheets and resources. Yet 41% have had no AI training from their school, and almost half say their school has no written policy on staff use. That is a UK sample, but the pattern will sound familiar to a lot of US leaders: the tools arrived faster than the shared understanding of how to judge what they produce.

Why AI-generated assessments are a judgment problem

When I read these pieces together, I don’t see a story about bad technology. I see a story about confidence.

AI is very good at producing things that carry the look of authority. A tidy rubric. A score to two decimal places. A chart that slopes up and to the right. A meta-analysis with thousands of citations. None of that tells you whether the underlying measurement is sound. The researchers put it plainly: the AI will say it can do the job, and it will be very confident about that.

In Chapter 7 of Human Still Required, I argue that information has scaled but responsibility has not. The tool can generate the test, score it and summarize it. It cannot be accountable for what happens to the student whose score was wrong.

Confidence is not judgment.

Human Still Required

That line matters most for the students the researchers named. If a scoring tool systematically misreads a multilingual learner’s writing or a student with a disability, the error does not show up as a red flag. It shows up as a lower number. And a lower number, once it sits in a dashboard, starts making decisions: placements, interventions, conversations with families. Data informs. Humans decide. But only if humans are actually looking.

The research story is the same lesson one level up. Many leaders have been told “the research shows AI improves learning.” Now we know a good share of that evidence does not hold up. That doesn’t mean AI can’t help. It means the question “Is this working for our students?” can’t be answered by a citation count or a vendor slide.

This is what I mean in Chapter 10 when I talk about the shift from expertise to discernment. Leaders no longer win by knowing the most. They win by knowing what to trust and what to check.

Discernment is the ability to decide what matters when everything looks important.

Human Still Required

There’s also a quieter warning in the assessment piece that deserves attention. When educators hand off interpretation of data to AI summaries, they lose their own interpretation skills. That is the atrophy risk I write about in Chapter 5, showing up in the data room instead of the classroom. If adults stop reading the evidence themselves, we should not be surprised when the system loses the ability to tell good evidence from bad.

What leaders can do now

• Ask the validity question out loud. Before any AI-generated assessment informs a grade, placement or intervention, ask: what is this supposed to measure, and how do we know it does? If no one can answer, it is a draft, not a measure.
• Build a human-check routine. Following the researchers’ advice, have teachers independently score a sample of AI-scored work and compare. Make this a normal part of PLC time, not an audit.
• Look at subgroups first. Check AI scoring results for multilingual learners and students with disabilities before anyone else. When you talk to vendors, ask whether they report accuracy and efficacy by subgroup.
• Treat research claims as questions. When a tool comes with “proven to raise achievement,” ask which study, with which students, measuring what. Notice how rarely the answer involves K-12 students.
• Train judgment, not just tool use. If many teachers are building materials with AI and no one has talked with them about assessment quality, that is the professional learning priority. Practice reading data together so the skill stays human.

If your leadership team wants to work through these questions together, the 5-Week Book Study is built for exactly this kind of conversation.

Frequently asked questions

Should teachers use AI to create assessments?

AI can help draft questions, but a draft is not a valid assessment. Assessment researchers writing in Education Week caution that AI-generated tests can look valid without measuring what they claim. Use AI to get started, then have educators check alignment, reading level and whether results match their own professional read of student work.

Is AI grading accurate for English learners and students with disabilities?

Not reliably, according to the researchers on a National Council on Measurement in Education taskforce, who warn that AI platforms can systematically misjudge scoring for these students. Leaders should require human scoring checks and ask vendors for accuracy data broken out by subgroup.

Does research show AI improves student learning?

The evidence is much weaker than it appears. An audit reported by Education Week found that 14 widely cited meta-analyses claiming positive effects all failed basic standards, and a Stanford review found no strong cause-and-effect studies involving K-12 students. Leaders should judge tools by their own local evidence, carefully gathered.

The test still needs a human

None of this is an argument against AI. I am not anti-AI; I am pro-thinking. It’s an argument against letting something that looks like evidence stand in for evidence.

AI can build the test. It can score the test. It can draw the chart. Deciding whether any of it is true, and what it means for a real child, is still our work.

So here’s my question for you: what is one number in your school right now that everyone trusts, and when did someone last check whether it deserves that trust?

Sources

• You Can Use AI to Create Assessments in a Snap. You Shouldn’t (Opinion) — Education Week, November 2, 2026
• The Research on AI in Education Is a Mess — Education Week, November 2, 2026
• The state of AI in education 2026: Rising teacher use is not always translating into shorter working hours — EdTech Innovation Hub (Bett / YouGov report), November 5, 2026