The Score Was Confident. The Teachers Disagreed.

Automated scoring is supposed to make testing faster and more consistent. In Texas this year, it also produced something no one designed for: tens of thousands of educators reading their students’ essays, looking at the number the system assigned, and saying, “That’s not right.”

That isn’t a story about a broken machine. It’s a story about who still has to read the work.

Because a number can be precise, fast and confident and still need a human to check it.

What’s happening

On October 7, The Texas Tribune reported that STAAR rescoring requests from Texas schools tripled in one year. The Texas Education Agency began using an automated scoring engine in December 2023 to grade open-ended questions on the state test, with humans manually reviewing a quarter of exam scores. Requests to rescore open-ended reading responses went from 2,758 in spring 2024, to 21,620 last year, to 64,489 this spring. More than 250 districts and charter networks submitted requests.

The state raised more than 13,300 reading scores after review. Computers had assigned 58% of those original grades and humans 42%. All of the rescoring was done by people. Dallas ISD alone sent in more than 5,100 reading tests, and 34% came back higher; the district estimated the rescoring helped five campuses improve their state ratings.

To be fair to the state, the scale matters. More than 3.2 million students took the reading test, and fewer than half a percent of open-ended reading responses changed. A TEA spokesperson, Jake Kobersky, told the Tribune that scoring every essay by hand would take four to five times as many human scorers and cost an additional $15 million to $20 million a year. He described the approach this way: “TEA uses a hybrid model where human experts drive the scoring.”

Rachel White, an education policy professor at the University of Texas at Austin, read the trend differently. She told the Tribune the tripling suggests “educators who read their students’ essays are increasingly disagreeing with the scores that are assigned,” and that, machine or person, “there is a bit of a trust problem in how these are being scored.”

A second piece this week came at the same problem from the classroom side. In an October 13 opinion essay for The 74, Chris Minnich, CEO of MetaMetrics, the company behind the Lexile Framework, argued that AI-powered instruction needs far more precision than general chatbots provide. His company tested ChatGPT, Gemini, Claude and Copilot on 1,807 passages and found that only 3% of the Lexile measures the tools estimated matched the official measures. Teachers, he wrote, paste a passage into a chatbot and “receive a remarkably confident answer.” Minnich has an obvious stake in Lexile measurement, and leaders should read his piece with that in mind. But the pattern he describes is worth taking seriously on its own.

And on October 8, Chalkbeat’s Matt Barnum asked a question every parent and principal has wrestled with: how can you tell whether an elementary school is good? Looking at one Brooklyn school, he found that New York City’s growth measure rated it well below average, while the state’s growth measure rated it slightly above average. Same school. Two official numbers. Two different stories. Brown University researcher Peter Hull summed it up simply: “It’s a hard problem.”

Why automated scoring is a leadership question

Put those three stories side by side and you see one pattern. A system produces a number. The number looks authoritative. And the people closest to the work are the ones who notice when it doesn’t match what they see.

In Chapter 7 of Human Still Required, I write about the difference between information and judgment. Information has scaled. Responsibility has not. A scoring engine can process millions of essays. A chatbot can return a reading level in seconds. A growth model can rank every school in a city. None of them carries responsibility for what happens next to the student, the teacher or the school.

Confidence is not judgment.

Human Still Required

That line is about leaders, but it applies to tools too. The Lexile answer came back confident. The STAAR score came back official. The growth rating came back as a clean number. Confidence is what these systems are good at producing. Judgment is still our job.

This is also why I don’t read the Texas story as an argument to throw out automated scoring. Fewer than half a percent of responses changed. Hand-scoring everything has real costs in money and time. The machine can do a lot of the sorting. The question is what we build around it: who reads the work, who can question the number, and whether that review is a right every school knows about or a quiet process only some districts use. Notice that the Tribune reports more districts requested rescoring as superintendents talked to each other and the process got more attention. That means that, for a while, whether a score got a second human look depended partly on whether your district knew to ask.

Data informs. Humans decide.

Human Still Required

Chapter 6 makes a related point: AI didn’t create a learning crisis so much as reveal a measurement problem. Many of the measures we rely on were already stretched. Chapter 14 goes further and argues that grades were never meant to carry the weight we put on them. In Texas, a few points on a written response can feed into campus A-F ratings, and under state law persistent low ratings can lead to state intervention. When that much rides on a number, the human read isn’t a luxury. It’s the safeguard.

Context is lived, not labeled.

Human Still Required

The Chalkbeat story is the reminder that this isn’t only about AI. Two respected growth models disagreed about the same school. The answer isn’t a better dashboard. It’s someone walking the halls, talking with families and looking at student work. Barnum’s own next step was to report back from a visit to the school, including conversations with parents. That’s the right instinct.

What leaders can do now

• Read the work behind the number. Pick a sample of student responses from any automated or AI-assisted score your school relies on, and have teachers score them blind. Where your people and the system disagree, find out why before the score drives a decision.
• Know your review rights and use them. If your state or vendor offers rescoring or appeals, make sure your instructional leaders know how it works, what it costs and when the window closes. A safeguard nobody knows about isn’t a safeguard.
• Ask the precision question before adopting a tool. When a product or a chatbot gives a measure, such as a reading level, a rubric score or a risk flag, ask: compared to what, and how often does it match a trained human? If no one can answer, treat the output as a draft.
• Never let one number decide alone. Pair every high-stakes score with at least one human source of evidence: student work, a conference, a classroom visit. That’s the D.E.C.I.D.E.R. thinking from Chapter 6 applied to adults.
• Talk about it openly with staff. Teachers who disagree with a score should feel safe saying so. Their disagreement is data.

If your leadership team wants to work through questions like these together, the 5-Week Book Study was built for that kind of conversation.

Frequently asked questions

Is automated scoring the same as generative AI?

Not necessarily. Texas officials have stressed that their scoring engine is different from tools like ChatGPT or Google’s Gemini: it was trained on thousands of human-scored responses and does not “learn” from one response to the next. But the leadership question is the same either way: who checks the output, and who is accountable for it?

Should schools stop using AI to score student work?

Not automatically. Automated scoring can save time and handle volume that people can’t. The risk comes when a score is treated as final without a human review process, especially when it feeds into high-stakes decisions about students or schools. Use it to prepare the decision, not to make it.

How can a principal check whether an AI score is accurate?

Start small. Have two or three teachers score a sample of the same student work without seeing the machine’s score, then compare. Look for patterns in where they disagree. If the gaps are large or consistent, raise it with the vendor or state and adjust how much weight that score carries locally.

The human read is the point

The most hopeful part of the Texas story is not the numbers. It’s that thousands of educators read their students’ writing, trusted what they saw and spoke up.

That’s what we want. Use the machine for speed. Keep people responsible for meaning.

So here’s my question for you: in your school or district, which numbers are making decisions that no one has checked against the actual work?

Sources

• STAAR rescoring requests from Texas schools triple in one year, The Texas Tribune, October 7, 2026
• AI-Powered Instruction Needs Precision. Right Now, It’s Not Even Close (opinion, Chris Minnich), The 74, October 13, 2026
• How can parents tell if an elementary school is good? The data does not have clear answers., Chalkbeat, October 8, 2026