We keep talking about AI literacy as if it were a technology skill. Which tools to use. Which buttons to press. How to write a prompt. This week, an essay from Stanford’s Sam Wineburg and researcher Nadav Ziv, drawing on their own classroom pilot data, pointed somewhere else. Students know how to operate the bots. What they don’t know is when to believe them.
That’s not a tech gap. It’s a thinking gap.
And thinking is something schools already know how to teach. The question for leaders is whether we’re giving teachers the time, practice and support to teach it in a world where the most confident voice in the room is now a chatbot.
Key takeaways
- AI literacy is mostly critical thinking: tracing where an answer comes from, asking follow-up questions and checking whether evidence supports a claim.
- In a pilot with 117 high schoolers, nearly half were “not sure” whether a detailed but false chatbot answer was true, and none of the students who said they would follow up actually did.
- Teachers in any subject can teach these habits, but only if their own professional learning is practice-based, school-based and ongoing.
- States are starting to require AI training for educators. Leaders decide whether that becomes tool training or real learning.
What’s happening
On October 7, Education Week published an opinion essay by Sam Wineburg, professor emeritus at Stanford and co-founder of the nonprofit Digital Inquiry Group, and researcher Nadav Ziv, titled “Think Your Students Know More About AI Than You Do? Read This”. Citing a nationally representative Common Sense Media survey, they note that seven in 10 teens rely on AI for schoolwork, but only 26% say they’ve been taught to judge whether a bot’s information is accurate. Their own pilot data, they write, show students don’t have a stronger grasp than teachers when it comes to evaluating what a chatbot tells them.
The most striking example comes from a pilot with 117 high school students. Students were shown a detailed but false chatbot answer, with no links, about the history of a Black neighborhood in Sacramento during the Gold Rush. About three in 10 said they trusted it. About two in 10 said they didn’t. Nearly half said they were “not sure.” Seventeen students said they would follow up and had internet access to do it. None did.
Wineburg and Ziv name four things students misunderstand: bots aren’t oracles, a first response is a first draft, the same question can produce opposite answers, and uncertainty can’t be the final word. Their fix isn’t a new app. It’s classic critical thinking that any teacher in any subject can model: trace where an answer comes from, ask follow-up questions, check the credibility of sources, and judge whether the evidence supports the conclusion. They point to studies finding that even a few hours of instruction substantially improved students’ digital savvy.
The same day, K-12 Dive’s Ed Finkel reported on what makes professional development stick. Elaine Allensworth of the University of Chicago Consortium on School Research, who studied professional learning tied to Chicago Public Schools’ math and literacy curriculum work, said, “Practices are often hard to change, but they seem to matter a lot, maybe more than the content.” She also said, “School-based professional learning seemed to be a critical component.” A more recent study she described found that online-only PD produced lower engagement and teachers found it less helpful. Lara Ohanian of TNTP added that generic, decontextualized PD that is treated as an add-on won’t last; what works is a cycle where teachers practice with their own students and collaborate around evidence of student learning.
And policy is moving. WKMG ClickOrlando reported on October 6 that Orange County Public Schools is updating its AI policies after the Florida State Board of Education adopted a revised AI rule on September 16. Among the 18 requirements: teachers and administrators must receive AI training, and district policies must, when appropriate, require students to show mastery without AI. Districts have until July 1, 2027.
Put those together and you get one story. Students need to learn to question AI. Teachers are the ones who will teach that. And training is about to be required whether we’re ready or not.
What this reveals about AI literacy
In Chapter 5 of Human Still Required, I argue that the real risk with AI isn’t cheating. It’s atrophy. When a tool hands you a fluent, finished answer, the temptation is to stop thinking. The Sacramento example shows exactly what that looks like in a classroom. Students weren’t fooled because they lacked access. They had the internet. They stopped because the answer felt complete.
That’s why I keep coming back to the idea of AI as a thinking scaffold, not a thinking replacement. Used well, a chatbot doesn’t end the inquiry. It starts it.
AI doesn’t answer the question. It keeps the question alive longer.
Human Still Required
Wineburg and Ziv’s advice to treat the first response as a first draft is that idea in practice. Ask again. Ask for sources. Ask the same question twice and compare. Every one of those moves keeps the question alive.
Chapter 7 adds the second half. Chatbots sound certain. One student in their pilot said the bot “seems to know what it’s talking about and is very certain.” That is the trap, for students and for adults.
Confidence is not judgment.
Human Still Required
Judgment is the human work of deciding what to believe and what to do next. A tool can sound sure. Only a person can be accountable for acting on it. If we want students to build that judgment, we have to teach it on purpose.
Which brings me to the adults. In Chapter 12, I argue that most AI professional learning has been built around tools, and that this produces exactly the wrong outcome.
Professional learning that centers on tools produces trained technicians. Schools need adaptive professionals.
Human Still Required
If AI literacy is really critical thinking, then a two-hour session on a district-approved chatbot won’t get us there. Teachers need to practice the moves themselves: run the same prompt twice, chase a claim back to its source, catch a confident error. Then they need to try it with their own students and talk with colleagues about what happened. That is what the Chicago research describes. Practice, in school, with feedback, over time.
Here’s the hopeful part. None of this requires teachers to become AI experts. History teachers have been teaching sourcing for decades. Science teachers teach students to weigh evidence. English teachers teach students to question an author’s claims. The skill isn’t new. The context is.
What leaders can do now
- Redefine AI literacy in your own documents. Look at your AI guidance and PD plan. If AI literacy is described mainly as tool use, add the thinking moves: verify, trace sources, ask follow-ups, compare answers, decide.
- Run the Sacramento test with your staff. Give a leadership team or department a confident, unsourced chatbot answer on a topic they know. Ask who would trust it, who wouldn’t, and who would check. Then talk about why so few people actually check.
- Build AI training as a cycle, not an event. If your state is about to require AI training, design it the way the research says PD sticks: school-based, practiced with real students, revisited in teams, with coaching and feedback.
- Make it every subject’s job. Ask each department to name one existing critical-thinking routine it already teaches and show how it applies to a chatbot answer. You’ll find most of the work is already in your building.
- Model it yourself. When you use AI in your own work, say so, and say how you checked it. Students and staff learn more from watching a leader verify than from reading a policy.
If your leadership team wants to work through these ideas together, the 5-Week Book Study is built for exactly that kind of conversation.
Frequently asked questions
What is AI literacy for K-12 students?
AI literacy is the ability to use AI tools well and judge what they produce. Operating a chatbot is the easy part. The harder and more important part is critical thinking: checking where an answer came from, asking follow-up questions, and deciding whether the evidence supports the claim.
Do teachers need to be AI experts to teach AI literacy?
No. The core skills are ones many teachers already teach: sourcing, weighing evidence and questioning claims. Wineburg and Ziv argue teachers in any subject can do this. What teachers need is time to practice applying those skills to chatbot answers and support from colleagues as they try it with students.
How should districts design required AI training for teachers?
Design it as ongoing, school-based learning rather than a one-time session. Research on professional development reported by K-12 Dive points to practice with real students, collaboration and feedback as what makes new practices last. Focus on thinking and judgment, not just tool features.
The work machines can’t do
Students are going to keep asking chatbots questions. That part is settled. What isn’t settled is whether they’ll learn to treat the answer as the start of their thinking or the end of it.
That’s a teaching decision. And behind it is a leadership decision about what kind of learning we give our teachers.
So here’s my question for you: in your school, who is teaching students to doubt a confident answer, and who is teaching the teachers?
Sources
- Think Your Students Know More About AI Than You Do? Read This (Opinion) — Sam Wineburg and Nadav Ziv, Education Week, October 7, 2026
- How can leaders ensure professional development sticks? — Ed Finkel, K-12 Dive, October 7, 2026
- Orange County schools face new requirements for classroom AI use — Tylisa Hampton, WKMG ClickOrlando, October 6, 2026