The Department of Education Just Set the Bar for Classroom AI. Most Tools Have Never Been Asked to Clear It.
On August 20, 2026, the U.S. Department of Education sent every state and school district a Dear Colleague Letter on classroom technology. It was signed by Kirsten Baesler, Assistant Secretary for the Office of Elementary and Secondary Education. It did not ban a single AI tool. It did something quieter and, for most districts, harder: it made the invisible question visible. Does this tool actually work?
The letter reduces every piece of classroom technology, AI included, to five questions. If a superintendent, a principal, or a vendor cannot answer all five, the guidance says the tool has not earned its place in the classroom yet.
What are the Department of Education’s five questions for ed tech?
Here they are, as close to plain language as the guidance allows:
- What learning problem does it solve? Not what it does. What problem it fixes.
- When should it be used? A tool is not a default. It has a moment.
- For whom should it be used? Not every student needs the same intervention.
- For how long should it be used? Duration is a design choice, not an accident.
- What evidence demonstrates that it improves student learning? This is the one most tools cannot answer yet.
Notice what is missing from that list: engagement scores, screen-time minutes, teacher enthusiasm, how many licenses got activated. None of that counts as evidence under the new guidance. Learning gains do.
The real message: use is not the same as improvement
The letter draws a line most districts have blurred for two years. It states plainly that recreational technology and education technology should not be viewed as synonymous. A tool that keeps students occupied is not the same as a tool that teaches them something.
That distinction sounds obvious on paper. It is not obvious in a budget meeting. When a vendor demo gets applause, when a pilot group says the tool is “fun,” when adoption numbers climb, it is easy to mistake usage for outcome. The guidance’s real contribution is forcing that mistake into the open. The question was never whether teachers and students use the technology. The question is whether the technology improves learning. Those are two different claims, and only one of them has ever been measured in most districts.
The secretary who praised AI also admitted the evidence gap
This is where the guidance gets interesting, and where the tension The Spotlight Machine keeps returning to shows up again: enthusiasm and evidence rarely arrive together.
Education Secretary Linda McMahon spoke favorably about AI in classrooms around the same guidance, describing its promise this way: “The beauty of that is that a student almost has a one-on-one tutor.” She was not wrong about the promise. Adaptive, individualized instruction is a real and valuable use of AI. But she also acknowledged there is little evidence yet showing how the technology actually affects student learning.
Read those two statements side by side. The country’s top education official can see the appeal of AI tutoring and admit, in the same breath, that nobody has proven it works. That gap between what leaders feel and what leaders can show is exactly what the five-question framework was built to close. It does not ask districts to stop believing in AI. It asks them to stop treating belief as evidence.
What happens when a tool fails the test?
The guidance does not stop at evaluation. It sets an expectation for what comes after. When evidence shows a tool is not improving learning, districts are expected to change course. When repeated findings confirm the shortcoming persists, the tool should be removed.
That is a harder standard than most procurement processes were built for. Most districts have a process for bringing a tool in. Very few have a process for taking one back out once the invoice is signed and the staff are trained on it. The guidance treats an AI tool the way a good manager treats a hire: on trial, reviewed against outcomes, kept only if it earns its place.
How should a district actually answer these five questions?
Start by refusing to let the vendor answer for you. A sales deck is not evidence. Ask for independent evaluation data, not testimonials, and treat “our customers love it” as a marketing claim, not a learning claim.
Assign a single owner for each tool who is responsible for tracking the five answers over time, not just at purchase. Evidence collected once at rollout goes stale. A tool that showed promise in September needs a fresh answer by spring.
Timebox every pilot with a real end date and a real decision point. If the evidence is not there by that date, the default should be to stop, not to renew out of habit.
And build the muscle for reading evidence before you need it. This is where most of the gap actually lives. Boards can mandate the five questions. Central office can build the tracker. But someone in the building has to know how to look at a learning outcome and tell the difference between a real signal and noise. That is not a policy problem. It is a training problem, and it is the same tension every rollout runs into: the tools change fast, the people making the calls need a way to keep up that does not depend on trusting the sales pitch.
This is the method Aziz Aghayev teaches when he works with school leadership teams: give people a repeatable way to ask the right question of a new tool, under pressure, without a data science degree. The TADA Framework, developed at flowlyst where Aziz is CEO, exists for exactly this moment, when a district has a mandate to evaluate but no shared language for doing it consistently across every building.
The Department of Education just handed every school system in the country a five-question test. Most tools have never been asked to answer it. Most staff have never been trained to ask it either.
If your district needs a way to turn that federal guidance into something your team can actually run, not just file away, The Spotlight Machine’s training programs are built for that exact handoff, from policy on paper to a habit in the building.