Same researcher, opposite results

If you want a glimpse into cutting-edge research on artificial intelligence in education, look to Jing Liu, director of the Center for Educational Data Science and Innovation at the University of Maryland. Two studies released in October offer a striking and enlightening contrast. One hinted that a little AI-powered math practice might help struggling middle schoolers improve their skills. The other found that an AI assistant undermined college students’ engagement and hurt their grades.

Liu wanted the studies to be released together, he told me, to show that AI in education is neither “all good” nor “all bad.” The field is still figuring out how to design tools that improve learning over time. And that means studying not just the technology, but also how students respond to it — including in ways its creators never intended.

In the first experiment, a math practice website called Goblins was rolled out to 6,000 middle schoolers in Deer Valley, Arizona, during the 2025-26 school year. Teachers assigned problems to their students. An AI tutor offered step-by-step help through text chatting, drawing on a virtual whiteboard and even spoken conversation. Teachers were encouraged to use it with struggling students, though some assigned it to their entire classes and others skipped it altogether.

Students who solved at least two problems a week — more than 50 over the school year — scored significantly higher on a year-end standardized assessment in math than similar students who used Goblins less or not at all. Accelerate, an independent, nonprofit research organization that commissioned the study, equated the statistical difference of 0.11 standard deviations to about three additional months of learning.

But hold off on the celebratory bot music. The study cannot prove that Goblins caused the gains. Students who used it more might simply have been more motivated to practice math and might well have tested better in math with or without Goblins. Accelerate hopes to commission a randomized controlled trial to settle the matter.

Still, one result from this study is worth highlighting: Half the 6,000 students used Goblins to tackle at least two problems a week, the dosage associated with higher math scores. 

That’s noteworthy because education technology is plagued by something that education researchers call the “five-percent problem.” A tool may work beautifully for students who use it as directed, but as a rule of thumb, only about 5 percent actually do. The other 95 percent either don’t log in at home or sit idly through the minutes of scheduled screentime in class. Studies on Khan Academy, IXL and other platforms have well documented this challenge of getting children to use their instructional software consistently.

Goblins appeared to reach some students who are often hard to engage, including English learners and students with disabilities. Liu suspects that’s because students could talk aloud to a cartoon avatar of their teacher, which could talk back. Moreover, the AI tutor could speak and understand Spanish and six other languages.

“It was like talking to your own teacher,” said Liu. “It made the students want to use it.”