Artificial intelligence is reshaping educational methodologies, though a recent study indicates that this technology might be most effective when it encourages students to slow down rather than speed through their learning tasks.
A comprehensive experiment involving over 6,000 middle school students in Tennessee revealed that math proficiency improved slightly when AI tutors helped students address their mistakes. The approach required them to demonstrate mastery by answering the same question correctly three consecutive times before advancing.
Researchers examined four instructional methods for practicing fractions, comparing conventional techniques against those augmented by AI tutoring. Within this framework, half of the students in both groups needed to correctly respond to practice questions three times following any errors. This framework emphasized the importance of solidifying understanding through repetition.
Participants utilized software akin to the interactive videos and exercises offered by Khan Academy, dedicating 50 minutes in math class to this learning experience. A following test measured retention one week later, revealing that the combination of AI-tutored sessions and repeated practice led to an increase of approximately three percentage points in scores compared to traditional computer-based instruction. Although this improvement was modest, it stood out as a notable finding.
Philip Oreopoulos, the study's lead author and an economist at the University of Toronto, expresses a cautious optimism: "I don’t want to jump out and say we’ve demonstrated that AI is going to be the solution that we hope it is. But it might be the first kind of evidence that shows there’s at least some hints that it has some positive value against no AI at all."
This insight is particularly relevant considering the growing concerns surrounding AI's negative impact on student learning, such as hastily providing solutions which could undermine comprehension.
The Role of AI in Learning
The study titled “Making AI Tutoring Productive: Evidence from a Mastery-Based Math Practice Experiment” was a collaboration between researchers from the University of Toronto and the Wharton School at the University of Pennsylvania. The findings are set to be presented in a working paper by the National Bureau of Economic Research soon.
A key observation from the study is that AI can effectively walk students through errors, allowing them to understand the underlying concepts instead of simply presenting a final answer. Without this guidance, students can easily overlook their mistakes and move on without grasping the material.
Engagement and Understanding Through Practice
Using the AI tutor, named Numi, students received tailored feedback that enabled them to reengage with their incorrect responses. This interactive approach resulted in students from the AI-enhanced “mastery learning” group spending more time contemplating individual questions, indicating a deeper engagement with the subject matter. Notably, these students were also more successful in answering subsequent questions correctly after previous errors.
It’s essential to recognize, however, that merely answering three questions correctly doesn’t guarantee a thorough mastery of the skill at hand. Some students may guess their way to the correct answer or rely on repetitive exposure without genuinely understanding the math concepts.
Despite the advantages seen in AI-supported learning, the benefits had clear boundaries. Students in the AI-plus-mastery group showed better performance mainly on simpler fraction problems that closely mirrored their practice questions. Their performance did not necessarily translate into improved capabilities with more challenging difficulties.
The study's short duration of only a 50-minute intervention raises questions about the long-term effectiveness of combining AI with mastery learning strategies. Oreopoulos emphasizes caution in interpreting the results: while the findings suggest that mastery techniques may offer benefits through AI, other methodologies could also enhance educational experiences and should be tested further.
Ultimately, the goal is to reveal that AI can hold some promise in educational contexts, even if that promise lies in assisting students with a slower, more deliberate approach to learning.
Kristin Fasiang, a graduate student focusing on computer science and learning sciences at Northwestern University, contributed to this analysis alongside Jill Barshay from The Hechinger Report.
For more insights, reach out to Jill Barshay at [email protected].
This piece discussing AI and mastery learning is produced by The Hechinger Report, a nonprofit organization devoted to education journalism. Subscribe for updates on new reports and newsletters.
