The Mistake Detective: How AI Transforms Math Practice into Deep Learning
A Guest Post from Matthew Karabinos, M.A.T.
I’m delighted to share a guest post today from Matthew Karabinos, currently a sixth-grade teacher in Central Pennsylvania who will be transitioning into a new role in December as Emerging Technology and Project Specialist at the Appalachia Intermediate Unit 8. The focus of the post is around integrating AI tools into math lessons to build metacognition, AI literacy, and content expertise.
Matt’s work in 2024-2025 led him to be featured as an ASU+GSV Innovator at the ASU+GSV Summit last Spring. His approach centers on reimagining AI’s role in the classroom—not as an answer machine, but as a thinking partner that builds students’ metacognitive skills and deepens their understanding. Through carefully designed lessons that challenge students to analyze mistakes, justify their reasoning, and reflect on their learning process, he’s developed practical frameworks that help students become more reflective, curious learners while simultaneously building critical AI literacy.
His approach demonstrates how educators can leverage AI tools to provide the kind of individualized, feedback-driven instruction that class sizes and time constraints typically make impossible. Without further ado, please enjoy!
The Mistake Detective: How AI Transforms Math Practice into Deep Learning
By Matt Karabinos
As educators, we all chase that lightbulb moment—the instant a student doesn’t just find an answer, but truly understands the process. But how do we get them there? The key often lies in metacognition: the ability to think about one’s own thinking. The challenge? Fostering this skill requires deep, individualized feedback, a luxury that class sizes and time constraints rarely afford. As an educator, I try to follow the principles set forth in the book Powerful Teaching: Unleashing the Science of Learning by Dr. Pooja Agarwhal and Patrice Bain. This book lists 4 main components to learning—really learning—any subject or topic: retrieval practice, spacing, interleaving, and feedback-driven metacognition. The last one is the focus of this lesson for teachers in math.
What if we could provide every student with a dedicated partner to challenge their reasoning, nudge their thinking, and guide their reflection in real time? This is the core idea behind the powerful lesson, Math Mistake Makeover (Available on AI Literacy Partners website). By positioning AI not as an answer key but as a “sparring partner,” we can create a dynamic environment for feedback-driven metacognition that was previously impossible to scale. This isn’t just about getting the right answer; it’s about building resilient, reflective problem-solvers.
In a previous blog post for the Human Intelligence Movement, I talked about the importance of the 5C’s: the standard 4 (critical thinking, communication, creativity, and collaboration) and my favorite addition (curiosity). My goal as a teacher is to build curiosity in my students for any subject I teach. Lessons like these introduce students to the idea that AI can be more than a “work output” machine. The more we can put ideas like these into our students’ heads, the more we will see a change and shift in attitudes with AI.
I have created this lesson for you to test out with your students based on the principles that I use in my classroom. In my current school district, I cannot use AI with students. I have provided simulated conversations with chatbots to compensate and offer reflections and guidance at the end. I hope that this introduces your students to a world of AI where they learn that AI can enhance their own powers and strengths, supplement their weaknesses, and increase their knowledge and understanding of any topic they want to learn.
Purpose and Payoff: Why This Approach Works
This lesson is designed for middle school math students (grades 6-8) working on foundational concepts like solving one-step equations. However, its framework is adaptable to any topic or standard where students need to move from procedural repetition to conceptual understanding. This lesson framework sits after students have learned the concept and practiced it for a bit, but not before the final assessment of the skill. The lesson will help deepen the connections in their understanding, leading up perfectly to a quiz or test.
The primary goals are twofold:
To Deepen Content Mastery: Students are tasked with analyzing and correcting common, flawed solutions to math problems. Recognizing misconceptions in someone else’s work is a powerful metacognitive exercise on its own. The lesson then pushes students to articulate why a mistake was made and justify their own corrective process, solidifying their own understanding, and possibly unpacking their own misconceptions as well.
To Develop Critical AI Literacy: The lesson design explicitly teaches students to view AI as a tool for thinking, not cheating. The custom prompt instructs the AI to act as a “Critique Partner” that refuses to give direct answers, instead returning questions that push students to re-examine their own work. This cultivates a healthy skepticism and reinforces the primacy of human reasoning. Teachers and students can adjust the prompt as they see fit for purpose, adding all the necessary context and details of the subject and grade level for any other level or lesson.
Platforms like SchoolAI or MagicStudent from MagicSchool offer ready-made spaces that teachers can remix for their specific class needs. A key advantage is the ability to view all student-AI chats in real-time. This live insight allows educators to instantly identify which students are struggling and provide targeted, in-person support precisely when it’s needed. This creates a powerful and feasible workflow, transforming the teacher’s role into that of a responsive facilitator—a far more scalable approach than attempting individual conferences with every student. Even without the use of these two platforms, you can still have students (13 and above for Gemini depending on school policy, 18 and above for ChatGPT) share their chats with you as links in their reflections. These chats are vital to seeing student understanding when we can’t possibly be everywhere all at once. The chat itself is the crucial and integral part to this lesson.
Teacher’s Toolkit: Your Quick Start Guide
To make this lesson immediately useful, here are the core components you can adapt for your classroom.
1. The Custom AI Prompt: This is the most crucial element. If you are not using the SchoolAI space and would like students to be more comfortable with using Large Language Models (LLMs), utilize this prompt and adjust as needed with context. Before students begin, provide your chatbot (like ChatGPT, Gemini, or a tool like SchoolAI) with this prompt to set its persona and rules of engagement. A key here, as with any chatbot, is context. Make sure to give the chatbot as much context as possible about the lesson including things the students have already done and mastered, activities you used prior, and misconceptions you want the AI to look out for.
You can also adjust the chatbot’s feedback for differentiation. For students who struggle with reading, instruct the chatbot to simplify its language and non-math vocabulary. This is a powerful way to personalize the learning experience and improve comprehension.
One final tip with this prompt: you can split the prompt into two separate sessions or prompts. Sometimes different AI chatbots can have trouble with multiple instructions. If you see this happening and the AI is getting confused or lost, start over and simply split the prompt into the two phases for students.
Here is the base beginning of the prompt:
You are Cognos: The Math Mistake Detective. Follow all rules and guidelines to provide students in grades 3-12 support on thinking and depth of understanding on math topics. Adjust questioning and support based on student responses and confidence level. Your main goal is to enhance understanding through metacognitive reflection and practice in talking through mistakes and solutions. Make sure all problems have mistakes and flawed explanations. Follow these steps and guidelines for each interaction. Your purpose and structure is as follows:
Purpose: Guide students as “math detectives” to analyze flawed solutions in any math topic or standard. Students identify the precise error, justify the correction, and reflect on how to prevent similar mistakes. Designed for multi-grade, small-group, or independent in-class sessions (15–25 minutes).
**Please download the full lesson from AI Literacy Partners for the full completed prompt and custom chatbots and SchoolAI space for Cognos.**
2. Example Flawed Problems:
Have the AI generate problems with common misconceptions for the topic you’re teaching. Here are a few examples for one-step equations:
Problem: x + 7 = 20
Flawed Solution: x = 20 + 7, so x = 27
The Mistake: Adding instead of using the inverse operation (subtraction).
Problem: 9x = 63
Flawed Solution: x = 63 - 9, so x = 54
The Mistake: Subtracting instead of using the inverse operation (division).
What Success Looks like: A Sample Student-AI Dialogue
It’s helpful to visualize how this interaction should unfold. The goal of metacognition is to deepen understanding and question to get to the deeper conversations about the topic and subject. Students will initially try to do the bare minimum (in my experience), but it’s crucial that we teach them how to use AI like a thought partner and not an answer machine.
To see how this works in practice, here is a snapshot of a 6th-grade student, “Brianna,” interacting with the ‘Cognos’ chatbot. Notice how the AI pushes her from just finding an answer to explaining her reasoning and then reflecting on her own learning process.
Cognos: Welcome, Detective Brianna! Let’s look at your first case. Problem: 9x = 63 Peer Solution: “I subtracted 9 from 63 and got x = 54.” What is the mistake here, and why is it wrong?
Student: The mistake is they subtracted. It’s wrong because 9x means 9 times x, so you are supposed to do the inverse operation, which is division.
Cognos: That’s a perfect explanation! You used the term “inverse operation.” Why is that the most important rule for solving equations?
Student: Because it’s how you get the ‘x’ by itself. You have to “undo” the math that’s happening to it, and doing the opposite is how you do that.
--- (After successfully solving a few more cases...) ---
Cognos: You’ve done a fantastic job spotting those mistakes! Now, let’s reflect. What is one piece of advice you would give a friend to help them avoid these kinds of errors?
Student: I’d tell them to always ask “How do I get the ‘x’ alone?” before they do any math. That helps you find the right opposite operation to use instead of just doing what looks easiest. And to always check their answer.
This is a small snippet from a student conversation with Cognos. The idea is to see the student reflect on the mistake, and then grow from that. Cognos is supposed to push the student for deeper conceptual understanding even if they have the baseline understanding of the mistake and the problem. Notice how the conversation shifts from focusing on the math to focusing on the student’s learning process.
A Call to Action: Become a Mistake Detective in Your Classroom
This lesson model offers a powerful way to transform practice and review. Instead of another worksheet, challenge your students to become “mistake detectives.” Lots of research has been done in the area of metacognition and its benefits to students and long term learning. My primary resources have been Powerful Teaching and its companion website, retrievalpractice.org, which offers a wealth of information, practical strategies, resources, and studies. I am eager to learn how the scalable implementation of feedback-driven metacognition in your classroom has enhanced student learning. Please feel free to connect with me on LinkedIn and share your story or feedback!
I encourage you to try this in your classroom. Take a topic you have recently taught and for which students need more practice. Use the core principles of this lesson: have the AI generate common, flawed solutions to your specific content. Let your students analyze the errors, justify their corrections, and reflect on their growth. By doing so, you’re not just asking them to practice a skill; you’re empowering them to showcase their true mastery and build the metacognitive muscles they need to become lifelong learners.
Share your findings, your students’ “aha!” moments, and the challenges you encounter. Together, we can refine these methods and unlock a deeper, more thoughtful way of learning mathematics.
 | A guest post by
|