This reframes transcripts perfectly, not proof of misconduct, but artifacts of process.
In my cybersecurity & AI courses we annotate chats as if they were audit trails provenance (model/version/source hints), chain-of-custody (what moved from AI → student work), and a short refusal-with-reason note before acceptance. It turns metacognition into governance literacy.
If you had to choose one beginner tag that changes student behavior fastest provenance, refusal-with-reason, or risk-of-misuse which would you start with?
Thanks for developing and writing up this approach. I work with advanced undergraduate and graduate students who are integrating NotebookLM and other AI tools into their learning and research. After reading your article and following several of the links, I'm going to try the annotation approach with them, to help them take greater agency in their role in shaping the iterations more deliberately.
Yes, it's definitely hard - but that's what makes it good!
To your point, the annotation keys change depending on the bot, task, assignment, and goals laid out by the educator.
But that's no different than changing annotation keys for fiction, non-fiction, poetry, rhetorical analysis, speechwriting, and more.
Each one has a different annotation key, and they change depending on the level of the student, the level of the text, and the goals of the educator.
As an example, in teaching Middle School Literature and Writing, we sometimes asked students to annotate the same text 2-3 times with 2-3 different annotation keys - each one focusing on a different element. (Character Development, Theme, Setting, etc.)
Next week I'm going to release a lesson plan with an annotation key for interviewing a fictional AI character chatbot. It's pretty intuitive, once you get the hang of it.
I'd love yours and other's feedback when it comes out.
Wow, looking forward to seeing how this is laid out, Mike. Sounds like a strong fit for English classrooms, so I’ll be interested to see how it translates beyond that.
How receptive have teachers and school districts been to your methodology so far?
Agreed! I think it works especially well in Humanities, but can work in Sciences as well. Mathematics is the toughest.
Teachers and districts are most receptive to the "sparring partner" concept - the idea that we can set up AI interactions as tests. Universities are more interested in "comparative transcript analysis" and the idea of research-based experimentation with grading and annotating chats. It's still early days, and the method should undergo a lot more testing, but I'm excited to share Aimee and I's paper when the time comes (it is under peer review now.)
I read your article on the sparring partner concept, and I can see why teachers would really like that idea. That was a great post. I’m looking forward to when your article comes out. I’m really interested in the results you both found.
And thanks for the work you’re doing in this area, because there needs to be more of us doing it. It’s not going away, and people pretending it’s not happening, or that they don’t have to adapt, are just delaying the inevitable. In the meantime, students are the ones taking the hit.
This reframes transcripts perfectly, not proof of misconduct, but artifacts of process.
In my cybersecurity & AI courses we annotate chats as if they were audit trails provenance (model/version/source hints), chain-of-custody (what moved from AI → student work), and a short refusal-with-reason note before acceptance. It turns metacognition into governance literacy.
If you had to choose one beginner tag that changes student behavior fastest provenance, refusal-with-reason, or risk-of-misuse which would you start with?
Thanks for developing and writing up this approach. I work with advanced undergraduate and graduate students who are integrating NotebookLM and other AI tools into their learning and research. After reading your article and following several of the links, I'm going to try the annotation approach with them, to help them take greater agency in their role in shaping the iterations more deliberately.
Interesting concept. I imagine the open-ended nature of chats could make this tricky without a clear structure.
Yes, it's definitely hard - but that's what makes it good!
To your point, the annotation keys change depending on the bot, task, assignment, and goals laid out by the educator.
But that's no different than changing annotation keys for fiction, non-fiction, poetry, rhetorical analysis, speechwriting, and more.
Each one has a different annotation key, and they change depending on the level of the student, the level of the text, and the goals of the educator.
As an example, in teaching Middle School Literature and Writing, we sometimes asked students to annotate the same text 2-3 times with 2-3 different annotation keys - each one focusing on a different element. (Character Development, Theme, Setting, etc.)
Next week I'm going to release a lesson plan with an annotation key for interviewing a fictional AI character chatbot. It's pretty intuitive, once you get the hang of it.
I'd love yours and other's feedback when it comes out.
Wow, looking forward to seeing how this is laid out, Mike. Sounds like a strong fit for English classrooms, so I’ll be interested to see how it translates beyond that.
How receptive have teachers and school districts been to your methodology so far?
Agreed! I think it works especially well in Humanities, but can work in Sciences as well. Mathematics is the toughest.
Teachers and districts are most receptive to the "sparring partner" concept - the idea that we can set up AI interactions as tests. Universities are more interested in "comparative transcript analysis" and the idea of research-based experimentation with grading and annotating chats. It's still early days, and the method should undergo a lot more testing, but I'm excited to share Aimee and I's paper when the time comes (it is under peer review now.)
Thanks for your engagement and support!
I read your article on the sparring partner concept, and I can see why teachers would really like that idea. That was a great post. I’m looking forward to when your article comes out. I’m really interested in the results you both found.
And thanks for the work you’re doing in this area, because there needs to be more of us doing it. It’s not going away, and people pretending it’s not happening, or that they don’t have to adapt, are just delaying the inevitable. In the meantime, students are the ones taking the hit.
Your case for moving from skimming chats to annotating them is the missing bridge between “AI use” and actual learning. You’ve put it out really well.
If you had to choose one lever for a starter protocol, which would it be:
(1) Prompt card (goal + constraints)
(2) Evidence tagging (quote/claim/source)
(3) Bias check (what did the model make too easy to accept), or
(4) Revision note (what I kept/changed and why)?
I’m trialing this as a 15-minute routine; curious which single move you’d weight most on day one. Loved the piece.