What Happened When We Taught AI Literacy Like Writing
Results from a four-week experiment in a Grade 12 classroom in Geneva.
Last week I wrote about treating AI conversations as texts — using comparative textual analysis methods to teach students how to engage with AI the same way we teach them how to write. At the end, I mentioned that Aimée Skidmore and I tested the method with her Grade 12 students at the Collège du Léman in Geneva. This is that story, told by Aimée. The results were promising, imperfect, and worth sharing.
We hope you enjoy,
Mike
What Happened When We Taught AI Literacy Like Writing
When my Grade 12 students first started using ChatGPT, it wasn’t the cheating panic I expected. No one was pasting in an essay and turning it in as their own. What I saw was different, and in some ways harder to solve.
Many of my multilingual students were using AI to “fix” their writing. Others leaned on it just to finish a task, any task, as quickly as possible. The goal wasn’t deeper thinking. It was getting through.
That was my first wake-up call. AI wasn’t replacing student work. It was replacing student struggle. If I wanted them to do more than outsource the hard parts of learning, I needed a way to make their interaction with AI visible, structured, and worth reflecting on.
Why It Mattered
This wasn’t a lab experiment. It was a real class of 21 Grade 12 students in Media and Communications at a large international school in Switzerland. They came from 14 countries. Most spoke English as an additional language. A few had identified learning needs. Seven lived in the boarding house. All of them had already used AI before, both for schoolwork and in their personal lives.
That mix made the stakes higher. These were students on the edge of university who needed to build confidence in their own voices and ideas. If they learned to lean on AI as a shortcut, they risked missing the very skills the course was designed to teach: how to analyze, question, and communicate with independence. What I wanted was not to stop them from using AI. It was to help them use it well, in ways that were strategic, ethical, and that deepened their thinking instead of replacing it.
The Experiment
The structure and design for this project came from a method that Mike developed and shared with me in the spring of 2025. The idea reframes how we think about AI in schools: instead of judging the outputs that AI produces, we look directly at the interaction itself. The chat transcript becomes the assessable artifact. By focusing on the prompts, questions, and reasoning that students bring to the exchange, we can evaluate strategy, reflection, and ownership of thinking: skills that usually stay invisible.
To make this practical, we used Comparative Transcript Analysis (CTA). Students started by examining strong and weak examples of AI conversations, then co-created benchmarks for what “good” use looks like, and finally produced and annotated their own transcripts. Mike built the framework, we collaborated on the strong and weak examples of AI use in our project context, and I brought it into my Grade 12 classroom to see how students would take it up.
The short version: after four weeks of structured work with transcript analysis, 85.7% of students changed their approach to AI use, 47.6% became significantly more strategic in their interactions, and 81.0% endorsed continuing the method in school. Here's how we got there.
We ran the project over four weeks, step by step.
Week 1 started with a baseline. Students completed a short pre-survey on their AI use and compared exemplar and non-exemplar transcripts. This gave us a shared language for what makes an AI interaction effective.
Week 2 focused on criteria. Together, we built a rubric for “good” use of AI. Two habits became central: giving explicit rationale in prompts and asking “why” questions back at the AI. Both kept ownership with students and forced them to think about their choices.
Week 3 applied the method in a real assignment- Media in Society: Representation project. Students analyzed how a social or cultural group was portrayed in two media texts. AI could support brainstorming, but every transcript had to show reasoning and questioning.
Week 4 wrapped up with projects, reflections, and a post-survey on confidence, strategy, and attitudes toward AI.
Snapshots from the Room
One student moved from “slightly confident” to “completely confident” in obtaining responses, while still reporting “not at all confident” in evaluating accuracy. Her reflection showed how careful she remained: “I’m still very hesitant to use it in my schoolwork because I feel like it’s not my own and I’m not learning anything, but for this class I think it’s fun to use for certain projects.” Her growth was about building strategy while holding onto healthy skepticism, not slipping into overconfidence. (Student 7)
Another student represented a very different outcome. He showed dramatic confidence increases but with dismissive attitudes toward learning. His transcripts showed little reflection, and he dismissed some of the caution built into our rubric. This was flagged in the results as concerning overconfidence, a reminder that more confidence in using AI does not always mean better or more thoughtful use. (Student 2)
A multilingual student showed balanced development. He gained confidence while also recognizing the risks of AI. His comment highlighted this awareness: “Phrasing is extremely important when asking ChatGPT things because not including certain things can change the whole outcome of a response by AI.” His case demonstrated how careful attention to language could support more deliberate, metacognitive use of the tool. (Student 19)
What Changed
Students shifted how they used AI.
After working with examples and benchmarks, 85.7% of students reported changing their approach. For many, this meant slowing down and thinking more carefully about what they were asking. One student explained, “I now think more about my own input and how to change or extend it later in the interaction.” (Student 5)
Students became more strategic and reflective.
Almost half the class (47.6 %) said they started thinking more about their input before hitting send, a sign of iterative, metacognitive work. Another student put it simply: “Phrasing is extremely important when asking ChatGPT things because not including certain things can change the whole outcome of a response by AI (even just by adding something small).” (Student 19)
Students valued the method and wanted more.
In the post-survey, 81% endorsed continued use of the method and activity in schools. Their reasons showed a sophisticated awareness of what was at stake. One student wrote, “I think this kind of activity should be used and taught more often in school, since AI is evolving really quickly nowadays and it is important to know how to use it responsibly and as efficiently as possible.” (Student 19) Another added, “Either way students are using it, we might as well learn how to use it well.” (Student 21)
Together, these shifts showed that when AI interaction itself became visible and assessable, students not only changed their behavior but also recognized its value for their learning.
What Didn’t Work
The rubric we built together was too fixed. Some chats fit neatly into the categories, but many did not. Stronger interactions showed depth without much help from a rubric, while the weakest were obvious on their own. It needed more cycles and greater precision.
The project task was also too simple at first. Students did not need AI for the basics, so their use stayed shallow. Later, some turned to it for parts we had not asked them to, and that use was not as thoughtful as I hoped for. They drifted back to letting it do the heavy lifting, which made it clear that placement of AI use in the task matters.
Even with structure, a few students still tried to shift the hardest thinking onto the tool. That habit will take longer to change.
Our Takeaways as Teachers
This project reminded us that students can learn to use AI in thoughtful, ethical, and strategic ways, but it takes design that values process over product. Students need time and low-pressure opportunities to treat AI as a partner in thinking rather than a shortcut to completion.
What stood out most was that complexity changes the way they engage. When the work was routine, they complied. When the work was uncertain or demanding, they leaned in more carefully, treating AI as a tool to test and refine their ideas. That is where the most growth happened.
The rubric helped open the door, but it cannot be static. It has to shift with the task and with students’ developing fluency.
The larger takeaway is hopeful. Students want to understand how to use AI well. When we make their interactions visible and assessable, we give them a chance to build an intentional relationship with the tools they are already using.
What this means for other teachers
Try this tomorrow
Bring two transcripts to class, one strong and one weak, and ask students to compare them.
Ask students to explain their reasoning, to add something of themselves, inside the prompt: “Analyze this poem for themes of isolation because I noticed repeated imagery of walls and barriers.”
Require one “why” question back to the AI so students have to challenge its assumptions.
For school leaders
Treat AI interaction itself as assessable. The transcript is the artifact. What matters is the process students show, not just the product they hand in.
Limitations & Guardrails
This was a small pilot in one classroom with 21 students. All findings are based on self-report surveys and reflections, so they show how students described their process rather than direct evidence of skill change. The study took place in a single school and measured only short-term shifts. The project served the purpose intended inside a real curriculum rather than purely for research. These limits mean results should be read as preliminary. Running the project ethically mattered to us: participation was voluntary, no student was penalized for opting out, and all work remained in-class with student identity and data protected.
A Way Forward
This experience affirmed the direction forward. Students can learn to use AI ethically and strategically, and they want to understand it. When we design learning that treats AI use as a legitimate and assessable part of the thinking process, and when we value process over product, we help them build a more intentional relationship with the tools they are already using.
The key is not to wait until everything is perfect. Try something small. Step in, take action, and let students experiment alongside you. They are asking for it, and they do not expect you to have every answer. What matters is the willingness to move forward boldly and then pay attention to what happens. The real measure is whether those experiments bring students closer to the kind of thinking, doing, and deciding we know they are capable of.
A full account of this experiment, including complete survey data and analysis, will be published in the Elsevier volume GenAI in Higher Education, forthcoming summer 2026.
 | A guest post by
|
I think this is really valuable and thoughtful. At the same time, I also struggled with some sense of loss wondering what it might have been like if students had spent 4 weeks working so intensely and thoughtfully on how to talk to *each other* about the work they were analyzing and how to improve their skills at that as much as they were working to improve their skills with tech tools.
"AI wasn't replacing student work. It was replacing student struggle." I want to put that sentence on a wall somewhere.
I research creativity in education, and what you're describing — students offloading the hard cognitive parts — is what happens when years of compliance-based schooling teach people that struggle signals failure rather than growth. By the time students reach you, escaping the hard parts feels like the smart move. You're essentially teaching them to want the friction back.
What I love about CTA is that it locates the creative act where it actually happens: in the framing of the question, not the quality of the answer. The prompt is a thinking artifact. You found a way to make that visible and gradable, which is genuinely hard to do.
I'm building something I'm calling "Outwondering the Algorithm" — the premise that curiosity is the foundational human skill in an AI world. Your rubric's two core habits (give rationale, ask why back at the AI) are curiosity made structural. That's not a small thing.
Thank you for sharing what didn't work too. That student with dramatic confidence gains but no reflection — that's the cautionary tale the field needs to hear.