Harvard AI Pedagogy Project: Safe and Effective Use of Personality Bots in the Classroom

Controversial bots can be used in ways that protect and encourage critical thinking while developing AI Literacy

My classroom journey with AI began with a personality bot, so I was thrilled when my AI activity “Interviewing a Fictional Character” was accepted into the Harvard AI Pedagogy Project.

For those who are unaware, the project brings together submissions from educators all over the world that are designed to integrate AI tools into the classroom experience in ways that are responsible and effective.

I would highly recommend checking it out as there are many top-quality assignment ideas and an AI Guide that is useful for beginners.

You can view the nuts and bolts of the project at the link in the first paragraph, but I thought I’d highlight a few aspects of the activity as a “Companion Resource” that will allow you to implement the assignment in a nuanced and effective manner.

The General Structure

The assignment utilizes three necessary steps for engaging in a way that both protects and stretches your student’s critical thinking and analysis skills.

1. Generating thoughtful, open-ended interview questions that take into account the personality, history, or relevant background information of the fictional or historical figure.

2. Interview the bot with a teacher-generated rubric for the interaction - reviewed as a class beforehand. Students will share the chat with the teacher afterwards for assessment against such a rubric. The guidelines should be tailored to the circumstances, personality, and background of the figure or character. 

   1. Tailoring Example 1: If my students were interviewing Aristotle, my rubric would ask students to engage in a philosophical debate, demonstrate an understanding of logic and Aristotelian philosophy, and showcase argumentation skills learned during the unit. 

   2. Talioring Example 2: If they are interviewing Napoleon Bonaparte, I would ask them to press the General on key decisions he made in his various military engagements, show an understanding of the relevant trigger points in Napoleon’s personal life, and/or engage in a productive way that deepens the user’s understanding of the unanswered questions that surround his mercurial personality and story.

   3. Tailoring Example 3: For Holden Caulfield, I had my students focus on active listening and empathy, since he is depressed and suicidal.

3. Teacher evaluates and assesses the chats against the rubric. 

4. Students complete a writing assignment where they evaluate the effectiveness of the bot in mimicking the character. They can also evaluate the effectiveness of the bot in furthering their understanding of the character.

Privacy and Safety

Character.ai is not compliant with FERPA rules. But, SchoolAI is and allows you to create your own bot, much like a CustomGPT. This might sound daunting, but the process is entirely language-based. You can upload a summary of the book or of a set of historical articles pertaining to your figure. You can write in key points of emphasis in the back-end and provide specific instructions in an effort to ensure that the bot interacts in the way that will complement your instruction.

SchoolAI has also generated their own fictional and historical figures, but the company and many other EdTech personalities will steer you to use the bot as a “deliverer of content.” This is the wrong way to think about these bots, since they produce inaccuracies and misleading statements.

Instead, think of the inaccuracies and misleading statements as a feature, not a bug. The bots are not perfect, and that’s the point.

The mistakes that the bot makes can and should be ferreted out by your student and/or included in the final analysis piece. If the bot mimics the character perfectly, the student should be able to prove it based on the information they learned earlier in the unit. This requires critical thinking, argumentation, comparing and contrasting, and many more skills. Put differently, the “untrustworthiness” of character bots is what makes them such a fertile ground for student thinking and analysis.

Further Considerations

Each situation is different, as are your students. In that vein, here are a couple of companion considerations to take into account as you design your assessment:

1. Timing: Is this occurring at the beginning, middle, or end of the unit? I would recommend using this assignment after your students have studied the figure. But it may be possible to execute it earlier in a unit, with additional considerations.

   1. Beginning: If students engage with the bot before knowing the character, the main activity could be generating a set of predictions. The students can also utilize broad-based questions provided by the teacher or generate their own and create a character profile based on the information provided. This is not just a creative writing or literature skill, but also a metacognitive and analytical thinking skill. However, there are some notable obstacles here, not least of which is the fact the bot might reveal everything about the person’s life — fictional or otherwise — thus frontrunning or spoiling your plans for the unit. Or, it can produce inaccuracies that are difficult to extract from your student’s mind later in the unit.

   2. Middle: In the middle of the unit, students will have some but not all information about the figure or character. In this place, students can critically review the bot to some extent in combination with a set of predictions about how the story “ends.” This is also tricky for a number of reasons, not least of which because the bot may give away the ending. However, critically reviewing information based on what you already know about a “person” in combination with the imaginative practice of making predictions can, potentially, be an effective assignment.

   3. End: In my opinion, the cleanest route is to have them analyze the bot after they have already studied the character. This creates a robust opportunity for analysis, not only of the bot itself but of the character from the book or history against the transcript of the mimicking robot, since they have the information already in tow.

   4. Combo: If the beginning/middle approach makes you uneasy, you might consider combining conversations from the beginning and end of unit. A comparison of transcripts might yield some fascinating revelations about the efficacy and nature of these personality bots, thus developing AI Literacy in conjunction with the delivery of content and character understandings.

2. Questioning Skills: Do your students know how to ask good questions? If not, you may have to run through a few practice rounds before putting the bot in front of them.

3. Journalistic Thinking: This activity is an interview, which means your student should put on their Reporter or Biographer Hat. During the question-writing portion, encourage your students to toss their bland questions into the bin. We want to push (and pull) the character to be revealing, interesting, and dynamic.

4. Analysis Essay: When you students analyze the bot post-conversation, it is important to recognize that every student will have a different experience, even with the same LLM. As a result, their final analysis pieces will all be different. This creates a great opportunity to evaluate their argumentation skills, as there is no one right answer. It also makes grading significantly more interesting.

Reach out for more information or guidance. I’m happy to share. 

Happy hunting as you start the school year!

P.S.: Harvard made a few edits of which I do not approve, including the idea that students should use the voice chat feature. This is not a deal-breaker, but I personally would lean away from this aspect for now.