Blending Human Knowledge with Generative AI: A Writing Exercise for Students

By Nathan Favero

Like many instructors, I find it intimidating to broach the topic of ChatGPT (and other generative AI tools) with my students. Nonetheless, through conversations with colleagues and inspiration from teaching examples posted to Twitter, I decided to try developing an exercise where students were required to use generative AI this semester. I found it to be a fruitful exercise, so I thought I would share some details with other instructors.

Motivation for the assignment

I teach courses in two master’s programs where written analysis is one of the key skills our students bring to the workforce, so the prospect of an AI system that can produce reasonably high-quality written work on complex topics has prodded me to revisit a lot of my assumptions about the kinds of jobs our students will work and the skills they will need in the workforce. I don’t claim to have answers to the various questions that have been raised by the emergence of generative AI, but I know that our students are concerned about these questions and looking for ways to explore them.

One of the key questions underlying my development of this assignment was: What is the role of a knowledge worker in the age of ChatGPT?

One working hypothesis I had in mind as I created on this assignment was: AI is a partner to human knowledge, not a substitute for it.

Assignment description

You will create 2 essays using generative AI—one on a topic where you do the readings and one where you skip the readings. Each essay should be around 500 to 1000 words.

Step 1: For the topic I’ve assigned you to read, spend 20-45 minutes skimming through the 4 topical readings on Canvas to get familiar with them.

Step 2: Use generative AI to create an essay on this topic you’ve just read about.

• Try using Claude (https://claude.ai), which can read documents you upload. When uploading the materials to Claude, use the text files (not the PDFs) I provided because they take up less space.
  • If Claude is not working, feel free to use ChatGPT or another similar program to write the essays, and it is OK that it cannot access the materials I provided.
  • With Google Bard or Microsoft Bing Chat, you can also try feeding it a URL for one or more of the papers I provided.
• If you think the initial draft can be improved, ask the generative AI to make revisions until you are happy with the essay. Try your best to get the generative AI to generate a high-quality essay.

Step 3: Repeat the process of creating an essay for the other topic.

• This time, do not read any of the materials yourself, but do feel free to upload the materials I’ve provided to Claude.
• Again, try to use the generative AI to produce a high-quality essay.

Step 4: Print out 4 copies of each essay and bring them to class so that you can show them to your discussion group.

Step 5: Write a 500-1000 word essay in which you reflect on your experience with the AI exercise.  Please be sure that your essay addresses all of the following items:

• A comparison of how things went for each of the two topics (healthcare.gov and environmental justice)
• Some consideration of what you learned about the ways in which generative AI can used as an effective writing tool for analyses of policy/public administration
• The limitations of generative AI as a writing tool

Instructor reflections for assignment development

Create Flexibility & Anticipate Access Issues: Don’t expect the AI tools will allow your students to interact with them the same way you did. Several students encountered issues such as their account being blocked (for a few hours or more permanently) by a particular tool. New features are constantly and inconsistently being rolled out, so try to make any assignment flexible enough to allow the student to adapt to whatever happens.

Expect Varied Experience & Some Wariness: During class discussions prior to this exercise, I learned that students varied widely in terms of their prior experience and attitude toward generative AI.  Some had intentionally avoided it and expressed slight anxiety about having to use it for this assignment. Others had been using it daily, including at their workplace. Regardless of background, I think most students got something out of this exercise, based on their reflection essays.

Acknowledge & Discuss Ethical Concerns: Some students have ethical concerns (as do I) about the development and use of generative AI. I acknowledged these concerns when explaining this assignment but also conveyed that regardless of whether you feel it is appropriate to use these tools in your everyday life, it is important to understand something about how they work because people around you will be using them.

Align the Assignment with Course Learning Outcomes: Two of my course objectives involve analyzing public administration dilemmas and proficiency in writing, so the reflection essays directly asked students to consider generative AI as a potential tool for written analyses of policy and public administration. Grading criteria for this assignment consisted of (1) completeness and relevance, (2) depth of content (with conclusions that are well-supported with examples and clear reasoning), and (3) general clarity and professionalism.

Notes on Assignment Design: With the two topics, I wanted them to ultimately compare whether the generative AI is simply better at writing about one topic as well as how their own knowledge of the topic affects their experience of using the AI. For this reason, the two topics I chose (rollout of healthcare.gov and environmental justice) are not equivalent in scope or difficulty. Similarly, I wanted students in their discussion groups to interact with others who bring different levels of personal understanding of each topic to the exercise, so I tried to form blended discussion groups of students who’d looked at the readings for each of the two topics. A sample template of group discussion questions is provided below.

Group discussion questions

1. Were you happy with the essay generated by the AI? Why or why not?

2. What differences did you notice across the two topics? Did the AI generate better content (e.g., more specific, easier to understand, more comprehensive) for one of the topics? Was it easier to write effective prompts for the topic where you looked at the readings?

3. What strategies did you use when writing prompts for the AI? How might using a different approach change the outcome of the essays?

4. What can you learn from seeing the other students’ essays in your group? Knowing what you know now, would you utilize the AI differently if you were to repeat the exercise?

5. Did you learn anything about how human knowledge can be effectively combined with generative AI for writing? Do you see AI any differently after completing this exercise?

Giving students a venue for wrestling with how generative AI might affect their work

Doing this assignment showed me that when it comes to generative AI, many of my students are wrestling with the same sorts of questions I am. And we are both longing for structured ways to explore these questions. My experience this semester has also highlighted the importance of approaching these topics with a posture of learning together with my students—not every behavior of the AI tools matched my expectations, and even AI experts are struggling to keep up with the breakneck speed at which the technology is evolving.

I encourage all instructors, regardless of discipline, to consider how generative AI might change the nature of knowledge work in their field, and to explore this question together with students in some form—whether it be a single class discussion or a series of assignments that build on one another. Like me, you won’t have all the answers. But you can help your students explore their own questions about how AI is changing their professional world.