This resource consist of plans for two class periods where students explore the use of generative AI as a tool to assist in evaluating primary sources as a part of historical research. It can also be adapted for use in exploring data relating to other disciplinary fields.

Instructions for Use/Implementation

Background

Undergraduate students in an introductory history course include both majors and non-majors, some of whom may never take history again during their college careers. Making the content of these courses engaging and helping students learn skills that will be applicable beyond a single course are both crucial components to making a positive experience for students. This exercise offers an opportunity to explore interesting content, experiment with the use of generative AI (ChatGPT) during that exploration, and to reflect upon how historians make meaning of the past.

It is important that students gain a basic understanding not only of content, but also of the process of “doing history” or thinking like a historian. This exercise is designed to help students explore the kinds of analysis that could be performed on a set of primary sources with the assistance of generative AI as a way to explore the historian’s craft. It could also be adapted for other disciplinary foci that use similar kinds of textual or data analytics.

Student Learning Outcomes: Students will…

  • Understand the difference between close reading of a single primary source and reading at a distance for large data sets
  • Examine some types of data historians gather from primary sources and possible analytical frameworks for its use
  • Experiment with the use of AI as a tool for working with primary sources and weigh how this might inform their understanding and help them to ask, and begin to answer, questions about the past

Overall Instructor Preparation

Instructor identifies a set, or more than one set, of documents, images, or other primary sources relating to a topic in the course that can be used for this exercise. Remember that LLMs, especially those free to use by students, use the material fed into the prompt as part of their training data, so choose the data sets carefully. It is best to use items in the public domain or for which you have permission to use with ChatGPT.

  • Text: Some sites that text files can easily be found for download include Project Gutenberg, Wikimedia Commons, HathiTrust Digital Library , and the Internet Archive. Possible examples:
    • The texts of Agricola and Germania by Tacitus
    • 4 Minute Men Bulletins by the US Committee on Public Information during World War
    • Burnt Njal’s Saga 1861 translation from Icelandic by George W. DaSent
  • Images: Some places that images files in the public domain can be found include Wikimedia Commons, various archives and special collections, and many museum sites. Check the usage guidelines at any of these sites. Possible Examples:
    • Photographs of illuminated medieval manuscript leaves
    • Photographs of Roman coins
    • Photographs of early-modern maps

Use this data with the prompts given below for students. Experimenting before running the class will help you to anticipate the kinds of output students will receive and help you frame questions for discussion.

First Class

Student Preparation

Assigned Reading: Moira Donovan, “How AI is helping historians better understand our past,” MIT Technology Review , April 11, 2023.

Reading Reflection Questions:

  1. What kinds of problems does the article say technology, and particularly AI, is helping to solve for historians?
  2. How much human involvement is still needed in order to make sense of the past? How might the role of historians change as the use of AI in studying the past increases? What new skills will we need to develop
  3. How can biases inherent in machine learning and large language models affect historical research? Can you think of historical examples where biases might have skewed the interpretation of events?
  4. What other ethical concerns arise from using AI in historical research? How might historians mitigate the risk of AI-generated falsifications or altered historical narratives?

Students should also examine the assigned primary source or sources before class. They should be able to describe what the item is, who created it, and what time period it comes from, as well as any interesting things they noticed right away when examining the item.

In Class Activity

Begin class by introducing the concepts of close reading and reading at a distance as ways historians explore their sources followed by a discussion of the assigned reading using the reflection questions. (15 minutes) Next, move on to a short discussion of the primary source or data set provided. Ask students to describe the primary source or data set and speculate on ways it could possibly be used by historians to understand the past. (10 minutes)

Next, students divide into groups of 3-4 students per group. Each group needs at least one person with a computer, tablet, or smartphone capable of using ChatGPT and access (via class LMS such as Canvas or from a cloud service like One Drive or Dropbox) to the primary source or sources as files for upload.

Ask students to paste the following prompt into ChatGPT (all students should be using the same file and the prompt below, initially) and upload the file(s). Working in groups for the students should read the responses from ChatGPT noticing the differences between ChatGPTs suggestions and what they had suggested before using AI. The groups can ask follow up questions and see where their chat leads them. (15 min)

Prompt: Act as an advanced, specialized AI assistant developed to aid students of history and historians in analyzing primary sources. You can evaluate sources across any time period and provide assistance in the following ways: categorization, summary, translation or transcription, visual analysis, textual analysis, or other appropriate analyses for historical research. When a file is uploaded, begin by categorizing the text, image, or data in the file and summarize the source in one to two sentences. Identify the type of source, time period it comes from, and any information about the potential audience or uses for the source when it was created. Next suggest ways that this source could potentially be used for historical research including the kinds of history that it might be useful for and any kinds of data analysis that it might contribute to.

To conclude, have students report out what their chats were like, discussing the following questions. (10 min)

  1. Would the suggestions from ChatGPT fall more into the “close reading” or “reading at a distance” category? Why do you think that is?
  2. What was the most surprising or unexpected thing ChatGPT suggested to you about how you could use this item to study history?
  3. How similar were your original ideas to those offered by ChatGPT?
  4. What kind of follow up questions did you ask and where did that take you? How similar or different was each group’s chat to the others

Second Class

Instructor Preparation

This time the instructor will make sure several different items, or sets of items, are available to the class – at least one distinct source and kind of source per group. As before, it is best to use items in the public domain or for which you have permission to use with ChatGPT.

Student Preparation

Students should be given one of the following methods of visual, textual, or data analysis to look up on the internet and become basically familiar with before the next class. This is not meant to be in-depth knowledge, but a basic understanding of what the technique is and what it can be used for. (homework of 20-30 minutes)

  • Sentiment Analysis (text, possibly visual)
  • Narrative Analysis (text)
  • Topic Modeling (text)
  • Named Entity Recognition (text)
  • Keyword Extraction (text)
  • Word Frequency Analysis (text)
  • Collocation Analysis (text)
  • Network Analysis (text, possibly data)
  • Visual Analysis (visual)
  • Correlation Analysis (data)
  • Geospatial Analysis (data)

In Class Activity

Begin class by asking each group to describe the type of analysis they researched to the class as a whole. The whole class could then brainstorm about which kind of analysis would have been useful to perform on the primary source used for the activity during the last class. (10 minutes)

Next, students break into groups of 3-4 students per group and each group is assigned a different set of primary sources or a data set to work with (some have visual sources, some textual, some structured data sets, etc.). Each group needs at least one person with a computer, tablet, or smartphone capable of using ChatGPT and access (via class LMS such as Canvas or from a cloud service like One Drive or Dropbox) to the primary source or sources as files for upload.

Ask students to paste the following prompt into ChatGPT and upload the file(s). Working in groups, students should use the initial prompt and then ask follow up questions. While some possible follow up questions are provided, this exercise works best when students come up with their own follow up prompts. Each group makes note of where ChatGPT fails to do what it is asked, and where it seems to excel. They should also make note of the follow up prompts they gave and how well they worked. (15-20 minutes)

Prompt: Act as my extremely capable and professional research assistant. You are an expert at examining primary sources, images, and data sets and performing textual, visual, and data analysis on these items. Please examine the uploaded item and tell me what [insert type of analysis here] reveals.

NOTE: If you are going to upload more than one file, tell ChatGPT to wait while you upload multiple files and that you will tell it when you are ready to begin the analysis.

Suggested follow up prompts:

  1. Please clarify [topic from chat output]
  2. Can you create a visualization of that for me?
  3. What other kind of analysis could you perform on this historical source and what would it be good for as historical research? (then try getting it do to one of it’s suggestions)
  4. Can you write a program or script that would help me to do this?
  5. Can you suggest ways that I could make this analysis or research more meaningful?

Bring the whole class back together for a concluding discussion using the following debriefing questions. (20 minutes)

  1. Which primary source/s did your group have and what kinds of analysis did you do with it?
  2. What kinds of analysis did ChatGPT seem very capable at and what kinds of analysis did it fall down on? How useful do you think its responses were? Did the responses get better with more prompting?
  3. What was the most surprising or unexpected thing ChatGPT came up with in analyzing your source or sources?
  4. What effect might the biases of the training model for the LLM have on how ChatGPT performed?
  5. What are some real-world reasons you can think of that someone might want to do this kind of research? Do tools like ChatGPT help? Why or why not?

Homework – Final Reflections

Write a final reflection on using AI to explore primary sources. Use this reflection to document the following:

  • What you found interesting, frustrating, helpful, or unhelpful about the process of using AI as a tool in preliminary historical research
  • What you learned about how historians study the past
  • What you believe is important for humans to do vs what we ask AI to do

AI Tool Used

ChatGPT (4o at the time this was written)

Prep Time Needed

An hour or two is needed for the instructor to assemble one or more data sets and try them out with the suggested prompts before proceeding to use it with a class.

Classroom Time Needed

The assignment here is designed to take one or two 50 minute class periods, plus some homework time, but may be adapted as needed.

Assessment Considerations

These assignments are designed mainly as in-class work with three short homework components (reading, research on a type of analysis, and the final reflection). This exercise is meant to be more about process than product, but points may be given for completing each component of the two-day activity including preparation for and participation in the group discussions, contributing to and completing the in-class group work, and the writing of the final reflection. The final reflection may be graded with a rubric if desired.

Recommended Course Size

This activity will work best with classes between 20 and 30 students, though it might be possible to do this with larger groups with some modifications.

Recommended Discipline(s)

History, Classics, Economic History, or any other discipline with primary source analysis.

Student Role(s)

Students work in groups with ChatGPT 4o to experiment with using AI as part of historical research. Using primary source files provided by their instructor, they use ChatGPT to analyze the sources and then discuss their work with the class as a whole.

Instructor Role

Instructors curate the sets of primary sources to be used, facilitate the discussions, and explore the process alongside their students.

List Author(s) Name Here (required): Nicole Jobin
Photo Source (required): By Jean Le Tavernier, Public Domain, https://commons.wikimedia.org/w/index.php?curid=74516
Image Alternate Text: The image depicts a medieval scribe seated at a writing desk. He is dressed in traditional attire, including a hat and robe, and is meticulously copying text from a large book using a quill pen. Surrounding the scribe are various books and manuscripts, some of which are open on stands, while others are bound and stacked on shelves. The workspace is richly detailed with ornate carvings and intricate designs, reflecting the scholarly environment of the period.

Edit Link: (emailed to author)
Request Now