Announcing the AI Review Quiz Maker
Generative AI has a lot of possibility in the higher education space — and a lot of peril. Dropping a chatbot into class websites would be a big mistake, as the chatbot itself would likely confabulate information. Any use of generative AI in our courses, including the CoursePlus learning management system, needs to be carefully considered.
In the Spring of 2024, the CoursePlus team worked with a small group of faculty on potential uses of generative AI in CoursePlus. After discarding some initial ideas, one idea that gained traction among the working group was using generative AI to assist in the creation of review quizzes based on course lectures. Time and again, literature on effective teaching shows that practice testing is the most effective way of helping students remember course content. The challenge, of course, is that faculty and TAs are very time-pressed, and creating the questions for review quizzes is often beyond the capacity of very busy faculty and TAs.
Analysis and summarization of text is one area where generative AI tools like ChatGPT truly excel. Given that we have text-based transcripts for all CTL-produced lectures, could we build a tool which creates review quizzes based on remembering and recalling information from the lecture content?
We could, and we did.
When you edit the Page Builder page for a CTL-managed lecture in CoursePlus, and that lecture has transcripts in the Lecture Materials box, you will see a button for the “AI Review Quiz Maker.”
Clicking this button brings up a short form and some very important information about how this tool works. This includes:
- Using the tool is absolutely optional.
- The transcripts from the lecture are sent to OpenAI (the makers of ChatGPT) for processing and creation of the review quiz questions.
- The content of the transcripts (and therefore the lecture) is not available to or used to train ChatGPT in any way.
- The transcript files are kept completely private from OpenAI and are immediately deleted after use.
- The tool uses the same, advanced large language model (GPT-4o) that you can use in the paid version of ChatGPT.
- You can run the tool on a lecture as many times as you want. You will get different generated content each time.
- You can generate 5, 10, or 20 questions.
- It takes about 3-4 minutes to generate the quiz questions and answers.
- You will receive an email containing the generated quiz questions, answers, and explanations for the correct response.
- If your lecture has more than four sections, you will receive two emails. Each email will cover the content of half of the lecture sections. This is done to ensure that the content in the transcript doesn’t exceed GPT-4o’s available “memory,” also known as the context window.
There are two important caveats about the use of this tool:
First, this tool does not create actual quizzes in the Quiz Generator in your CoursePlus site. The tool only creates the questions, answers, and an explanation of why the generative AI reasons that the correct answer is correct. That information is sent to you via email so that you can review the questions, the answers, and their accuracy. It’s still possible – though unusual – for GPT-4o to get things wrong in this context, particularly if there’s incorrect information in the source transcript. Individual questions need to be evaluated by an expert before being given to students as part of any quiz or exam — even a review quiz. With the questions, answers, and answer explanations created for you, however, you can then much more easily create in-lecture quizzes or formative assessments for your students.
Second, the quiz questions and answers are based on the content in the lecture transcript files. These transcripts are not 100% accurate all the time, and may have errors and misspellings, particularly in highly technical scientific and medical terms. This is another reason why careful review of the generated content is important prior to creating an in-lecture quiz or review quiz.
Note: The AI review quiz maker does not work with classroom recordings made in Panopto or Zoom. This is something we may consider in the future.
The CoursePlus team is looking at ways of incorporating generative AI into other CoursePlus tools. We will continue to prioritize the usefulness of the potential functionality alongside data privacy in any future AI-based enhancements.
As always, if you have questions about this new feature, please reach out to the CTL Help team, and we will be happy to answer any questions you might have.