With the Fall term wrapping up in Northern American higher ed, a lot of my fellow humanities professors are reflecting on the way they feel forced to change their writing assignments to respond to LLMs. A lot of people are moving to only doing in-class writing — and bemoaning the loss of longer, more involved paper assignments.
This semester I taught an upper-division undergrad course that satisfied a “Writing in the Discipline” GE requirement, for which students are supposed to write roughly 5,000 words. So, to pass the class with a B,1 this semester my students needed to write four short papers (highly structured, 4 paragraphs, 300-400 words each), revise two of those short papers based on my feedback, revise and expand one into a mid-length paper (about 1k words), and then revise and expand it again into a “full paper” (about 2k words). To get an A, in addition they needed to either write a second mid-length paper or submit a draft of the “full paper” about a month early for an extra round of feedback and revision.
None of this writing was done in class. Out of 24 students, I had one who used an LLM in a way that violated the class LLM policy, and that was on a single assignment. Perhaps there are a few more that I’ve missed; but I’m not seeing “they’re all cheating!” like some of my colleagues are reporting.
The purpose of this post is to document what I did. Maybe I got lucky? Maybe it won’t work in the future? Or maybe it’ll work for you too?
First, I had the students read a pair of essays on the value of first drafts.2 The first is Anne Lamont’s “Messy First Drafts,” from the ’90s; the second is a recent blog post that references the first and argues against using an LLM to write the first draft of a paper. (In the future, I plan to include this taxonomy of LLM usage as part of the assigned reading.)
Second, I anonymously collected students’ experience and thoughts on LLMs. I think anonymity was important for candor here. In class, we discussed the two essays, and I had them read and react to another student’s anonymous paragraph.
Third, I had the students determine our class LLM policy. The taxonomy I linked up above was really helpful for structuring this discussion and the resulting policy. The class deliberated in small groups, then collectively, and voted to accept the policy for each potential use. There was some good debate over “brainstorming” and the exact wording for “co-create and revise.” The vote was generally unanimous; for one or two levels a couple-three were opposed. This was the result, as added to the syllabus:
In October, the class discussed the use of LLMs (large language models, such as ChatGPT) on course assignments. Using this taxonomy, we deliberated and voted to adopt the following policy. Violations are a form of academic misconduct.
No use: Always permitted, except when a particular assignment explicitly requires use of a LLM. Except for those assignments, you’re never required to use a LLM in any way.
Organizational: Permitted, but you’re encouraged to check the LLM output for accuracy.
Brainstorming: Permitted only for developing the “big picture” of an assignment. You’re also encouraged to check the LLM output for accuracy.
Feedback: Permitted, but you must explicitly state that you used the LLM and note how.
Co-create and revise: Depends on how the LLM is used to co-create and revise your work. For developing an outline, you may use the section headings and overall structure, but not further details that the LLM provides, such as a suggested thesis or argument. For drafting the paper, LLM use is prohibited.
Unrestricted, attributed: Generally prohibited. Some particular assignments may explicitly permit or require using a LLM.
Unrestricted, unattributed: Always prohibited.
Personally, I would not have permitted “brainstorming” or the outline-development use under “co-create and revise.” But I suspect respecting their autonomy — literally! — was important for the (apparent) high rate of compliance.
Fourth, each writing assignment had to include a wrapper, which among other things asked what resources were used on the paper, how they were used, and which was most useful. The resources included things like meeting 1-1 with me, the Writing Center, and Grammarly, along with an LLM. Grammarly is extremely popular; most of my students reported using it. Only a few students reported using an LLM, and most often it was to help summarize a reading or other source (which falls under “organizational” use).
The only violation I identified and acted upon happened a couple of weeks after the class formulated their policy. The fifth thing I did was handle this as a learning opportunity rather than the first stage of a disciplinary process. I asked the student to meet with me, and started by saying that I didn’t want this to become a disciplinary process. We talked about how they had used the LLM on the paper, and the stylistic telltales that made me suspect the policy had been violated. In particular, I explained how ChatGPT is very good at using sophisticated vocabulary and complex grammar to make very generic, shallow points. It’s not very insightful, and can’t put together a very compelling argument by philosophical standards. These points seemed to be well-taken. We also reviewed the LLM policy, and I reminded them of some elements of the class discussion behind it. After about 15 minutes of discussion, the student agreed that they had violated the policy, and accepted my offer to rewrite the paper from scratch. They did so, I gave them full credit, and we were able to move on.
More generally, I think it’s important to avoid an antagonistic relationship with my students. In my view, academic misconduct has two primary causes: the intense demands on students’ time and mental health; and a purely credentialist attitude towards education, that is, an attitude that the sole value of education is a formal credential (and not even instrumentally useful skills). Increased surveillance and threats likely exacerbate the demands on students and further alienate them from their coursework. This means they’re more rather than less likely to engage in misconduct. On the other hand, accommodating students’ needs and getting them to reflect on skills like writing a first draft likely reduce the chances of misconduct.
This approach requires being comfortable with “false negatives,” students who engage in misconduct — say, having ChatGPT write their papers from scratch — but aren’t caught and disciplined. Here I like the adage “You can lead a horse to water but you can’t make them drink.” Students who engage in misconduct in my classes are likely either struggling with major personal crises or have become irremediably cynical about the value of education. In both cases, they’re beyond my ability to help. That’s a sad and frustrating realization, but it’s one that allows me to devote my time and attention to teaching rather than prosecuting misconduct cases.
Footnotes
A couple years ago, I switched to primarily effort-based grading, and generally only give two grades, A or B.↩︎
“Right, like they’d actually do the reading.” I might need to write a post about this sometime, but I run my upper-division courses off of slides that students create before class meets. This has been very effective at getting them to do the reading.↩︎