You have /5 articles left.
Sign up for a free account or log in.
Occasional appearances to the contrary, I am not a generative AI refuser.
What I am is a skeptic and (perhaps) resister who, when evaluating possible use of the technology, first looks at what is important about the human activity and experience that the large language model may be substituting for.
Which brings me to some things I have to say about peer review.
There are two categories of what we call “peer review.” The first is the system of evaluation attached to scholarly publishing, and just about a year ago I declared that this should be one of our clear nonstarters in education, given that this technology cannot read, think, feel, reason or communicate with intention.
I understand that there have been some experiments showing that LLM review may have similar outcomes to human peer review, but who cares? Either scholarship, research and publishing are human enterprises or they aren’t. A large language model is in no sense a peer, so any publication that chooses to use these tools as substitutes for peer review should be transparent that they’ve done so and drop the pretense that they still participate in a system of peer review.
They are choosing to accept a simulation in place of the actual thing.
The second category of peer review is a frequently deployed teaching practice where students exchange drafts of a writing assignment with one of their colleagues and then evaluate it according to criteria established by the instructor. In my view, a better name for this practice is “peer response,” which is the term I’ll be using from now on.
There is a common misconception, one that I had myself even years into my work teaching writing, that the primary purpose of peer response was to get feedback on your draft that could be incorporated into revision.
This is one possible purpose and is indeed the one that students assume is the primary purpose, something I know because I would have students complain (privately, thank goodness) that a classmate gave them little or unhelpful feedback.
But the core purpose of peer response is not to receive feedback, but to give it. This is what I would tell students prefacing an in-class peer response workshop:
“Everyone in this room is attempting to solve the same (or at least very very similar) writing problem, albeit in a unique way that reflects their own sensibilities. We have the parameters of the problem—the message, the audience, the purpose—and we have an attempt at solving the problem in the form of a draft.
“Your role as the responder is to do your best to consider the draft relative to the parameters of the problem. Is the document fulfilling the needs of its audience relative to the occasion of its writing?
“While you are doing this, you should find yourself simultaneously reflecting on your own effort regarding this particular writing problem. After you’ve each read the other’s draft, and responded according to the specific criteria below, you will have a conversation about this writing problem and your attempts to solve it.”
The specific criteria would hark back to the assignment itself and was ideally a reminder and reinforcer of the core of the writing problem.
This approach had multiple benefits: One, it helped put students in a good mindset to pay attention to the big picture goals of the experience. Pro forma peer response would often have students focused on surface-level features like syntactical correctness rather than the deeper questions of a rhetorical situation. I wanted to break that mentality.
Two, foregrounding student thinking about their own work while they read the work of another actually made the feedback better and more relevant. If they felt their own approach was superior, they would have to articulate a facet of their work to their peer. If they thought the peer’s work was better, they may have something they could make use of in their own work.
Third, this peer-response process provided fodder for a subsequent and ongoing conversation about their writing. The peer-response process was a vehicle to provide the energy for that conversation where new insights could be discovered. I can testify that it works, because how couldn’t it?
Contrast this to reaching out to a large language model that cannot read, think, feel or communicate with intention, but which can produce a simulation of a peer response—or even an “expert” response—on a piece of writing.
An exchange where one side is responding on the basis of linguistic and syntactical probabilities is not the same as a conversation with another human who has read a piece of writing and can consider it in the context of a particular rhetorical situation. The simulation is not the thing itself.
It just isn’t.
I have had numerous people tell me that they understand this distinction and yet they find feedback from LLMs helpful in developing their writing. I do not doubt this, but I nonetheless must insist that there is a difference between that and a peer response, the same way me hitting a tennis ball at a wall when I was kid was not the same as playing tennis. Anyone who says that a large language model is their “thought partner” must know that all the thoughts in that partnership are their own.
I am concerned about habituating developing writers to a writing process that removes one of the opportunities for reflecting on their writing and the writing of others and substituting the response of the nonthinking, nonfeeling, unable-to-communicate-with-intent technology.
Well, I am told, sometimes teachers and peers aren’t available, and they can always go to a large language model for feedback.
Again, I am not a refuser, but I start with the human-oriented questions. Why are peers and/or teachers not available and should we do something about that?
And have we demonstrated an actual benefit of on-demand LLM feedback to the development of a robust writing practice? Maybe we learn more when feedback is not on demand and we have to sit with a particular problem for a period of time.
I’m afraid that embracing AI feedback is an example of privileging schooling over learning, a streamlined process toward a grade, as opposed to the productive struggle that results in learning. Obviously, the use of a large language model does not by definition have this negative effect, but I very rarely see interventions and use of the technology in contexts of teaching writing that does not carry this risk.
What we call generative AI, of which large language models are a subset, is really a form of automation, and while automation can deliver many benefits, it will never be human.
