AI, ChatGPT, and the Coming Revolution
[note, for an update in Nov 2023, please click here]
This week (early Jan. 2023) I’ve been experimenting with and reading/having conversations about ChatGPT, a new AI program that is currently available free in a test version (soon to require a paid subscription, I’m sure). Artificial Intelligence is not new, but this example has more expansive datasets and can do a lot more…and ChatGPT is surely the precursor to even faster and more capacious programs that are going to change the way we work and the way we learn. Seriously—there’s a revolution coming!
Let’s start with what it does and doesn’t do for its users, and then move to how that affects us who are (English) teachers. I’m going to have to split this into two posts. For teaching tips, skip straight to the second one.
ChatGPT can have a conversation with you—it’s a chat bot, after all—but that’s not how we’re mainly going to use it and other programs like it. ChatGPT can write instant essays and reports, do instant non-academic research or an instant fake of academic research, summarize articles, create quizzes, write computer code, imitate famous writers, and give feedback on your writing. You can ask it to write a short essay on the narrator of Jane Austen’s Northanger Abbey that’s under 1000 words and includes five quotations from Austen and three quotations from published articles, and it begins scrolling onto your screen in under five seconds. ChatGPT does not create any original ideas: the essay will present information about Austen’s playful narrator clearly, blandly, and concisely, in correct but uninteresting academic English. For example, here’s an intro paragraph:
The narrator of Jane Austen's Northanger Abbey is a complex and multifaceted character whose voice guides the reader through the novel's events and themes. Through the use of irony, humor, and insight, the narrator plays a crucial role in shaping the reader's understanding of the novel's characters and their motivations.
(You can see two essays I generated on this page.) ChatGPT’s conclusion paragraph will exactly repeat its unoriginal thesis. It will usually be a C-range English paper—in other disciplines, where written assignments don’t require analysis or original thinking, I hear the chat bot can get As. Except that in this case it quoted an article by Jocelyn Harris that doesn’t exist and gave it coming from a journal where there’s an article by someone else on E.L. Doctorow, not Jane Austen, on those pages—so the essay gets an F for falsifying evidence and is reported to the Registrar’s office! Other than that, there’s likely also some undetectable plagiarism, because ChatGPT’s bland unoriginal ideas are paraphrased from existing texts—but it will only acknowledge the sources of quotations, not its paraphrases of what it considers common knowledge.* If you ask it to generate a second paper on the same topic, you get mostly the same ideas but not the same phrases and sentences expressing them, so someone else can make the same request and not end up with exactly the same paper. Close enough to give the marker headaches. And, of course, there’s no need for the student essay producer to read Austen’s novel. One big drawback is that the free test version doesn't seem to give the user more than 1000 words or so.
As a side note, when I asked ChatGPT to write an essay on Rebecca Stead’s When You Reach Me, it was able to produce quotations from the novel even though this is a copyrighted text that should not be available full-text online! This says to me that the program also mines sites that pay no attention to copyright laws.
If you paste in a draft of an essay that you wrote (I used an old student paper) and ask ChatGPT to give you feedback on your writing, it will likely comment on your organization and clarity, be mostly useful, and say you’re doing well. If you ask for feedback on grammar, it will give you bland encouragement and a couple of reasonable suggestions (vary your sentence structure, etc.), not terribly useful. If you paste in a draft of an essay and ask it to identify all the grammar errors, it will give more useful specifics—tell you where the errors are and what the corrections should be. But most students are going to use this function as an instant Grammarly and just ask it to correct all their grammar errors without their looking at or understanding them.
If you ask the bot to summarize a published article, it won’t be even as informative as the author’s abstract, or at least my attempt (at the bottom of this page) was a sad failure. If you ask it to explain a tricky theoretical term, it will mine what I assume to be student help sites, Reddit, and professors’ public webpages and produce a simplistic but satisfactory explanation.
If you ask it to produce a 500-word personal essay on educational goals, from the point of view, say, of someone who is Korean-Canadian, born in 2004, wanting to be a lawyer, and raised by a single father, it will make up something blandly aspiring you could use in an application to a graduate program. Likely unsuccessfully.
I think that if you paste in a lot of data, you can ask ChatGPT to organize it, summarize it, and present it in a report, but I haven’t tried this myself. I’ve heard that there are employers who already ask their employees to use AI to write their reports—with immediate deadlines. No longer is it “have this to me by Friday”; it’s now “have this to me by 2:00 pm.” There will be no time to reread, check, revise, or proofread, as productivity takes precedence over quality of product. That scares me! A friend of mine, when I mentioned this to her, talked about how today’s generation wants a thirty-hour work week at the same salary, and this will be a boon. But my pessimistic nature went to visions of one person doing four people’s jobs and being on call 24/7, at lower pay because there’s less expertise required. Perhaps the real outcome will be somewhere in the middle. Here’s hoping.
I hear that ChatGPT is pretty good both at producing code from a set of parameters and at deducing parameters from code, but I can’t give personal testimony on that. [I've now heard secondhand that it doesn't write good code, either.]
In summary, ChatGPT is strong on plausibly presenting spurious sources (in fact, it produced me a list of five articles for my essay on Austen, none of which exists!); incapable of original ideas but good at mimicking, paraphrasing, and falsifying; and is good with English grammar of the white middle-class North American academic variety, but it writes in a completely uninteresting style. It’s a tool, one that our students are going to be using, and we’d better figure out how to make them ChatGPT-literate as soon as we can. So, on to part two, on how this is going to create a revolution in teaching.
********************************
*In a later experiment I asked ChatGPT to write the same essay but then list the websites from which it got information. Turns out, the bland C-range papers were paraphrased from gradesaver, shmoop, cliffnotes, and sparknotes!
Then I tried an Engineering essay and got semi-legit websites & a History essay that used legit govt websites. But no journal articles, even from freely available online websites. I went back to English and tried two more topics, on Shakespeare and Faulkner...and ChapGPT used gradesaver and shmoop and sparknotes again.
When I asked ChatGPT why it hadn't used articles from a specific freely available academic journal on Jane Austen, it told me it had no access to the internet and I should look at that website myself (?!).
My colleague Leanne Ramer suggested Perplexity to me: it appears not to write essays, but when I asked it to suggest sources on Austen's narrator it provided several legit ones in addition to shmoop.