I think it would be safest for us to assume that AI chatbots will become much smarter over the next few years, both in terms of the sources they're capable of accessing and what they do with them.
I enjoyed all the subtle and less-subtle digs employed in this conversation ("stochastic parrot" is so funny, partly because there's more irony in it than I think was intended), but let's look ahead to the future we can anticipate, not the one we may wish for. Doesn't "stochastic parroting" describe 99% of all human learning, too, from babies playing peekaboo to academics doing lit reviews? Obviously that last 1%, the anticipating-planning-innovating part, is really important. But I wouldn't look at AI figuring out 99% of a complex task across mere decades of trying, and conclude "oh, at least our last 1% advantage is safe".
So, I would advocate let's round up and assume that the AI can in fact do assignments, in any field, at an A- level at least - and plan our learning assessments around that assumption. In that way we don't have to start from scratch in a few years.
And would it be so bad if the AI can do referee reports? 😉
(Yes, of course it would be, if they were used to reject my papers... silly joke. But let's anticipate that anyway.)
Lucas
Sent: January 22, 2023 10:19:51 PM
To: Nicky Didicher; academic-discussion@sfu.ca
Subject: Re: more ChatGPT experiments, on its sources of data
I don't know the details of how ChatGPT does it, but similar models are created by doing things like scanning all Wikipedia articles, plus all posts on reddit.com (and for any high-scoring post that links to a website, also scanning that website), and other sources. I'd guess it writes the way it does because it saw vastly more examples of mediocre text, and comparatively few examples of high-quality text.
Toby
--
Toby Donaldson
School of Computing Science
Simon Fraser University
Sent: Sunday, January 22, 2023 9:07:58 PM
To: academic-discussion@sfu.ca
Subject: more ChatGPT experiments, on its sources of data
Hi all,
It occurred to me that a way I could get around ChatGPT’s not saying where its paraphrases come from (so you can check its sources) might be to ask for an essay and then say “ and follow it with a list of all the websites from which you took information for the essay.”
So, I went back to my Austen essay topic and it did tell me in a “sources” list where it was getting its information and its quotations: gradesaver, shmoop, cliffnotes, and sparknotes! That explains why all the English lit essays I’ve generated have been C-range. There are in fact plenty of more academic websites not behind paywalls that it could have used (the journal Persuasions, for example), but it went only for dreck by exploited student writers. I wondered why.
I next asked it to find three articles on Northanger Abbey from that online journal, knowing that a google search will in fact bring them up and link you to them. This is what ChatGPT said:
“ I'm sorry, but as a language model AI, I do not have access to the internet and I am not able to find articles from specific journals. However, I can tell you that Persuasions is the journal of the Jane Austen Society of North America (JASNA), which is a scholarly publication that features articles and essays about Jane Austen and her works. It is likely that the journal has published several articles about Northanger Abbey, as it is one of Austen's most popular novels. If you want to find more information about this, you can visit the JASNA website and look for the Persuasions Journal.”
It doesn’t have access to the internet? Then how did it find shmoop and gradesaver? Really, anyone who understands that,I’d welcome an explanation.
I next tried an essay about the uses of solenoid circuits with sources included, and the AI gathered its info from some websites I’m guessing to be ok but not great: engineersedge, allaboutcircuits, machinedesign. They all have ads on their pages but they don’t look as shady as shmoop.
And I tried an essay on the Canadian constitution with a list of the websites it used, and got the
Canadian encyclopedia, the govt of Canada, a couple of parliamentary papers, and the CBC. Pretty good.
This might explain why ChatGPT essays can get better marks in other disciplines than mine?
Nicky
Nicky Didicher, English,
Simon Fraser University,
778-782-4337, didicher@sfu.ca
pronouns she/elle
Reduce, Organize, Communicate, be Kind!
At Simon Fraser University, we live and work on unceded traditional territories of the Coast Salish peoples of the Səl̓ilw̓ətaʔɬ (Tsleil-Waututh),
Sḵwx̱wú7mesh (Squamish), and xʷməθkʷəy̓əm (Musqueam) Nations.
I am from British and German settler peoples. I was born in North Bay, Ontario, on the territory of
the Nipissing First Nation Anishinaabe, covered by the Robinson Huron and Upper Canada treaties. I want to acknowledge and make visible the harm that settler peoples have done and continue to do to the First Nations, and I hope that there can be a reconciliation
that actually recognizes their land rights. For a start, we SO need to change the name of this university!