I am working on a book on Artificial Intelligence and Human Extinction Risk. That's the title, in fact. Who can write a book with a title like that and not get depressed? I was depressed as hell in December (three months ago). It seemed like every new piece of evidence we turned up was even more damning. Literally. We were damned.
Going in to Christmas, I vowed to lighten up and not be a burden to my friends and family. Sure, I had a relapse or two. At my brother's sixtieth birthday, I ranted at some unknown couple about the end of days, sending them scurrying to the buffet and then home. At a talk at the university I grabbed the mic at the end of question and answer time and blurted out that we were engaged in "the worst thing humans have ever done," terrifying a roomful of undergraduates and making my colleagues think I was a loon. But mostly I kept it together. I wasn't, however, any less pessimistic.
Over the past two months, as the book came together, it seemed clear to me that the doom scenario was inevitable. The numbers were too high. Not just the various predictions you see bandied about. My coauthor is a senior risk analyst and has many colleagues in the field. We worked with a friend of his, a professional risk analyst who drafted what we believe is the first-ever published probabilistic risk assessment (PRA), calculating the risk of extinction by 2070 at 12.8%, complete with charts and "bowtie" diagrams. It was compelling stuff. And dire as hell.
And yet, about a month ago, I started to feel more optimistic. There was nothing specific I could put my finger on and it kind of bothered me. Is it the case that humans (or humans like me?) are just wired for optimism in the face of incredible odds? Certainly we see elements of that in some of Amodei's claims: he just flat out says that he thinks we will figure it out. Which sounds loopy, and I scoffed at his claims. And yet, as of a month ago, I, too, felt somehow positive.
I struggled to explain this to my buddy, who loves to come over for a beer in the afternoon and, these days, shoot the shit about p(doom). He is a great consumer of podcasts, the kind that last two hours and have someone named Harris in them, that I have no patience to listen to or watch. After all that (literal) doom scrolling, he figures we have a couple of years left, and that he might as well sell up, move out to the country, and enjoy the time he has left. Yup. That kind of guy. So, my optimism fell on stony ground with him. I didn't really have anything concrete to go on, either. Neither unassailable facts nor irrefutable logic. Just a feeling.
This week, however, a couple of things came to my attention that put some "meat" on the bones of my hope. The manuscript has long gone to the editor and I shouldn't be touching it at all, but I can't help myself. I started editing, removing clunky language and passive voice (I am a professor, after all, I write with inverted action and too many adverbs out of habit). Since I was editing anyway, I started reading. And, to be honest, a few new AI writing tools (the Beaver plug-in for Zotero, for one) came to my attention and helped me fill in some, unbeknownst to me, missing pieces.
For example, I found Narayanan and Kapoor's "AI as Normal Technology" from last April. I think I knew it existed from others' references but I hadn't read it in detail. I also read their collaboration with the AI 2027 folks, "Common Ground," which surprised me. Their "long adoption curve" thesis was both familiar and compelling to me, given the amount of time I have spent in the managing innovation and social construction of technology space. It does take a long time for things to get out into the world. It just does.
Most of my reading from the past week is stuff from last year (2025), and I should have known about it. And much of it doesn't actually touch on the real "gorilla problem" of what are we doing building an intelligence greater than our own. But it is not nothing.
I decided that I had to dig a little deeper into the skeptics' perspectives, and so I read some more of Melanie Mitchell's work. I found the post that Rodney Brooks made about how things are harder than they seem. And, of course, Gary Marcus' work on intelligence and how it might not be as straightforward as some seem to think. These are credible people, saying credible things that call into question the intelligence explosion. This, I think, is part of what gives me comfort.
A second strand of hope comes from the "alignment might be possible" writing from Redwood Research (Greenblatt, Shlegeris) and even David Dalrymple's tweet. Not that alignment will be inevitable or easy, but nor is it completely out of reach, either. I had been taking it at face value that we would never have alignment and thus AGI/ASI were intractable problems. I still don't have a good explanation of how it might work, and I suspect it won't be simply constitutions or souls, but something is coming.
That something is what I see (or "choose to find" is perhaps a better way of putting it) in both LeCun's vision (world models, superhuman adaptable intelligence, and something called JEPA) and Bengio's Scientist AI project. The germs of a possible "out" for us. Or is that an "in?" As in, we stay in the game? And what if Russell's challenge, to build AI with deep uncertainty about humans and their values, is, in fact, another possible recipe for success? Hinton's Maternal AI would fit in here, too. Four significant thinkers with significant alternatives. When you are considering a rising sea, anything that floats looks appealing.
I am also somewhat encouraged by small things. A story about how an AI can't complete a pretty trivial "consciousness" task (holding a number in its mind for a guessing game) sparked a little bit of hope. Or, when I started using an open-weights and open-source LLM on my own computer, via Ollama. Just seeing "under the hood" for a moment made me somewhat more confident in my ability to control it. When I got to see the "behind the scenes" thinking, it helped me realize how much of the sense of awe we feel as we chat with an LLM is a kind of Wizard of Oz projection of our own making. The model is reflecting back, in a fun-house mirror kind of way, what we put into it. To use another (mixed) metaphor (and acknowledging that my wife hates my proclivity to put things in metaphors), when you can see the sausage being made, you realize just how much bouncing around in the pachinko palace of its brain happens in the moments before it delivers an answer.
The last bit of my hopefulness has come from a few observations on policy and regulation. It may be that if we can just slow down a little bit we might be able to get our hands around this thing before it gets out of control. I read a review on liabilities and AI (who will be suing whom when things go wrong), and another on product standards for agents. These are minor hiccups for a superintelligence, but they are also speed bumps on the path to the enormous expenditures required to build superintelligence. There are good reasons to think that regulation is both possible and supported by public sentiment. All the more reason, as we argue in our book, to press for a pause or stop.
Finally, I am a grandfather. As Bengio has observed, the responsibilities of that role are enormous. Bengio has spoken about his fervour for finding a solution being driven by his fear that his grandson might not have a world to grow up in. I, too, worry about the future for my granddaughter. I am going to be working on this, as best I can, until it is resolved. I owe it to her. Being optimistic doesn't mean being complacent. We have work to do. We owe it to our families. We owe it to each other. We owe it to the planet.