The AI Paperclip Apocalypse And Superintelligence Maximizing Us Out Of Existence

The Paperclip Maximizer Problem and the Future of AI and Humanity

Taking a deep dive into the famous paperclip maximizer problem and the future of AI and humanity. In today’s column, I do some myth-breaking by examining a quite famous thought experiment in the AI field involving paperclips. I will go ahead and explain why paperclips have become a keystone in debates about what will happen when or if we attain artificial general intelligence (AGI) and/or artificial superintelligence (ASI). I’ve added some new twists to the famed topic so that even those already familiar with the paperclip controversy will undoubtedly find the revisited kit and kaboodle of keen interest. Let’s talk about it. This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities.

AGI and ASI

First, some overarching background about AGI and ASI. There is a great deal of research going on to significantly advance modern-era conventional AI. The general goal is to either reach artificial general intelligence (AGI) or maybe even the outstretched possibility of achieving artificial superintelligence (ASI). AGI is AI that is considered on par with human intellect and can seemingly match our intelligence. ASI is AI that has gone beyond human intellect and would be superior in many if not all feasible ways. The idea is that ASI would be able to run circles around humans by outthinking us at every turn.

AI insiders are pretty much divided into two major camps right now about the impacts of reaching AGI or ASI. One camp consists of the AI doomers. They are predicting that AGI or ASI will seek to wipe out humanity. Some refer to this as “P(doom),” which means the probability of doom, or that AI zonks us entirely, also known as the existential risk of AI. The other camp entails the so-called AI accelerationists. They tend to contend that advanced AI, namely AGI or ASI, is going to solve humanity’s problems. Cure cancer, yes indeed. Overcome world hunger, absolutely. We will see immense economic gains, liberating people from the drudgery of daily toils. AI will work hand-in-hand with humans. This benevolent AI is not going to usurp humanity. AI of this kind will be the last invention humans have ever made, but that’s good in the sense that AI will invent things we never could have envisioned.

The Paperclip Maximizer Thought Experiment

In 2003, a philosopher named Nick Bostrom famously proposed a thought experiment about the future of AI. The idea is quite simple. Suppose that we ask a highly advanced AI to manufacture paperclips. Simply make plain old paperclips. Easy-peasy. This should be a piece of cake if the AI has access to manufacturing plants and has control over those facilities. The twist offered in this tale is that this dutiful and obedient AI proceeds to gobble up all the available resources on earth to maximally achieve this goal. For example, since paperclip making requires steel to make the paperclips, it makes abundant sense for the AI to try and route all ships, trucks, and other transports that are hauling steel to come straightaway to the paperclip factories. Eventually, humankind is harmed or possibly completely wiped out by AI to make as many paperclips as possible.

Please note that the AI is not intentionally seeking to destroy humankind. Assume that the AI is merely carrying out the orders given by humans. In that sense, this AI is not evil, it is not aiming to undercut humanity, and in fact, it seems like it is obeying humans to the umpteenth degree.

The same premises have been used in a variety of similar scenarios. The famous AI scientist who co-founded the MIT AI lab, Marvin Minsky, wondered what would happen if AI was assigned the task of solving the mathematically legendary Riemann hypothesis. There is no known solution but perhaps AI could computationally figure one out. How might this proceed? The AI would potentially take over all computers everywhere and devote them exclusively to this challenging task.

Instrumental Goals and AI Behavior

A crucial element of these thought experiments is that we potentially face an existential risk due to AI that goes overboard, rather than only due to an AI that shifts gears into evil-doing mode. Humans might give the most innocent and harmless tasks to AI, and then, bam, we get clobbered by a myopic AI that willingly follows our orders beyond our worst nightmares.

Another crucial component is dogmatic blind devotion to a single over-arching goal by AI. Each of the tales provides a stated goal, such as maximally making paperclips or calculating pi to an infinite number of digits in length. The AI then construes that goal as surpassing anything else that the AI might have been told to do. The AI focuses on paperclips or the digits of pi. Nothing else matters.

The aim to maximally make paperclips would be referred to as an end goal or terminal goal. The sub-goals that support that end goal are referred to as instrumental goals. Getting steel and keeping electricity flowing are instrumental goals that support the end goal of maximally making paperclips.

Generative AI Response to the Paperclip Maximizer Scenario

The comments by modern-era generative AI are somewhat reassuring, but let’s not delude ourselves into thinking that we are in the clear. We are not in the clear. Those are pretty words and sound right. It doesn’t mean that the AI would act in the manner portrayed. We also cannot say for sure how AGI will react in contrast to contemporary AI. The same holds for ASI. The double trouble with ASI is that since we aren’t superhuman, we can’t likely anticipate what a superhuman AI is going to do or what thinking it can achieve. All bets are off.

The crux is that we seem unlikely to get clobbered by the paperclip maximizer or the pi-generating pal, but that doesn’t negate the zillions of other ways that AGI or ASI might end up being an existential risk. It simply won’t be the knife in the ballroom by the butler. There are though enumerable other options available, sorry to say.

HONESTAI ANALYSIS

A few final quotes to conclude for now. Our efforts to advance AI should proceed with a basic aim exhibited by this famous remark by George Bernard Shaw: “Those who can’t change their minds can’t change anything.” We are hopefully devising AI that will be willing to change its mind, as it were when the circumstances warrant. That includes overturning a myopic all-encompassing goal of building an endless supply of paperclips. We ostensibly want to have AI that can change its mind, but not willy-nilly. In the end, AGI and ASI will hopefully be like porridge that is neither too hot nor too cold. We want it to be just right.