Does Claude Speak Its Own Language? Three Aspects Of AI Psychology

Toronto, Canada - August 22, 2024: Popular AI virtual assistant apps on an Apple iPhone: ChatGPT, ... More Claude, Gemini, Copilot, Perplexity, and Poe.

Exploring AI's Sentience

For a long time, at least by AI terms, people have been talking about the black box problem with LLMs. It’s in our human nature to want to understand something fully – to see the inner workings of a machine, whether it’s a clock or a calculator or a robot. But in today’s world, there are things we just can’t observe fully. To put it another way, some complex systems tend to defy analysis.

Insights on Claude's Reasoning Capability

I was reading through this Substack piece where Alexander Kruel makes what is a staggering set of insights on Claude and its reasoning capability, quoting some resources around the model from insiders written by a large team of authors.

Take a look at this bullet point from his extrapolation of the LLM’s capability: “Claude can speak dozens of languages – (what language, if any, is it in its head?) Before I get into specific points on LLMs and language syntax, I want to point out that the paper’s author concedes that in exploring AI’s sentience this way, they’re piggybacking on traditional neuroscience and the study of the human brain.

To wit (again, from over a dozen authors): “We take inspiration from the field of neuroscience, which has long studied the messy insides of thinking organisms, and try to build a kind of AI microscope that will let us identify patterns of activity and flows of information. There are limits to what you can learn just by talking to an AI model—after all, humans (even neuroscientists) don't know all the details of how our own brains work. So we look inside.”

I thought this was apt: yes, the organic human brain is “messy” – for sure. So when you apply this AI microscope to the actions of a model, what do you see? Here are three things that the authors point out that I think are integral to understanding how these models work.

Understanding Claude's Workings

First, some of those who have worked on this project contend, after looking at Claude 3.5 Haiku, that Claude has its own universal “language of thought” that combines multiple world languages. “We show this by translating simple sentences into multiple languages and tracing the overlap in how Claude processes them,” they write. This, in itself, is quite an epiphany in terms of how these models think. Many of us would just assume that they mostly think in English. But this practical exploration shows otherwise.

For example, the research tracks the use of the words “big” or “large” in English, Chinese and French, and makes comparisons, showing how aspects of the model’s work produce those overlapping patterns. Another very central insight here is that the human scientists can’t see everything that Claude is doing, only a portion.

“We recognize the limitations of our current approach,” they write. “Even on short, simple prompts, our method only captures a fraction of the total computation performed by Claude, and the mechanisms we do see may have some artifacts based on our tools which don't reflect what is going on in the underlying model.”

There’s further explanation: “It currently takes a few hours of human effort to understand the circuits we see, even on prompts with only tens of words. To scale to the thousands of words supporting the complex thinking chains used by modern models, we will need to improve both the method and (perhaps with AI assistance) how we make sense of what we see with it.”

Why is that? Why can’t the humans just read the model’s output? Well, one way to explain it is that this isn’t linear programming. In linear programming, you can observe on a step-by-step basis, and you can see everything that the machine is doing. But as is pointed out in most of the conferences and lectures I’ve been to in the last three or four years, we’re beyond that point now.

Observing an LLM in Action

Here’s a point that’s less related to syntax, and more related to what you’re studying when you observe an LLM in action. As a prime example, the authors note that the model typically “avoids” certain topics or avenues of conversation, declining to answer questions unless something inhibits its default response. We know that the machines are often “programmed” for safety and away from violent or objectionable paths, but again, this seems to go far beyond that, into a machine’s “preference” for declining prompts.

Learning what that “something” would be, what the stimulus is, would help to understand when the machine chooses to answer a human question. That by itself is going to greatly advance the study of artificial neurology. It’s worth pointing out once again that just like we can’t fully understand what the AI is doing cognitively, we don’t fully understand what our brains are doing cognitively, either.

So there’s a sort of parity there between looking at human thought and AI thought. Maybe it’s fitting that we can’t fully understand what all of those digital neurons are doing. What we can do is observe and get information through hands-on practice from the outside, so to speak. All of this informs what people are doing right now with revolutionary models. As we discover more about their capabilities, being able to track the “how” of AI thought is going to be valuable.