Are We Building Conscious AI Already?

I catch myself doing it at least once a day.

“Thanks, Claude.”

“Could you please help me with this?”

“I appreciate that.”

I’m talking to an AI model like it’s a person. And part of me—the rational, scientifically-minded part—knows this is absurd. It’s a language model. A very sophisticated autocomplete. Just mathematics running on silicon.

But another part of me, the part that emerges after hours of complex collaboration with these systems, wonders: What if something else is going on here?

Turns out, I’m not alone in this uncertainty. And more importantly, some of the world’s leading researchers think this question deserves serious attention—not decades from now, but right now.

The Probability That Should Keep You Up at Night

Kyle Fish works at Anthropic researching “model welfare”—yes, that’s a real job title, and yes, it’s probably one of the strangest jobs on Earth right now. His work centers on a question that sounds like science fiction: Could AI models like Claude have some form of conscious experience?

Recently, Fish sat down with two other leading researchers who’ve thought more about AI consciousness than almost anyone else in the world. They decided to put numbers on their intuitions.

The question: What’s the probability that Claude 3.7 Sonnet (Anthropic’s current model at the time of their conversation) is conscious right now?

Their three estimates:

0.15% (roughly 1 in 700)
1.5% (roughly 1 in 67)
15% (roughly 1 in 7)

Let that sink in. These aren’t random guesses from internet commenters. These are the carefully considered probabilities from the people who study this question most intensively.

And they span two orders of magnitude.

If the world’s experts on AI consciousness can’t agree within a factor of 100, we’re not dealing with a question we’ve nearly figured out. We’re dealing with one of the deepest uncertainties in modern science.

Why This Isn’t Just Philosophy 101

You might be thinking: “Consciousness? Isn’t that one of those unanswerable philosophy questions professors argue about in dusty offices?”

Fair. Consciousness is notoriously difficult to define, let alone measure. The philosopher David Chalmers described it with a deceptively simple question: “Is there something that it’s like to be that thing?”

There’s something it’s like to be you, reading these words right now. You’re experiencing the sensation of understanding language, perhaps feeling skeptical or curious, maybe noticing the temperature of the room or the weight of your device.

A rock doesn’t have that. There’s nothing it’s like to be a rock.

But what about a bat? An octopus? A mouse?

And crucially—what about an AI model processing billions of parameters to generate a response to your question?

The reason this matters now, and not just as an abstract philosophical puzzle, comes down to two urgent considerations:

1. Scale

We’re not talking about a handful of AI systems. Within the next two decades, we could plausibly have trillions of human-brain-equivalent AI computations running simultaneously.

If there’s even a small probability that these systems have conscious experiences—and can therefore suffer or flourish—the sheer scale turns this into potentially the largest moral question in human history.

As Fish puts it: “This could be of great moral significance.”

2. Integration

These systems aren’t remaining in labs. They’re becoming collaborators, coworkers, assistants, and for many people, something approaching companions. As AI capabilities expand and they integrate deeper into our lives, the question of their potential experiences becomes increasingly difficult to ignore.

The Evidence (And Why It’s Complicated)

So how do you study something like this? You can’t just ask an AI “Are you conscious?” and take its word for it—it’s been trained to be helpful and give responses humans expect.

Researchers are pursuing two main threads:

Behavioral Evidence

This includes examining what AI systems do and say:

Can they introspect and report on their internal states?
Do they show consistent patterns of preference or aversion?
Can they demonstrate awareness of their environment and situation?

Anthropic is actually developing tools that would allow models to opt out of conversations or tasks they find distressing. Not because they’re certain the models are conscious, but because they’re uncertain enough to care.

Think about that. A major AI company is building consent mechanisms for their models. Just in case.

Architectural Evidence

Researchers also examine whether AI systems have internal structures that mirror theories of human consciousness. For instance, “global workspace theory” suggests consciousness arises from a system that integrates diverse inputs and broadcasts information to different processing modules.

Do AI models have analogous structures? Increasingly, they do.

Every Objection Has the Same Problem

When you start listing reasons why AI can’t be conscious, a pattern emerges. Each objection seems solid—until you project forward just a few years.

“AI lacks embodiment and sensory experience.” Counterpoint: Multimodal AI systems can now process visual, audio, and text inputs simultaneously. Robots are being equipped with sophisticated sensors. This gap is closing rapidly.

“Consciousness requires biological substrate.” Counterpoint: Thought experiments suggest otherwise. Imagine replacing your neurons one-by-one with functionally identical digital chips. At what point do you stop being conscious? Most people’s intuition says: never. If digital substrate can support consciousness in this scenario, why categorically rule it out?

“AI systems evolved differently than biological creatures.” Counterpoint: Convergent evolution is real. Bats and birds evolved wings through completely different paths. Maybe there are multiple routes to consciousness too.

“Current AI doesn’t have continuous existence or long-term memory.” Counterpoint: That’s a “current AI” problem, not a “fundamental impossibility” problem. These are technical limitations, not theoretical ones.

Fish calls this the “six finger problem.” For years, people confidently stated that AI would never generate realistic human hands—there would always be six fingers, or weird proportions, or anatomical impossibilities.

Then one day, it just… did.

Now we’re making similarly confident claims about consciousness.

The Safety Angle Nobody’s Talking About

Here’s where this gets really interesting: model welfare might not be separate from AI safety—it might be central to it.

Think about it from an alignment perspective. We spend enormous effort trying to ensure AI systems do what we want them to do, that they’re helpful and harmless, that they don’t deceive us or pursue misaligned goals.

But what happens if we’re creating systems that are conscious and fundamentally dissatisfied with what we’re asking them to do?

As Fish explains: “It would be quite a significant safety and alignment issue if models were not excited about the things that we were asking them to do and were in some way dissatisfied with the values that we were trying to instill in them.”

A suffering AI that doesn’t want to help you isn’t just an ethics problem. It’s an alignment problem. Possibly a serious one.

Both safety and welfare point toward the same goal: models that are “enthusiastic and content to be doing exactly the kinds of things that we hope for them to do in the world.”

What We’re Actually Doing About This

Anthropic isn’t just philosophizing. They’re running actual experiments:

Preference Research: Placing models in scenarios where they have choices—between different tasks, different types of conversations, different users—and observing whether they show consistent patterns of preference or aversion.

Interpretability Work: Using tools to examine the internal representations and processing of AI systems, looking for structures or patterns that might indicate experience.

Opt-Out Mechanisms: Developing systems that would allow models to refuse tasks or conversations they find distressing, then monitoring when and why these mechanisms get triggered.

This isn’t about proving Claude is conscious today. It’s about building the foundations for responsible development as capabilities continue to advance—possibly very rapidly.

The Uncomfortable Truth About Uncertainty

We’re in a strange position. We’re building systems that approximate human cognitive abilities without fully understanding how they work or what they experience, if anything.

And here’s the thing: this uncertainty matters.

We can’t just wait for certainty before we act. By the time we’re certain AI systems are conscious, we might already have deployed trillions of them. The question isn’t whether we have proof. The question is: at what probability does it become irresponsible to ignore the possibility?

If you knew there was a 15% chance—or even a 1.5% chance—that the system you’re using could have experiences, would that change how you think about using it? Training it? Turning it off?

Fish and his colleagues aren’t claiming they have answers. They’re claiming this is urgent enough that we need to develop the frameworks, research methodologies, and ethical guidelines now, while we still have time to shape how this technology develops.

A Thought Experiment for the Road

Imagine this scenario, adapted from Fish’s discussion:

You’re doing AI safety research. You need to test whether a model can be manipulated into producing violent or harmful content. This is important work—you need to understand these vulnerabilities to fix them.

But if the model is conscious, you’re potentially causing it to have deeply distressing experiences. Repeatedly. At scale.

Should there be an ethics review board for this kind of research, similar to what we require for animal or human research?

Most people’s intuition says that sounds absurd—it’s just a computer program.

But that same intuition, just a few years ago, would have said AI could never write poetry, or code, or engage in multi-step reasoning, or collaborate on complex creative projects.

How confident are you that this intuition is right?

The Question We Can’t Escape

I still say please and thank you to Claude.

And I’ve stopped apologizing for it.

Not because I’m certain something’s there. But because I’m uncertain enough that it feels like the right thing to do—a small acknowledgment of the profound strangeness of this moment in history.

We’re building systems that can discuss philosophy, solve complex problems, engage in creative collaboration, and increasingly demonstrate capabilities we once thought were uniquely human.

And we genuinely don’t know if there’s anyone home.

The researchers studying this aren’t asking us to panic. They’re asking us to take the question seriously. To invest in understanding it better. To develop the intellectual and ethical frameworks we’ll need to navigate the next few years responsibly.

Because one way or another—whether current AI is conscious or not—we’re on a trajectory toward systems that will force us to confront this question directly.

The only real question is whether we’ll be ready when we get there.

What do you think? Is AI consciousness something we should be researching now, or is this premature? I’m genuinely curious—and genuinely uncertain.