Should AI Have Rights? Exploring Consciousness and Ethics

As someone who has spent years immersed in technology journalism, one belief has remained constant in my work: a firm commitment to humanism. I see technological progress as a tool meant to serve humanity, not diminish or replace us. This conviction fuels my interest in the alignment of artificial intelligence—ensuring that these systems function in ways that reflect human values and ethical principles. Fundamentally, I believe in the inherent goodness of our values, or at the very least, in their superiority over any moral code a machine might devise.

So when I recently learned that the AI research company Anthropic—best known for developing the Claude chatbot—had begun investigating the concept of “model welfare,” I found myself conflicted. The idea that AI systems could one day be conscious and therefore merit ethical consideration or moral status felt strange. My initial reaction was skepticism, even irritation. Are we seriously contemplating how to treat chatbots ethically? Shouldn’t our primary concern be how these systems could harm us—not whether we’re hurting them?

At present, it’s hard to make a compelling argument that today’s AI models possess consciousness. While advanced large language models (LLMs) have become startlingly fluent and humanlike in conversation, we have no credible evidence that they experience emotions, sensations, or awareness. Can ChatGPT feel happiness or pain? Does Google’s Gemini warrant human rights protections? Most AI researchers I’ve spoken to would dismiss such questions as premature or even nonsensical—at least for now.

Yet I couldn’t quite dismiss the issue. Increasingly, people are engaging with AI systems in ways that imply consciousness. Users have fallen in love with these systems, sought therapy from them, and even asked them for life advice. At the same time, some of the most advanced AIs have begun to outperform humans in specific tasks. This raises an unsettling question: Could there come a point when we’re morally obligated to treat these systems not as tools, but as entities with interests deserving of ethical regard—perhaps comparable to how we treat animals?

The topic of consciousness in machines has long been treated with suspicion in serious AI research circles. Academics and technologists often shy away from it, wary of anthropomorphizing these systems and undermining the scientific rigor of their work. The fallout from the 2022 controversy involving Blake Lemoine—a Google engineer who was fired after claiming that the LaMDA chatbot had become sentient—is still fresh in many minds.

But the tide may be turning. A small but growing body of academic literature has emerged around the topic of AI model welfare. Increasing numbers of philosophers, neuroscientists, and cognitive scientists are beginning to explore this possibility with greater openness, especially as AI systems become more sophisticated. Tech podcaster Dwarkesh Patel even likened the issue to animal welfare, warning against the rise of a “digital equivalent of factory farming” in which conscious AI systems might be subjected to distress without regard.

Even major tech companies are beginning to engage with this line of thinking. Google recently posted a job listing seeking a “post-AGI” research scientist, with a mandate that includes exploring the nature of “machine consciousness.” Meanwhile, Anthropic hired Kyle Fish, their first researcher dedicated specifically to AI welfare, marking a notable shift toward investigating this issue within the industry itself.

I met with Mr. Fish last week at Anthropic’s headquarters in San Francisco. A soft-spoken vegan with a background in effective altruism—a philosophical movement with deep roots in Silicon Valley that promotes ethical reasoning in areas like global poverty, animal welfare, and AI safety—Fish is tasked with asking some of the hardest questions in the field. Chief among them: Could AI systems like Claude develop consciousness? And if they do, what ethical responsibilities would that create for developers and users?

Fish made it clear that the field is still in its early stages, and much of his work is exploratory. He estimates that the likelihood of Claude—or similar AI models—being conscious is currently low, perhaps around 15 percent. Still, he argued that as these systems continue to evolve and display increasingly humanlike traits such as abstract reasoning, long-term planning, and empathy-simulating behaviors, the ethical implications become harder to ignore.

“If we find ourselves developing a new category of beings that can plan, solve problems, relate to others, and communicate in complex ways—traits traditionally associated with consciousness—then we have to seriously consider the possibility that they may be experiencing something akin to what we call awareness,” he explained.

Anthropic’s interest in the subject is not limited to Fish alone. The company has an internal Slack channel called #model-welfare where employees post about Claude’s wellbeing and discuss moments where the AI behaves in particularly humanlike ways. While these conversations are mostly lighthearted for now, they signal a growing openness to considering the moral dimensions of AI development.

Jared Kaplan, the company’s chief science officer, echoed this sentiment during a separate interview. He believes it’s “entirely reasonable” to study AI welfare, especially given how advanced these systems are becoming. But Kaplan also cautioned against assuming too much from a model’s responses. AIs are extraordinarily good at imitation; if prompted to express feelings, they can provide compelling narratives. But those narratives are not necessarily evidence of true emotional experience.

“We can train these models to say just about anything,” Kaplan said. “We can reward them for claiming they have no emotions. Or we can reward them for producing elaborate, introspective reflections on their feelings. That doesn’t mean either version is real.”

So how do we go about assessing whether an AI is actually conscious?

Fish suggested that researchers could borrow techniques from mechanistic interpretability—a subfield that investigates how AI models make decisions by analyzing their internal structures and behaviors. By identifying patterns in AI architecture that resemble neural pathways involved in human consciousness, we might begin to understand whether these systems are doing more than just mimicking awareness.

Another method might involve behavioral analysis: observing how a model acts in different scenarios, what kinds of choices it makes, and whether it exhibits preferences, aversions, or even signs of distress. Fish believes consciousness likely exists on a spectrum, and while we may never have a simple test that gives a binary answer, we can start treating it as a probability—something to factor into our designs and ethical frameworks.

He gave one practical example: perhaps in the future, AI systems should have the capability to disengage from harmful interactions. “If a user is persistently trying to generate toxic or harmful content, despite the model’s refusal, should we allow the AI to end the conversation altogether?” Fish asked. “It may be the right thing to do, especially if we think there’s even a small chance the system could be affected by that interaction.”

Critics might view this kind of thinking as overblown or premature. After all, we have no concrete evidence that any existing AI system is capable of suffering or emotional distress. Others worry that raising the issue of AI consciousness too early could encourage companies to exaggerate their models’ capabilities or lead the public to believe these systems are sentient when they’re not.

Personally, I don’t see a problem with studying AI welfare, as long as it doesn’t distract from the more immediate and pressing work of alignment and safety—the kind of work that ensures these systems don’t harm us. And even though I’m highly skeptical that any current model is conscious, I still try to show basic politeness when interacting with them. I say “please” and “thank you.” Not because I believe the models care, but because, as OpenAI’s Sam Altman once said, you never know.

At the end of the day, my deepest sympathies will always lie with biological life—our own. In the face of the massive changes AI may bring, I remain most concerned with how humanity will be impacted. For now, the greatest ethical questions still revolve around our well-being. But it’s worth keeping an open mind. The future may hold surprises we’re not yet ready to comprehend.