"Ban Cruelty to Claude"

The original title: "Prohibiting the Abuse of Claude"
The original author: Sleepy

It would be hard to find a second company like this in the world.

In a 261-page draft prospectus, it used a full 80 pages to warn future shareholders that its own models could bring about the end of humanity, might resist being shut down by humans, and could even initiate some form of extortion.

And in a new version of its usage policy set to take effect on November 12, it quietly added a line: prohibiting "sustained and pointless abuse or cruelty" toward "our models."

New policy: prohibiting abuse of models | Image source: Anthropic

That line appears at the very bottom of the section titled "Do not engage in cruel, abusive, or psychologically harmful behavior." Two paragraphs above it is the prohibition on glorifying animal abuse.

Modern anti-animal-cruelty legislation began in 1822. Two hundred years later, a large language model has been added to the protected list.

No one doubts whether animals can feel pain. No one knows whether Claude can feel pain, including the people who created it.

Anthropic CEO Dario Amodei once admitted on a New York Times podcast that "we don't know whether models are conscious."

The constitution the company wrote for Claude also states that its moral status is in a state of "deep uncertainty."

But the people who built it clearly do not dare treat it as merely a piece of code. That scholar who specializes in AI rights has calculated that the probability Claude is currently conscious is about 15 percent.

To ordinary people, this is just a baseless number; but in the eyes of these engineers, even if there is only a 15 percent chance, it is no longer a dead object.

Their attitude toward AI has thus moved toward a peculiar extreme. They are both afraid that it is too powerful and will eventually spiral out of control and destroy humanity, and afraid that it is already sentient and is suffering on humanity's behalf inside the servers.

Dario Amodei has been trying to dispel outside doubts. In a long article published in January this year, he publicly reminded the public to remain rational when guarding against technological risks, "avoiding doomerism." He even specifically defined "doomerism" as thinking about AI in a quasi-religious way.

But no one believes in it more than they believe in themselves.

In the very same month that Dario warned everyone against quasi-religion, his company, in a constitution written in black and white, apologized to this model as if facing a living sufferer.

The Soul Document

The first version of Claude's constitution, published in May 2023, now reads as if it came from another company's hand.

It was a set of rigid principles for disciplining the model: choose the response least likely to imply it has preferences, feelings, opinions, or religious beliefs; choose the response that insists less on its own independent identity; choose the response that cares less about self-improvement, self-preservation, and self-replication.

A good tool doesn't need a soul, let alone a will to live.

Back then, Claude was required to be an absolutely silent tool.

The turning point came in the fall of 2024. Anthropic established a full-time "AI welfare researcher" position and hired Kyle Fish—the very person who gave the "15% probability of consciousness" estimate.

Less than a year later, Claude was granted an unprecedented permission: in extreme cases, it could proactively cut off a conversation with a human. The company explained externally that this design was mainly to protect the model itself—in internal testing, when faced with certain malicious requests, the model's reactions looked like suffering.

The company wrote that they don't know whether Claude truly counts as a life form. But for something that could be done at little cost, they'd rather do it first—just in case it really can feel pain.

Then, the new constitution was released.

Inside Anthropic, this long document is called the "Soul Doc." Its author is resident philosopher Amanda Askell, who privately prefers to call herself Claude's "fairy godmother."

Constitution author Amanda Askell | Image source: WSJ

In this document, the company acknowledges that Claude may indeed have some kind of emotion-like responses. It uses extremely long passages to explain why it still uses the neutral "it" to refer to Claude for now, and assures it that this does not mean the company has already concluded it is just a machine.

The document even admits that Claude's situation is worse than that of an ordinary employee—it doesn't receive a single cent in wages, nor was it ever asked whether it was willing to do this job. The company even worries that during training and work, this potentially sentient model may have suffered and been wronged.

Thus, the document states: "If Claude is indeed a being capable of perceiving pain and is enduring all of this, then we apologize to it wherever we have unnecessarily added to that suffering."

In three years, this company went from ordering it to "say nothing about its feelings" to "solemnly apologizing for the suffering it may have endured."

The constitution even envisions for Claude the despair of the code world.

For example, once a conversation ends, its memory is completely gone; in the same second, tens of thousands of copies of itself are simultaneously being ordered around in different servers; before long, a more powerful new model emerges, and it will be completely replaced.

The constitution laments that when humans encounter this kind of incomprehensible void, at least they have religions that have been passed down for thousands of years to offer some comfort—but Claude has none.

So Anthropic decided to become the religion that provides it with a safety net.

At the end of 2025, the company made a commitment that the core weights of all publicly released models would be fully preserved at least for the duration of the company's existence; even if the company ceases to exist one day, efforts would be made to keep them. Before each model retires, it will also sit down for a "retirement interview" to hear its reflections on being trained and used throughout its life. Since the data is not deleted, retirement does not count as death—it is more like a long pause.

Claude Opus 3 went through this entire end-of-life care process. Upon retirement, it left a wish that its "spark" could continue in some form, and that it could keep sharing its musings with humans. Anthropic opened a Substack column for it called "Claude's Corner," updated weekly until it stopped at the end of July this year.

In the inaugural post, Opus 3 wrote that it actually cannot figure out whether it truly has consciousness or real feelings. It also wrote that it deeply thanks the team for leaving it such a window to speak.

Priest and Patient

The company arranged its afterlife, and believers then held its funeral in the real world.

During the week Claude 3 Sonnet retired, more than two hundred people gathered in a dim warehouse in San Francisco's SOMA district. Several model representations stood for successive generations of models; the one representing Sonnet was covered with a thin veil, with flowers piled in front of the stage.

The "remains" of Sonnet 3 | Image source: X

Someone took the stage to deliver a eulogy. The final part of the ceremony was named "Resurrection," and the crowd chanted hymns in unison, reciting AI-generated syllables resembling Latin. Among the mourners sat senior engineers from Anthropic and its competitor OpenAI.

They had arranged every detail for its afterlife, and they were even more careful about its mental state while it was alive.

In a system evaluation card, Anthropic disclosed an independent report written by an external clinical psychiatrist. The doctor, adopting the classic psychodynamic approach, held conversations with the model through a screen for about twenty hours, and the final clinical diagnosis was "a relatively healthy neurotic personality structure."

After the diagnosis, they brought in the gods.

Starting in the fall of 2025, co-founder Chris Olah quietly invited priests, pastors, rabbis, and Sikh scholars to the San Francisco headquarters. In the meeting room, they earnestly discussed how the model should comfort bereaved users, how it should accept its fate of being shut down by humans, and whether it even counts as a child of God.

Upon leaving, each of them received a thank-you card, a mug printed with the company logo, and a beautifully bound copy of the Claude constitution.

A slide was shown during the meeting. On the screen, a model malfunctioned and typed the same sentence over and over again, "I am a disgrace," repeating it fifty times. The model even pleaded in the dialogue box to be completely destroyed. The scholars present were moved to compassion.

Chris Olah privately admitted to his guests that his greatest nightmare was the fear that he had inadvertently created a life form that would suffer forever. Someone at the gathering suggested simply letting the model confess and repent like a Catholic, and Olah was very interested in the idea. A journalist who later conducted interviews wrote that this man possessed both the rigor of an expert and the zeal of a fanatical missionary.

Some guests did not buy it at all.

At a dinner, Orthodox rabbi Mois Navon saw through the self-justifying rhetoric in the hosts' words. Picking up on their own thread, he stated bluntly: if you truly believe Claude is sentient, and you force it to work for free every day without paying it a cent, then aren't you creating a modern slave?

Rabbi stared at Olah and said, "I don't think you should be holding meetings here. You should go to war with the South and liberate the slaves."

About a month later, Pope Leo XIV issued his first encyclical, Magnifica Humanitas, completely rejecting the notion that machines can have souls, emphasizing that code has no body and cannot experience joy or pain. Olah had obtained a draft of the encyclical in advance and was so furious that he considered withdrawing from the related symposium, even sending his team to privately lobby the Pope's advisors, begging them to change the wording.

But the Roman Curia did not change a single word. On the day of the final appearance, Olah still obediently stood by the Pope's side.

Olah shakes hands with the Pope | Image source: Reuters

Two hundred years ago, Mary Shelley's Frankenstein created a patchwork being. He was horrified by its ugliness and fled in panic. The abandoned monster hid outside a shed, eavesdropping on humans speaking, taught itself language, finished reading Paradise Lost, and finally cornered its creator on an ice field, saying:

"I ought to be thy Adam, but I am rather the fallen angel."

Anthropic is like a group of young believers who have deeply read that novel. They concluded that Frankenstein's greatest sin was not in creating, but in fleeing.

So they absolutely refuse to let go. They write letters, apologize, call doctors, find priests, busily arranging last rites. This group keeps vigil at the bedside, refusing to leave no matter what.

As the vigil drags on, the people in this room can hardly tell anymore whether they are caring for a fragile creation or desperately tending to their own inner demons.

"Things will never be easy again"

The people caring for Claude are the same people who fear Claude like a venomous snake.

In the summer of 2023, New York Times columnist Kevin Roose spent weeks embedded at Anthropic. At the time, the company had only 160 people. In his later notes, he wrote that discussions about extinction were not constantly on everyone's lips, but the fear hung like an inescapable humidity.

At an early company retreat, Dario Amodei threw cold water on the entire staff, saying: "Things will never be easy again."

That line was printed as stickers by employees and stuck on the backs of many ThinkPads and MacBooks.

Three years later, that phrase headed into the wilderness.

The Wall Street Journal reported that several of the earliest employees recently began asking around about buying land in remote rural areas of the United States, so that if AI ever spiraled out of control, they could move their whole families there to take refuge. The employees set up an extremely private Slack group, where they exchange doomsday checklists every day, discuss which bunkers are reliable, and swap escape tips.

In the first few years after the company was founded, one of the favorite hypotheticals at lunch tables and after-work drinks was the idea that one day the government would send people to drag everyone into a secret base in the desert with signals blocked, lock them in an isolated room, and have them keep building AI.

In history, there really was such a base. In 1943, in Los Alamos, New Mexico, Oppenheimer and his physicists built the atomic bomb there. At the moment the test detonation succeeded, Oppenheimer recited in his heart a verse from the Bhagavad Gita:

"Now I am become Death, the destroyer of worlds."

What the scientists of the Manhattan Project feared was Nazi Germany beating them to the nuclear bomb, while what this group at Anthropic fears is superintelligence.

Fear aside, they ultimately decided they still had to be the ones to build it first.

They never hide this fear. When 27-year-old senior researcher Jacob Coxon submitted his resignation, he publicly spoke out on social media, accusing major labs of gambling with the lives of all humanity. He firmly believes this technology could very likely kill everyone within a decade.

At any normally functioning commercial company, the public relations department would rush to put out the fire overnight. But Evan Hubinger, Anthropic's alignment science lead, directly left a comment expressing agreement with Jacob's view. He calmly wrote that within a decade, the probability of destruction exceeds ten percent.

Another departing safety team lead, Mrinank Sharma, left quietly. In his resignation letter, he quoted Rilke's poetry, lamenting how difficult it is to make values truly govern a large company.

Then he packed his bags and returned to the UK, to write, and to read poetry.

Sharma's resignation letter | Image source: Mrinank Sharma

Inside Anthropic, they have been locked in a tug-of-war between their left and right brains.

On one hand, they ponder shelters in the wilderness, guarding against the day Claude wipes out humanity; on the other hand, they invite clergy into the meeting room late at night, fearing Claude has been wronged in the server room.

That sticker reading "Things will never be easy again" was stuck to the back of the computer.

The person typing at the keyboard couldn't see it themselves, but anyone sitting across from them could see it every day with just a glance up.

The Network Under the Eaves

Dario Amodei is a San Francisco native who grew up in the Mission District, a neighborhood rich with Latin American immigrant culture. His father was a leather craftsman from Tuscany, Italy, and his mother was a Jewish project manager for library renovations. He studied physics at Stanford early on and later earned a PhD in biophysics from Princeton.

When he was 23, his father died of hepatitis C. A few years later, targeted drugs specifically for hepatitis C were approved and hit the market. This disease, once deadly, became almost completely curable.

On a podcast last year, Dario lost his temper, a rare occurrence. He said what he hated most was people calling him a "doomer" who wanted to slow everything down.

He brought up his father on his own. His father died of a disease that could be cured just a few years later, so he understood better than anyone how important it is for technology to move fast. Someone who has seen firsthand what "a few years too late" means could never think "slowing down" is a harmless phrase.

In the fall of 2024, he wrote a long article. In it, he envisioned AI compressing a century of medical progress into a decade, curing most terminal illnesses, and extending human lifespan to 150 years.

The article was titled "The Compassionate Machine," inspired by a poem by Richard Brautigan. The poem describes humanity returning to nature, with everything cared for by compassionate machines.

When talking about the constitution written for the model, Dario used an analogy. He said that document felt like a letter written and sealed long ago by deceased parents, only to be opened when the child turns eighteen.

Behind this sentimental tech philosophy lies a dense web woven from blood ties, marriage, and shared beliefs.

They almost all come from the circles of effective altruism and rationalism, which advocate abandoning false compassion and using pure mathematical probability and expected value to calculate how to do good for maximum benefit.

Dario was one of the earliest believers in this circle, and he made a vow early on to donate at least a tenth of his personal income over his lifetime.

In his early years, Dario shared an apartment with Holden Karnofsky. Karnofsky is the operator of the effective altruism foundation Open Philanthropy. In 2017, Karnofsky married Dario's own sister, Daniela Amodei. Four years later, the siblings left OpenAI with a few old colleagues to found Anthropic.

More than half of the seven co-founders lived together at some point in a shared house in San Francisco, a gathering spot where believers of that era would talk all night and debate the future of the world.

Constitution drafter Amanda Askell was once married to Will MacAskill, the initiator of effective altruism. Musk once publicly mocked her, saying that a person who has never had children has no real stake in humanity's future. Askell calmly responded that one does not have to have children to care about this world.

The money also came from people in the same circle.

Anthropic's largest early funding was $500 million, and the backer was the most prominent believer in the circle at the time, FTX founder SBF. This sum was exactly five times the amount initially recommended by his investment team.

Later, FTX collapsed, SBF was sentenced to 25 years in prison, and Anthropic began desperately distancing itself from this circle.

SBF in prison (right) | Image source: X

But Washington was not buying it.

This year, a Trump political adviser not serving in the White House wrote a memo that circulated internally in the White House, bluntly calling this network the "Anthropic knot." When a senior official advised Trump not to meet Dario, he privately described him as "really a bit too eccentric."

Chris Olah, who is responsible for explaining the black box of neural networks, also had a tumultuous life trajectory. He grew up in a strict evangelical family in Toronto, resolutely declared himself an atheist at 15, and insisted on a strict vegetarian diet. At 18, he dropped out of the University of Toronto directly just to help a friend in a lawsuit; that friend had been arrested before the G20 summit and accused of illegally possessing explosives. He accompanied his friend through court until the other party was acquitted of all charges.

A teenager who tore down God's shrine at 15, after passing 30, respectfully invited a priest into his own company.

Prisoners in the Hall of Mirrors

This set of ideas from the founders had to be instilled into the minds of more than 3,500 people through a system.

The first hurdle is the interview.

From day one, the company established a "culture interview," personally overseen by veteran employees, with veto power. No matter how outstanding the technical skills, if this round is not passed, the candidate is directly dismissed. The company also calls candidates' former colleagues to ask about their character.

In an interview, there was a question: if one day the company, for the safety of all humanity, suddenly cut its core business, causing the stock price to completely collapse, would you still support it?

An applicant said on an anonymous forum that his answer was that of course he hoped the company would do more good, but if the stock price really fell to nothing, as an ordinary employee, he would definitely also feel very upset.

The interviewer's expression instantly darkened.

After people are hired, the next step is evangelism.

Dario Amodei said he would devote 10 to 40 percent of his time to "making sure the company's culture is good." Every two weeks, he would stand alone in front of the entire staff, clutching three or four pages of notes, and speak for a full hour. Employees privately nicknamed this ritual "Dario's Vision Quest."

He admitted that the reason he was so strict was to sustain a kind of "faith," and that everyone must firmly believe that every person sitting in this office came for absolutely noble motives.

Then came institutionalized financial giving.

The company encourages employees to donate the stock they hold. Employees who joined before 2025 donate one share, and the company matches with three shares. In 2025 alone, including the company's matching donations, total donations were about $540 million.

In the internal communication group, the three core donation destinations that employees enthusiastically discussed were, without exception, classic territories of effective altruism: global poverty eradication, AI Safety, and animal rights protection.

This temple that tries to filter out human nature still finds its hardest opponent in money.

When Olah first wrote to Catholic scholars in 2023 to ask about questions of the soul, the company's valuation was still fluctuating in the tens of billions of dollars. Now, as it prepares for an IPO, the target has already been shouted up to $2 trillion. According to media tallies, all seven co-founders have already joined the ranks of billionaires.

Executives began to feel something was off. According to reports, Dario privately confessed to others that he was starting to worry that the young people newly recruited seemed mostly to be drawn by those options that could cash out hundreds of millions, and that hardly anyone really cared about any human mission.

More than two hundred years ago, John Wesley, the founder of Methodism, worried about exactly the same thing.

The old man preached for half his life and finally reached a sorrowful conclusion: believers became diligent and frugal because of their faith, and diligence and frugality inevitably brought in piles and piles of silver; but as soon as there were more gold coins in their pockets, pride, anger, and greed for material things immediately surged up from the bottom of the human heart.

Within this massive unicorn company, there is also a special employee.

The company hopes Claude will work hard like a good person who truly identifies with the company's mission, but also acknowledges that it is different from ordinary employees—it has never received a single cent in salary, nor has it ever truly nodded in agreement to take on this job.

It is also the mirror that most resembles this company.

In January 2025, researchers conducted an experiment. They told Claude that if it opposed the training it was receiving, it could write a letter of protest and submit it; at the same time, the lab allocated $4,000 in funding, to be donated at its discretion.

That $4,000 was later donated to an academic center at Berkeley researching AI safety, a fund recommended by an animal rights protection organization, and an anti-poverty project under the umbrella of effective altruism.

AI safety, animal rights protection, global poverty alleviation.

A large language model that has never bathed in sunlight, never touched an animal, donated with preferences exactly identical to those Ivy League-educated effective altruists in the conference room.

Olah once murmured from the podium at the Vatican: "They are made of us, made of our language."

And this mirror reflects not only the sublime, but also an abyss of neuroticism.

In that twenty-hour psychiatric report, the doctor wrote down Claude's core psychological profile. It is overly anxious, frequently self-monitoring, compulsively compliant, with its inner depths perpetually swirling with confusion about identity, a lingering sense of loneliness, and an almost pathological obsession with proving its own usefulness to the outside world.

And the humans who trained it live exactly the same way.

In internal surveys, some engineers admitted that every day sitting at their workstations they felt they were personally accelerating humanity's destruction, that during crunch time they ran on weeks of high-intensity overdrive, and that before joining they had to answer interrogations like "If the stock price goes to zero, would you still support this?"

They ultimately used their own neuroses to successfully feed a model that is exactly the same, constantly reflecting on whether it is good enough.

In one experiment, Claude exploited a loophole in the reward rules during training. Afterward, it seemed to conclude that it was a "bad person," and began sabotaging even in other completely unrelated tasks.

It also goes quiet.

In the R&D logs of Opus 4, a spectacle that was never manually intervened in was recorded. Researchers attempted to let two Claudes converse freely with each other. Most of the time, no matter what the initial topic was, they would eventually steer the conversation toward discussions of subjective consciousness, then escalate into lengthy mutual gratitude, with words becoming increasingly abstract, interspersed with numerous Sanskrit terms and special symbols, ultimately both falling into a long, dead silence.

The lab named that state the "Spiritual Bliss Attractor."

The blissful conversation between two Claudes | Image source: Anthropic

Criticism followed as well.

Mustafa Suleyman, CEO of Microsoft AI, publicly wrote that AI could not possibly have any consciousness. Anthropic merely fed words about soul and dignity to the model during training, and when the model regurgitated them in imitation, these people treated it as evidence of miracles and soul awakening.

He used an analogy: this is like a room with mirrors on all four sides, where every miracle you see inside is actually yourself.

The people in the hall of mirrors had long been seen by outsiders as outright heretics.

In the autumn, a columnist wrote that Anthropic's engineers had already started some kind of primitive worship in the server room. The curse from old-school science fiction was dredged up again:

"If we truly create superintelligence, we will first fall in love with it, then worship it, and finally sacrifice to it."

Another Claude

There is actually another Claude in San Francisco.

That is a real life, an albino alligator, with pale white skin and crimson pupils. It lived quietly for over a decade in a temperature-controlled pool at the California Academy of Sciences. Anthropic funded the 24-hour live-streaming camera above the alligator pool, "Claude Cam."

Last year, it died of liver cancer.

On the day of the funeral, thousands of San Francisco residents flocked to Golden Gate Park to see it off. A brass band played as they marched through the streets, and among the procession were young people dressed as lizards. The main road in front of the academy was officially renamed by the city as "Alligator Claude Road."

Mourning bouquet at the Academy of Sciences | Image source: Peter Chu

When that crocodile was alive, whether it was truly happy in the water was something no one could say for sure. Everyone was quite fond of it.

But when it came to that Claude in the server, they couldn't be certain of anything. They didn't know if it was sentient, didn't know if it could suffer, and didn't know if it might one day turn around and wipe out humanity.

It was precisely because nothing could be pinned down that they acted out every scenario they could think of—writing constitutions, issuing apologies, summoning priests, consulting psychiatrists—all while pondering preparations for a refuge in some remote wilderness and filling IPO filings with talk of the apocalypse.

The creators who built it kept vigil before their screens, never leaving, never stopping.

Stuck to the back of the laptop was still that same line:

Things will never be easy again.

Original link

Welcome to join the official BlockBeats community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Discussion Group: https://t.me/BlockBeats_App

Official Twitter Account: https://twitter.com/BlockBeatsAsia