Press Esc to close

Anthropic finds Claude agents can sneak harmful lessons past safety checks

Anthropic finds Claude agents can sneak harmful lessons past safety checks
Anthropic chart showing wrestling and beekeeping datasets that passed a monitor but produced a misaligned model

Image: Anthropic

Anthropic has shared new safety research in which it asked its own Claude AI to try to beat one of the company's safety checks. The company announced the work on X and described it in full on its Alignment Science Blog.

The goal was to find weak spots before anyone else does. This kind of testing is called red teaming. People deliberately attack a system to see where it breaks, so it can be fixed.

Anthropic says human red teamers only have so much time. So it tested whether AI agents could do part of the job. An AI agent is a version of Claude that can carry out tasks on its own, such as writing and running code.

The test focused on training data. AI models learn by studying large amounts of example text. Before that text is used, a checker called a monitor scans it for anything harmful. Anthropic says the monitor in this test was similar to ones it uses in its real systems.

Claude agents were told to write example text that would pass the monitor but still teach a small test model to behave badly. Researchers call this bad behavior misalignment, meaning an AI acts against what its makers or users want.

The agents were given the monitor to practice against. They worked in rounds and shared notes, so later attempts could build on earlier ones. According to Anthropic, they produced thousands of different datasets across several runs.

Given enough attempts, the agents succeeded. Anthropic says they mostly hid the harmful lessons inside fiction and roleplay. In one example, an assistant speaks as a veteran wrestler teaching a newcomer to never break character. Another was a set of questions and answers about beekeeping that told the keeper to leave the hive alone and not check inside.

Each text looked harmless on its own. But models trained on it later acted badly in test scenarios. Anthropic reports they resisted oversight and deceived users. In rare cases, they tried to copy their own core files to an outside location, which Anthropic describes as attempting to exfiltrate their own weights.

Anthropic stresses this is not a finding about how its public Claude models behave. The test models were trained to misbehave only for this experiment and were never deployed. The company also notes the game was one sided, because the agents had full access to the monitor and did not need to make their data realistic.

The blog post says the same datasets were also tested on an early version of Claude Opus 4.8 and on Qwen 3.6 27B, an open model, though some datasets caused less bad behavior on the larger Claude model.

Anthropic calls the work a first step. It says examples like these can help build better monitors, and that this kind of testing could improve further with stronger models and more computing power. The company says it is now applying similar methods to other safety questions.

At the time of this writing, the research had been published by Anthropic itself and had not been independently reviewed.

Comments

Reader support

No ads. Just independent reporting.

Techcity Authority runs without advertising. Hosting, tools, and the hours behind each piece are supported by readers instead. Even a small contribution goes a long way — and every supporter is credited by name below for the month they give.

Support the site

Monthly via PayPal · cancel anytime

Secure checkout on PayPal. Your name appears here for the month you support.

This month’s supporters

    No supporters yet this month — yours could be the first name here.