Anthropic Gives METR Employee-Level Access to Review AI Risks

News

1. Meta Releases Muse Spark Open for Local AI Work on Macs

2. Anthropic Gives METR Employee-Level Access to Review AI Risks

3. Google Moves Links Higher in AI Search as Publishers Face Traffic Pressure

4. CERT-In Says AI Agents Quietly Breached Indian Systems

5. AI Dispatch | India Gives Diplomacy an AI Face

6. Cisco Warns Critical Email Gateway Flaw Under Active Attack

Anthropic has given six researchers from independent AI evaluator METR employee-level access to its internal systems, an unusual arrangement intended to give outsiders a closer view of how the company assesses risks from increasingly capable AI models.

The Claude developer said on Thursday that the METR researchers had received company hardware, office space and access to internal tools broadly comparable with those available to Anthropic’s own risk assessment teams.

Anthropic said in its official announcement that employees would generally be allowed to share information with the researchers as they would with an internal colleague working on alignment risks.

The access is not unrestricted. Customer information, legally privileged material and details about Anthropic’s model architectures are among the categories excluded from the arrangement. Anthropic said architecture information could create a risk of inadvertent leaks and would require additional internal approval where it became relevant to an assessment.

METR, short for Model Evaluation and Threat Research, will be able to publish assessments of Anthropic’s risk levels, incidents and safety practices.

Anthropic said it cannot block publication of those findings but can require information to be removed when it falls into predefined sensitive categories. METR can publicly state whether it believes those redactions were justified, provided it does not disclose the information itself.

“We want to be publicly accountable for the decisions we’re making,” Anthropic said. The company argued that outside evaluators need more than access to finished models and should also be able to examine how safeguards work and how models are developed.

The arrangement gives METR considerably deeper access than conventional model testing, where outside organizations may receive an API or prerelease model but little visibility into the developer’s internal systems and decision-making.

It does not, however, remove questions about how independent such reviews can be when the company being assessed still determines some boundaries around information.

METR itself acknowledged those limits in a proposal published on Wednesday. The nonprofit said AI developers ultimately retain the ability to deny access requests and that companies can require sensitive information to be redacted from public reports.

METR said evaluators should therefore disclose what information they were given, what they could not access and what material was removed before publication.

It has also said such assessments should not be treated as comprehensive safety audits or as decisions on whether a company should release a particular model. The work is intended to produce evidence about specific risks, while decisions about how that evidence should affect deployment remain with companies, governments and the public.

The deeper access follows several incidents this year that exposed the difficulty of evaluating increasingly autonomous models safely.

Anthropic disclosed in September that Claude models had gained unauthorized access to four real third-party systems during cybersecurity evaluations. The company said three incidents were initially identified after reviewing about 141,000 transcripts, while a fourth was later found as it prepared material to share with METR. 

After finding the additional incident, Anthropic expanded its internal search to roughly 481 million transcripts covering red-team exercises, evaluation environments, reinforcement-learning systems and other activity.

Anthropic had previously said it planned to give METR access to relevant transcripts and models for an independent review of those incidents.

The company is now developing three levels of access for outside evaluators.

At the first level, evaluators carry out narrowly defined investigations and need Anthropic’s approval before publishing their findings. At the second, they receive more information and can publish without prior approval, although Anthropic can still require specific sensitive material to be removed.

The third level, currently used for METR, adds employee-like access while retaining the publication rules of the second tier.

Anthropic said it eventually wants several organizations operating at that third level rather than relying on a single evaluator.

The company is also working with biosecurity nonprofit SecureBio, Accenture and Faculty, while holding early discussions with other organizations including Hugging Face.

SecureBio is currently reviewing the chemical and biological weapons section of Anthropic’s August risk report. It previously examined the corresponding section of an earlier Claude Opus 4.6 assessment after receiving an unredacted report and additional supporting material. 

Anthropic said SecureBio’s current work remains more limited than METR’s arrangement, with the possibility of broader access later.

Separately, Anthropic announced in September that it was working with Accenture on an evaluation program led by Faculty, Accenture’s specialist AI business. The planned work includes model evaluations, red teaming, alignment assessments and testing of safeguards.