Anthropic发布了重大更新

Anthropic update on independent evaluators \ Anthropic Skip to main content (https://www.anthropic.com/news/an-update-on-independent-evaluators#main-content) Skip to footer (https://www.anthropic.com/news/an-update-on-independent-evaluators#footer) (https://www.anthropic.com/) Research Policy (https://www.anthropic.com/policy) Commitments Learn News (https://www.anthropic.com/news) Try Claude (https://claude.ai/) AnnouncementsAn update on independent evaluators Oct 15, 2026Last month, we committed to stronger independent, third-party assessment (https://darioamodei.com/post/we-must-pace-the-frontier) of our models and safety practices. We also signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators. Today, we’re announcing that we’ve started to give employee-like access to a team from METR (https://metr.org/) , a nonprofit that evaluates frontier AI models. METR will publish the results of its risk assessments. Over the past year, we’ve worked with METR on several reviews of our Risk Reports (https://metr.org/risk-assessment/) . We’re also working with SecureBio (https://securebio.org/) and Accenture (https://www.anthropic.com/news/accenture-embedded-evaluation) as evaluators, and are in early discussions with several other organizations, including Hugging Face (https://huggingface.co/) . Here, we explain our approach to working with evaluators. Why we work with independent evaluators As models have become more powerful, they’ve required stronger safeguards and more challenging risk assessments. We want to be publicly accountable for the decisions we’re making. But we can’t publicly share all the information relevant to our safety work. Some of it could be useful to malicious actors, or is commercially sensitive, like compute or training information. Instead, we’re giving third-party evaluators deep access to our information, and are having them share evidence and reasoned conclusions about the risks involved in building our models. We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands. To evaluate frontier models, evaluators need the following: Expertise in assessing the capabilities of frontier models, including understanding the way evaluations can go wrong, such as alignment faking (https://www.anthropic.com/research/alignment-faking) . Independence: they shouldn’t hold equity in the company they’re evaluating, or be financially or operationally dependent on it. Enough access to meaningfully evaluate risks. Beyond access to the latest models, evaluators also need visibility into how those models were trained, into how our safeguards function, and more. Freedom to publish observations and assessments of a model or a process. Although the analogy isn’t exact, bank supervision operates on similar logic. Banks know more about their own risk positions than outsiders do, so regulators assign resident examiners to work inside the largest banks. If a bank’s risk-taking could threaten the public, third parties try to see that in advance. For the AI industry, this is a step toward robust safety oversight (https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdf) . We’re moving quickly with multiple evaluators and learning actively about what kinds of partnership work. We’re committed to sharing updates as we go. What we mean by “employee-like access” We’ve onboarded six METR employees with company hardware, office space, and access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to. Outside a small number of clearly defined categories (e.g., data about customers, attorney-client privileged information, or information about our model architectures), Anthropic employees are encouraged to share information with them just as they would with a fellow employee worki

Anthropic An update on independent evaluators An update on independent evaluators https://www.anthropic.com/news/an-update-on-independent-evaluators