Now loading...
In a significant move toward enhanced accountability in artificial intelligence, a major AI company has taken steps to involve independent evaluators in its safety practices. Following its commitment to more rigorous audits last month, the company announced it has started to provide “employee-like access” to a team from METR, a nonprofit dedicated to assessing AI models. This initiative forms part of the White House Accord on Super Intelligence, which encourages collaboration between AI firms and external evaluators.
METR is set to publish the findings from its risk assessments, building on a year-long partnership that included multiple reviews of the company’s Risk Reports. In addition to METR, the company is collaborating with SecureBio and Accenture for further evaluations and is engaged in initial talks with other organizations, such as Hugging Face, to broaden its oversight.
As AI systems grow in sophistication, the need for robust safety measures has become increasingly critical. The company aims to maintain public accountability while recognizing that not all relevant information can be shared due to commercial sensitivity and potential misuse by malicious entities. By granting third-party evaluators deep access to its data, they hope to bridge the understanding gap between internal systems and external assessments.
To properly evaluate advanced AI models, independent assessors require specific criteria, including expertise in identifying potential risks and the independence to operate without financial ties to the evaluated company. They also need sufficient access to relevant operational information and the freedom to publish their findings and assessments.
The practice mirrors banking regulations, where in-depth scrutiny by assigned examiners ensures that financial risks do not jeopardize public safety. In the AI sector, this framework is seen as a proactive step towards establishing comprehensive safety oversight. The company is keen on learning from its multiple evaluator partnerships and is committed to providing updates as the collaborations progress.
The arrangement with METR involves onboarding team members who have access comparable to the company’s internal risk assessment teams. This collaboration allows METR to analyze risks and practices without the company having any veto over its publications. However, there are safeguards in place regarding the sharing of sensitive information, which METR can discuss publicly without revealing specific redacted details.
In addition to METR, the company is working with other evaluators, though these partnerships are in earlier stages with more restricted access and publication rights. Their goal is to establish a three-tier access system for evaluators that will evolve as trust grows. The first tier focuses on narrowly scoped evaluations, while the second expands access and allows publication without prior approval. The third tier, like the arrangement with METR, provides substantial access and stronger publication rights.
SecureBio is currently in the initial tier, reviewing sections of a recent Risk Report. The aim is to potentially elevate its engagement as trust strengthens and its evaluative capacity develops. The partnerships with Accenture and Faculty are also set to broaden oversight, focusing on comprehensive model evaluations and safety assessments. By voluntarily embedding evaluators, the company reinforces its stance that this practice should become standard in the frontier AI landscape.
