Governance: Company self-governance and its critics

This content reflects the field as of October 2026.

When a new medicine or a new aircraft reaches the public, a government agency has usually examined it first. News coverage of AI often assumes something similar happens before a powerful new model is released. In most countries it doesn't.

As of October 2026, the main limits on how the most capable AI systems are built and released are limits the companies wrote for themselves. A few laws now touch these systems, and most of those require companies to publish or report things. The decisions that matter most for safety, such as how much testing is enough, what result would stop a release, and what protections a model needs, are mostly set by each company's own policy.

Those policies aren't purely private. In May 2024, at a summit in Seoul hosted by the governments of the United Kingdom and South Korea, sixteen organizations agreed to a common list called the Frontier AI Safety Commitments. Four more joined later. The signers include Amazon, Anthropic, Google, Meta, Microsoft, OpenAI, and xAI from the United States, along with companies based in Canada, China, France, South Korea, and the United Arab Emirates (Department for Science, Innovation and Technology 2024).

The document describes itself with care. The organizations "undertake to develop and deploy their frontier AI models and systems responsibly," following what the text calls "voluntary commitments" (Department for Science, Innovation and Technology 2024). They promise to assess the risks of their models, to set thresholds at which a risk would be intolerable, and to publish a framework explaining how they do both.

The word "voluntary" carries the weight. Two governments announced the list, and neither enforces it. The text names no inspector and no penalty.

People draw opposite conclusions from this arrangement.

• Some see promises with nothing behind them. On this view a company will keep a commitment until it becomes expensive, and then rewrite it.
• Some see a sensible first step. The technology is changing quickly, governments lack the expertise to write detailed rules, and public commitments at least give outsiders something to hold companies to.
• Some see a different problem. They argue that the largest companies talk about danger because rules built around their own safety practices would be hard for smaller rivals to meet.

Each of these views is held by serious people, and each group has interests of its own. Companies gain from being trusted without being regulated. Advocacy groups that warn about AI gain attention and funding when the warnings are believed. Firms that sell openly released models gain when restrictions are seen as self-serving.

You don't need to settle that dispute to read a commitment well. Three questions work on any promise a company makes about AI safety, whether it appears in a summit document, a company policy, or a press release.

1. What exactly is promised? Look for the verb and for the conditions attached.
2. Who checks? Someone outside the company either can see whether the promise was kept, or can't.
3. What happens if it's broken? There may be a penalty, a legal duty, or nothing.

References

• Department for Science, Innovation and Technology. 2024. "Frontier AI Safety Commitments, AI Seoul Summit 2024." May 21, 2024.
  • Free: GOV.UK

This content reflects the field as of October 2026.

Introduction

AI companies say they test their most powerful models for dangerous abilities before release. A reasonable follow-up is what they've promised to do if a test comes back with a worrying result.

This reading explains the documents in which several developers answer that question, how three of them are built, and what an independent comparison found across twelve.

What the Seoul Commitments Asked For

In May 2024 sixteen organizations, later twenty, agreed to the Frontier AI Safety Commitments at a summit in Seoul. They promised to assess the risks of their most capable models, to set out thresholds at which severe risks "would be deemed intolerable," to be transparent about how they do this, and to publish "a safety framework focused on severe risks" (Department for Science, Innovation and Technology 2024).

A frontier safety framework is a document in which an AI developer states how it will test its most capable models for severe risks and what it will do about the results. Different companies use different titles for it.

How a Framework Works

Most frameworks share one mechanism. A capability threshold is a specified level of ability at which a model is judged to pose a serious enough risk to require stronger protections. A safeguard is a protection a developer puts in place, such as security against theft of a model or limits on what the model will do for users.

The two are joined in an if-then commitment: a promise that if a model reaches a stated threshold, then the developer will apply stated safeguards before going further. For example, if a model could give real help to someone trying to make a biological weapon, then it won't be released until protections against that use are in place.

Three companies' frameworks show the pattern.

(Anthropic 2026; Google DeepMind 2025; OpenAI 2025c.)

The three differ in vocabulary and in which risks they track. They share a structure: named abilities, tests for them, and protections required once a model shows the ability.

What a Comparison Across Twelve Companies Found

METR, a nonprofit that evaluates AI models, compared the published policies of twelve companies in December 2025. Besides the three above, they included Amazon, Cohere, G42, Magic, Meta, Microsoft, Naver, Nvidia, and xAI (METR 2025).

It found nine elements that recur, among them capability thresholds, security for model weights, conditions for halting deployment and development, and a process for updating the policy. It also found differences. Some policies concentrate on catastrophic risks and others on a wider range of harms, and some lean more heavily on numerical test scores (METR 2025).

METR has evaluated models for some of the companies it compared, which gives it knowledge of their policies and a working relationship with their authors.

What a Framework Is and Isn't

A framework is a company's statement of what it will do. The company writes it, decides whether a threshold has been reached, chooses the safeguards, and can revise the document. Each of the three above has been revised at least once.

One law has changed the picture slightly. California's 2025 statute requires large developers of the most capable models to publish a framework on their websites (Office of the Governor of California 2025). The law makes publication a legal duty in that state. The thresholds and safeguards inside the framework remain the company's own choices.

Some developers also publish a second kind of document, under titles such as "model spec" or "constitution," describing how their models are meant to behave in ordinary use. Those documents concern everyday behavior. Safety frameworks concern rare and severe risks.

Conclusion

As of October 2026, several leading developers publish frameworks that tie stronger safeguards to specified levels of capability. The frameworks share a structure and differ in detail, and an independent comparison has documented both. They are public commitments written and interpreted by the companies that make them, with publication required by law in at least one US state.

Key Terms

• Frontier safety framework: A document in which an AI developer states how it will test its most capable models for severe risks and what it will do about the results.
• Capability threshold: A specified level of ability at which a model is judged to pose a serious enough risk to require stronger protections.
• Safeguard: A protection a developer puts in place, such as security against theft of a model or limits on what the model will do for users.
• If-then commitment: A promise that if a model reaches a stated threshold, then the developer will apply stated safeguards before going further.

References

• Anthropic. 2026. "Responsible Scaling Policy." Version 3.4, effective July 8, 2026.
  • Free: Anthropic
• Department for Science, Innovation and Technology. 2024. "Frontier AI Safety Commitments, AI Seoul Summit 2024." May 21, 2024.
  • Free: GOV.UK
• Google DeepMind. 2025. "Frontier Safety Framework." Version 3.0, September 22, 2025.
  • Free: Google DeepMind
• METR. 2025. "Common Elements of Frontier AI Safety Policies." December 2025.
  • Free: METR
• Office of the Governor of California. 2025. "Governor Newsom Signs SB 53, Advancing California's World-Leading Artificial Intelligence Industry." September 29, 2025.
  • Free: Office of the Governor of California
• OpenAI. 2025c. "Our Updated Preparedness Framework." Version 2, April 15, 2025.
  • Free: OpenAI

This content reflects the field as of October 2026.

Introduction

A company can publish a detailed safety policy and follow none of it, and from outside you couldn't tell. Whether a commitment means anything depends on whether someone other than the company can see what the company does.

This reading covers three ways commitments get checked, a 2026 proposal to strengthen one of them, and the weakness they share.

Government Institutes

Verification is checking, by someone other than the party that made a promise, that the promise is being kept. Several governments have set up a safety institute: a government body that tests advanced AI models and studies their risks.

The United Kingdom's is the AI Security Institute, part of the Department for Science, Innovation and Technology. It was founded as the AI Safety Institute and later renamed. It describes its work as "testing leading AI systems before they are released publicly," and says it has "pre-deployment access to leading AI models" through collaboration with AI companies (UK AI Security Institute n.d.).

The US counterpart has changed more. In June 2025 the Commerce Department turned the US AI Safety Institute into the Center for AI Standards and Innovation. The commerce secretary's statement said the center would set up "voluntary agreements" with developers and lead evaluations of "demonstrable risks, such as cybersecurity, biosecurity, and chemical weapons." It also said that "censorship and regulations have been used under the guise of national security" (US Department of Commerce 2025). As of October 3, 2026 the center's official page carries another new name, the Center for Advancing Innovation and Standards for Super Intelligence (National Institute of Standards and Technology n.d.).

Neither institute is a regulator. Neither can block a release.

Outside Evaluators

Third-party evaluation is the testing of a model by an organization that didn't build it. Nonprofits and specialist firms do this work, sometimes at a developer's invitation before a release.

What they can learn depends on access: what an outside party is permitted to see and test, and for how long. An evaluator given a few days with a finished model learns less than one who can see training methods, internal test results, and incident reports.

Employees Who Speak

A whistleblower is an employee or former employee who reports wrongdoing or danger inside an organization to people outside it. Employees see what no outside evaluator sees.

In June 2024 a group of current and former employees of OpenAI and Google DeepMind published an open letter titled "A Right to Warn About Advanced Artificial Intelligence." It asked AI companies to accept four principles ("A Right to Warn" 2024):

• not to make or enforce agreements that prohibit criticism of the company over risk-related concerns
• to provide an anonymous process for raising such concerns with the board, regulators, and an independent organization
• to support a culture of open criticism, with trade secrets protected
• not to retaliate against employees who share risk-related information publicly after other processes have failed

The letter is a request. Its authors argued that ordinary whistleblower laws cover illegal activity, while many of the risks they worried about weren't yet regulated at all.

A 2026 Proposal

In September 2026 Dario Amodei, chief executive of Anthropic, published an essay proposing "embedded evaluators": outside reviewers placed inside a company with "ongoing, employee-like access," able to check whether the company follows the practices it claims and to publish findings without the company editing them. The essay said Anthropic would do this on its own and urged other companies to follow (Amodei 2026b).

The news site Axios reported that Sam Altman, chief executive of OpenAI, said the same day that his company would give outside evaluators similar access (Berkowitz 2026). Both men lead companies that would gain from being seen as trustworthy, and the essay is its author's account of his own company's intentions. As of October 2026 these are announcements. Whether the access is delivered, and to whom, is something to watch for.

What's Still Missing

The three routes share a weakness. In each, the company controls the door.

Access that is granted can be narrowed or withdrawn, and an evaluator who depends on a company's goodwill has a reason to stay on good terms with it. The embedded-evaluator proposal tries to answer this with standing access and publication rights. It would still rest on the company's consent unless a law required it.

Conclusion

As of October 2026, company safety commitments are checked by government institutes with testing agreements, by outside evaluators, and by employees willing to speak. Each route has produced real information. Each depends on access that the companies grant, and proposals to make that access standing or mandatory haven't yet been tested.

Key Terms

• Verification: Checking, by someone other than the party that made a promise, that the promise is being kept.
• Safety institute: A government body that tests advanced AI models and studies their risks.
• Third-party evaluation: The testing of a model by an organization that didn't build it.
• Access: What an outside party is permitted to see and test, and for how long.
• Whistleblower: An employee or former employee who reports wrongdoing or danger inside an organization to people outside it.

References

• Amodei, Dario. 2026b. "We Must Pace the Frontier." September 2026.
  • Free: Dario Amodei's website
• Berkowitz, Ben. 2026. "Anthropic, OpenAI CEOs Call for Slowdown in AI Development." Axios, September 12, 2026.
  • Free: Axios
• National Institute of Standards and Technology. n.d. "Center for Advancing Innovation and Standards for Super Intelligence (CAISSI)." Accessed October 3, 2026.
  • Free: National Institute of Standards and Technology
• "A Right to Warn About Advanced Artificial Intelligence." 2024. Open letter, June 4, 2024.
  • Free: Right to Warn
• UK AI Security Institute. n.d. "About." Accessed October 3, 2026.
  • Free: UK AI Security Institute
• US Department of Commerce. 2025. "Statement from U.S. Secretary of Commerce Howard Lutnick on Transforming the U.S. AI Safety Institute into the Pro-Innovation, Pro-Science U.S. Center for AI Standards and Innovation." June 3, 2025.
  • Free: US Department of Commerce

This content reflects the field as of October 2026.

Introduction

AI companies point to their safety policies as evidence that they can be trusted with powerful systems. One group of critics reads the same policies and sees promises that are thin to begin with and that loosen when keeping them gets costly.

This reading sets out that criticism, the evidence its advocates cite, and the companies' reply. The dispute is unresolved.

The Grades

A voluntary commitment is a promise that an organization makes by its own choice and that no law requires it to keep. The critics' first claim is that the existing ones fall short of the risks the companies themselves describe.

The most systematic version is a safety index: a published scorecard that grades companies on their safety practices using stated criteria. The Future of Life Institute, a nonprofit, issues one. Its summer 2026 edition had a panel of seven researchers grade nine companies in six areas (Future of Life Institute 2026).

No company scored above C+. In the area the index calls existential safety, meaning preparation for risks that could threaten humanity as a whole, the report says: "No company exceeds C-; most score D or below." One of its headline findings is that "safety rhetoric outpaces revealed behavior" (Future of Life Institute 2026).

The Future of Life Institute campaigns for stronger regulation of advanced AI, and low grades support its case. The grades are its panel's judgments against its own criteria.

The Changes

The second claim is that commitments have loosened. The critics cite two documented cases.

In February 2026 Anthropic replaced its policy with a rewritten version. TIME magazine reported that the company had dropped the central pledge of its 2023 policy: not to train more capable systems unless it could guarantee in advance that its safety measures were adequate. Under the new version, the magazine reported, the company commits to delay development only if it considers itself the leader of the field and judges the risk of catastrophe to be significant (Perrigo 2026).

OpenAI's framework, as updated in April 2025, contains a clause about rivals. It says: "If another frontier AI developer releases a high-risk system without comparable safeguards, we may adjust our requirements." The clause sets conditions. The company would first confirm that the risks had actually changed, acknowledge the adjustment publicly, and keep safeguards "at a level more protective" (OpenAI 2025c).

The Argument

Critics connect these cases with one idea. Competitive pressure is the push a company feels to match its rivals or lose customers and investment to them. A company that holds back alone bears the cost of caution.

On this argument a voluntary commitment is weakest when a rival is about to get ahead. Chris Painter, policy director at the evaluation nonprofit METR, told TIME that without firm thresholds he worried about a "frog-boiling effect," in which danger rises gradually and no single step sets off an alarm (Perrigo 2026).

The critics' remedy is a binding rule: a requirement set by law, with a penalty for breaking it, that applies to every company alike. If every company must meet the same standard, none loses ground by meeting it. Bill Gates, co-founder of Microsoft, made a version of this argument in a September 2026 podcast interview, saying the industry can't be relied on to regulate itself (Klein 2026c).

The Companies' Reply

The companies give three answers.

• Frameworks have to change. The science of testing models is young. Jared Kaplan, Anthropic's chief science officer, told TIME: "We felt that it wouldn't actually help anyone for us to stop training AI models." The company's argument was that stopping alone would leave the pace to developers with weaker protections (Perrigo 2026).
• Changes are made in the open. The revised policies are published with their revision histories, and OpenAI's clause promises public acknowledgment of any adjustment. A later version of Anthropic's policy states that the company remains free to pause development "even if not required" by the policy (Anthropic 2026).
• Some voluntary restraint has happened. In September 2026, a monthly review of US technology policy reported, OpenAI paused training of its most capable models and said it wouldn't release a new model, citing safety concerns (Lau and others 2026).

Each answer comes from a party with a stake. The companies gain if voluntary commitments are judged adequate.

Conclusion

As of October 2026 it's documented that company safety policies have been revised, and that at least two revisions made a commitment more conditional. It's also documented that policies are published and that one company paused on its own. Whether voluntary commitments can hold under competition is a forecast on both sides. The critics' forecast leads to binding rules, and the companies' leads to continued self-governance with more transparency.

Key Terms

• Voluntary commitment: A promise that an organization makes by its own choice and that no law requires it to keep.
• Safety index: A published scorecard that grades companies on their safety practices using stated criteria.
• Competitive pressure: The push a company feels to match its rivals or lose customers and investment to them.
• Binding rule: A requirement set by law, with a penalty for breaking it, that applies to every company alike.

References

• Anthropic. 2026. "Responsible Scaling Policy." Version 3.4, effective July 8, 2026.
  • Free: Anthropic
• Future of Life Institute. 2026. AI Safety Index: Summer 2026. July 2026.
  • Free: Future of Life Institute
• Klein, Ezra, host. 2026c. "Bill Gates's Blunt Warning on A.I." The Ezra Klein Show, podcast, New York Times, September 29, 2026.
  • Publisher: The New York Times
• Lau, Rachel, Shirley Frame, Justin Hendrix, and Ashley Faler. 2026. "September 2026 US Tech Policy Roundup." Tech Policy Press, October 1, 2026.
  • Free: Tech Policy Press
• OpenAI. 2025c. "Our Updated Preparedness Framework." Version 2, April 15, 2025.
  • Free: OpenAI
• Perrigo, Billy. 2026. "Exclusive: Anthropic Drops Flagship Safety Pledge." TIME, February 24, 2026.
  • Free: TIME

This content reflects the field as of October 2026.

Introduction

When the head of an AI company warns that AI is dangerous, some listeners hear a responsible warning. Others ask why a company would advertise the dangers of its own product, and suspect that the warning is good for business.

This reading sets out the criticism that safety warnings help large companies hold off competitors, the related argument for openly released models, and the replies. The dispute is unresolved.

Regulatory Capture

Regulatory capture is a situation in which the companies being regulated shape the rules to serve themselves. The rules then protect those companies from competition more than they protect the public.

An incumbent is a company that already holds a strong position in a market. The capture argument says incumbents can afford safety teams, lawyers, and compliance staff. Rules that demand those things cost them little and cost a newcomer a great deal.

Andrew Ng, founder of the education company DeepLearning.AI and an investor in AI startups, has made this argument repeatedly. In March 2026 he wrote of occasions "when big AI companies argue that AI is dangerous to block the free distribution of open source projects that compete with their offerings" (Ng 2026a). In July 2026 he went further, writing that "a meaningful fraction of work on AI safety is no longer about safety but rather aimed at stoking fears to pursue regulatory capture" (Ng 2026c).

Jensen Huang, chief executive of the chip maker Nvidia, argued in a September 2026 podcast interview that warnings of catastrophe frighten the public without cause and that the industry needs no new regulation (Klein 2026b).

The Open-Model Argument

Much of this argument concerns one kind of product. An open-weight model is a model whose parameters have been published for anyone to download. Parameters, also called weights, are the numbers that hold what a model learned in training. Anyone with suitable hardware can run such a model, study it, and modify it.

Advocates make three claims for open release.

• It spreads capability. Researchers, small companies, and poorer countries can build on a model they couldn't afford to train.
• It allows inspection. Outsiders can examine an open model directly. Ng wrote that open release "casts sunlight on technology and ultimately makes it safer" (Ng 2026c).
• It prevents gatekeeping. Gatekeeping is control by a few parties over who may use something and on what terms. Yann LeCun, then chief AI scientist at Meta, said in a 2024 interview: "I see the danger of this concentration of power through proprietary AI systems as a much bigger danger than everything else" (LeCun 2024).

The Reply

The 2026 International AI Safety Report, an expert assessment chaired by the computer scientist Yoshua Bengio, accepts that open-weight models help research and give access to people with fewer resources. It then identifies what is different about them. "Once released, a model's weights cannot be recalled." Their "safeguards are easier to remove," and monitoring is harder because anyone can run them outside controlled settings (Bengio and others 2026, sec. 3.4).

The reply to the capture charge is that a motive doesn't settle whether a claim is true. A company can profit from a warning that is also correct.

One Proposal, Two Readings

A September 2026 episode shows both readings at once. Dario Amodei, chief executive of Anthropic, published an essay arguing that companies and governments should slow the rate at which AI capabilities improve, with outside verification (Amodei 2026b). Sam Altman, chief executive of OpenAI, said he agreed, as did Elon Musk of xAI (Berkowitz 2026).

Supporters read this as leading companies accepting limits on themselves. The technology news site The Register wrote that the proposal reads "like an attempt at 'regulatory capture'" and would give the largest companies a truce in which they needn't compete so fiercely (The Register 2026). Chamath Palihapitiya, a venture capitalist, called the essay an effort to stop open source and concentrate power with its author's company (Berkowitz 2026). Amodei's essay acknowledges that his company's advocacy has led to accusations of "regulatory capture" (Amodei 2026b).

Who Benefits From Each Position

Every party here argues from interest. That doesn't show that any of them is wrong.

Conclusion

The capture critique holds that warnings about AI danger help incumbents restrict competitors, above all the makers of open-weight models. The reply holds that open release carries risks that can't be reversed, and that a self-interested warning can still be true. As of October 2026 both sides can point to named evidence: experts document the risks of open release, and critics document who would gain from restricting it.

Key Terms

• Regulatory capture: A situation in which the companies being regulated shape the rules to serve themselves.
• Incumbent: A company that already holds a strong position in a market.
• Open-weight model: A model whose parameters have been published for anyone to download.
• Gatekeeping: Control by a few parties over who may use something and on what terms.

References

• Amodei, Dario. 2026b. "We Must Pace the Frontier." September 2026.
  • Free: Dario Amodei's website
• Bengio, Yoshua, and others. 2026. International AI Safety Report 2026: Extended Summary for Policymakers. Published February 3, 2026.
  • Free: International AI Safety Report
• Berkowitz, Ben. 2026. "Anthropic, OpenAI CEOs Call for Slowdown in AI Development." Axios, September 12, 2026.
  • Free: Axios
• Klein, Ezra, host. 2026b. "Jensen Huang Thinks A.I. Alarmism Has Gone Too Far." The Ezra Klein Show, podcast, New York Times, September 23, 2026.
  • Publisher: The New York Times
• LeCun, Yann. 2024. Interview by Lex Fridman. Lex Fridman Podcast no. 416, transcript, March 8, 2024.
  • Free: Lex Fridman Podcast transcript
• Ng, Andrew. 2026a. "How Anti-AI Propaganda Hurts the Public." The Batch, DeepLearning.AI, March 27, 2026.
  • Free: The Batch
• Ng, Andrew. 2026c. "When Guardrails Go Wrong." The Batch, DeepLearning.AI, July 24, 2026.
  • Free: The Batch
• The Register. 2026. "Big AI Sets Out Its Terms for Regulatory Capture and Calls It 'Pace the Frontier'." September 14, 2026.
  • Free: The Register

Introduction

The Frontier AI Safety Commitments are often cited as proof that AI companies have agreed to safety standards. The document is short, and reading it closely shows what was agreed, who decides whether it's been done, and what the text leaves out.

This reading goes through the commitments in order and asks the same three questions of each: what is promised, who decides, and who checks.

Locating the Passage

The text is on the UK government's website, GOV.UK, under the title "Frontier AI Safety Commitments, AI Seoul Summit 2024." It's free and listed in the References. The page was published on May 21, 2024 and last updated on February 7, 2025. All quotations are from this page (Department for Science, Innovation and Technology 2024).

The page opens with a list of organizations. A short preamble follows, and then eight commitments numbered with Roman numerals, I to VIII. They're grouped under three headings that the document calls outcomes. This reading cites commitments by their numerals.

The preamble sets the terms. The organizations "undertake to develop and deploy their frontier AI models and systems responsibly," in accordance with what the text calls "voluntary commitments." A footnote defines frontier AI as "highly capable general-purpose AI models or systems that can perform a wide variety of tasks."

Walking Through the Passage

Step 1: Read the commitment to assess risks and identify who does the assessing

Outcome 1 reads: "Organisations effectively identify, assess and manage risks when developing and deploying their frontier AI models and systems." Five commitments sit under it.

Commitment I begins: "Assess the risks posed by their frontier models or systems across the AI lifecycle, including before deploying" them. The lifecycle means the whole span from training a model to running it for the public. So the promise covers testing during development as well as before release.

The subject of the sentence is the organization. Each company assesses its own models. The commitment goes on to mention evaluations from inside and outside the company, and it leaves the choice of whether and when to use outsiders to the company.

Step 2: Read the commitment on thresholds and find who sets them

Commitment II reads: "Set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable." A threshold here is a line: a level of risk that the company says it won't accept without protections.

The verb is "set out," and again the subject is the organization. Each company draws its own line. The text adds one qualification: "thresholds should be defined with input from trusted actors, including organisations' respective home governments." That is input. The governments aren't given a decision, and "trusted actors" isn't defined.

Commitments III and IV follow from this. Under III, organizations "articulate how risk mitigations will be identified and implemented to keep risks within defined thresholds." Under IV, they "set out explicit processes they intend to follow" if a model's risks "meet or exceed the pre-defined thresholds."

Commitment IV contains the strongest sentence in the document. In the extreme, "organisations commit not to develop or deploy a model or system at all, if mitigations cannot be applied" to keep risks below the thresholds. This is a promise to stop. Whether the condition has been met is for the company to judge, against a threshold the company set.

Commitment V is to "continually invest in advancing their ability to implement commitments i-iv."

Step 3: Read the commitment on transparency and its exceptions

Outcome 2 is about accountability inside the company. Commitment VI promises "internal accountability and governance frameworks," with people assigned roles and responsibilities.

Outcome 3 turns outward: "Organisations' approaches to frontier AI safety are appropriately transparent to external actors, including governments." The word "appropriately" already signals a limit.

Commitment VII promises public transparency about how the earlier commitments are carried out. It then gives an exception: "except insofar as doing so would increase risk or divulge sensitive commercial information to a degree disproportionate" to the benefit to society. Two reasons for withholding are named, safety and commercial sensitivity, and the company weighs them. The commitment says more detailed information can go to "trusted actors, including their respective home governments."

Commitment VIII is to explain how "external actors, such as governments, civil society, academics, and the public are involved in the process of assessing" risks. The promise is to explain the involvement. It sets no minimum for how much involvement there must be.

Step 4: Look for an enforcement mechanism

Read the document again with one question: what happens to a company that doesn't do these things?

The text has no answer. It names no body that receives the frameworks or judges them. It sets no audit, no penalty, and no procedure for removing a signer. The two governments "announced" that the organizations had agreed. They didn't sign the commitments as parties with duties of their own.

The one concrete deliverable has a date. Organizations were to show how they'd met the commitments "by publishing a safety framework focused on severe risks by the upcoming AI Summit in France," which took place in February 2025. Publication can be checked by anyone. The quality of what was published, and whether a company follows it, can't be checked from the text.

Step 5: List who signed and from which countries

The page lists sixteen initial signers: Amazon, Anthropic, Cohere, Google, G42, IBM, Inflection AI, Meta, Microsoft, Mistral AI, Naver, OpenAI, Samsung Electronics, Technology Innovation Institute, xAI, and Zhipu.ai. Four were added later: Magic, Minimax, 01.ai, and NVIDIA.

The page gives names only. From public information about where each is based, most are American. The others are in Canada (Cohere), France (Mistral AI), South Korea (Naver and Samsung Electronics), the United Arab Emirates (G42 and Technology Innovation Institute), and China (Zhipu.ai, Minimax, and 01.ai).

That spread is notable. Chinese and American companies signed the same text, at a time when their governments agreed on little about AI. It also shows the limits of the list. Signing was open to those who chose to, and developers who didn't sign aren't covered at all.

Key Considerations

The verbs carry the meaning. The commitments say "assess," "set out," "articulate," and "invest." Each describes something an organization does on its own account. None says "submit," "obtain approval," or "comply." A reader should also notice the qualifiers: "appropriately transparent," and transparency "except insofar as" the company judges disclosure disproportionate.

A common mistake is to read a voluntary commitment as a rule with a penalty. Headlines that say companies "agreed to safety standards" invite this. The companies did agree, and the agreement is public, which lets journalists, researchers, and governments compare conduct with words. A rule would add a party with the power to check and to punish. The document doesn't.

A second mistake runs the other way: to conclude that the commitments are empty. They produced published frameworks from many signers, which didn't exist in comparable form before. The commitments created something outsiders can read and criticize. They didn't create a way to compel anything.

Summary

The Seoul commitments are specific about what companies will write down and publish, and they leave every judgment to the companies themselves. The result of the reading is this table.

References

• Department for Science, Innovation and Technology. 2024. "Frontier AI Safety Commitments, AI Seoul Summit 2024." May 21, 2024.
  • Free: GOV.UK