California’s Transparency in Frontier Artificial Intelligence Act

California has enacted the Transparency in Frontier Artificial Intelligence Act (Cal. SB 53). The new law does not take effect until January 1, 2027.

Transparency in Frontier Artificial Intelligence Act

This new law is a pared-down version of 2024’s SB 1047 (Cal. SB 1047), which the California Governor rejected. The focus is on very large AI models (“frontier models”) developed by very large companies (“large frontier developers”).

A ”frontier model” of an AI system is

A foundation model trained using a quantity of computing power greater than 10^26 integer or floating-point operations.

“Foundation model” means an artificial intelligence model that is all of the following:

1. Trained on a broad data set.
2. Designed for generality of output.
3. Adaptable to a wide range of distinctive tasks.

For those readers who didn’t quite follow the math in the definition, suffice it to say that the AI models would have to be trained using very large amounts of computing power—in other words, this definition appears to be aimed at only the largest AI models currently being used.

Somewhat similar to its predecessor bill, this new law is concerned with “catastrophic risks”:

“Catastrophic risk” means a foreseeable and material risk that a frontier developer’s development, storage, use, or deployment of a frontier model will materially contribute to the death of, or serious injury to, more than 50 people or more than one billion dollars (US$1,000,000,000) in damage to, or loss of, property arising from a single incident involving a frontier model doing any of the following:

1. Providing expert-level assistance in the creation or release of a chemical, biological, radiological, or nuclear weapon.
2. Engaging in conduct with no meaningful human oversight, intervention, or supervision that is either a cyberattack or, if the conduct had been committed by a human, would constitute the crime of murder, assault, extortion, or theft, including theft by false pretense.
3. Evading the control of its frontier developer or user.

Large frontier developers

The new law defines a “large frontier developer” as “a frontier developer that together with its affiliates collectively had annual gross revenues in excess of five hundred million dollars (US$500,000,000) in the preceding calendar year.”

The large frontier developers must create and “conspicuously publish” on their website a “frontier AI framework,” which SB 53 describes in depth. The framework must describe how the large frontier developer approaches 10 different items, including how national and international standards and industry-consensus best practices are incorporated in its frontier AI framework, to using third parties to assess catastrophic risks, to “cybersecurity practices to secure unreleased model weights from unauthorized modification or transfer by internal or external parties,” and “assessing and managing catastrophic risk resulting from the internal use of its frontier models, including risks resulting from a frontier model circumventing oversight mechanisms.”

SB 53 also requires an annual review and updating of the framework, or more frequently in the event of a material modification.

In addition, the large frontier developer must publish a “transparency report” before, or concurrently with, deploying a new frontier model or a substantially modified frontier model. The report must contain several named items, ranging from languages supported and modalities of output supported, to intended uses of the model, to summaries of its catastrophic risk assessments.

SB 53 permits the developer to redact trade secrets, the developer’s cybersecurity, or national security, and certain other information from both the framework and the transparency report, but must retain the redacted information for five years.

Critical safety incident reporting

Somewhat similar to security breach reporting forms that some states require, the California Office of Emergency Services will create a form for developers and the public to report a critical safety incident relating to a frontier model. According to the new law:

“Critical safety incident” means any of the following:

1. Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury.
2. Harm resulting from the materialization of a catastrophic risk.
3. Loss of control of a frontier model causing death or bodily injury.
4. A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk.

The frontier model developer must report within 15 days of discovering a critical safety incident. That reporting deadline shortens to 24 hours if a critical safety incident poses an imminent risk of death or serious physical injury

Penalties

With respect to penalties, the new law states that if a large frontier developer

• fails to publish or transmit a compliant document required to be published or transmitted under this chapter,
• makes a statement in violation of subdivision (e) of Section 22757.12,
• fails to report an incident as required by Section 22757.13, or
• fails to comply with its own frontier AI framework;

the developer will “be subject to a civil penalty in an amount dependent upon the severity of the violation that does not exceed one million dollars (US$1,000,000) per violation.” There is no private right of action, as the new law provides that the penalty “shall be recovered in a civil action brought only by the Attorney General.”

Whistleblowers

In contrast, there is a private right of action with respect to whistleblowers. The private plaintiff whistleblower may bring an action for an injunction, but the law specifically permits the court to award “reasonable attorney’s fees to a plaintiff who brings a successful action for a violation of this section.”

The large frontier developer must establish a mechanism where employees can report to the developer “if the covered employee believes in good faith that the information indicates that the large frontier developer’s activities present a specific and substantial danger to the public health or safety resulting from a catastrophic risk or that the large frontier developer violated” this law. The new law also requires that the employer post notices about this reporting procedure or provide an annual notice directly to employees. With respect to the request for injunctive relief, the law adds: “In addition to any harm resulting directly from a violation of this section, the court shall consider the chilling effect on other covered employees asserting their rights under this section in determining whether temporary injunctive relief is just and proper.” In addition, “injunctive relief granted pursuant to this section shall not be stayed pending appeal.”

Final note

Although the new law is focused on very large models and very large developers, note that the beginning of the law states: “In the future, foundation models developed by smaller companies or that are behind the frontier may pose significant catastrophic risk, and additional legislation may be needed at that time.” We will have to wait and see if California expands the reach of this new law.