Over the last year, a California startup has built an AI customer-support platform. Engineers have fed hundreds of thousands of real customer-support conversations into their model in order to train to better answer user questions. The model has produced exactly the results the company was anticipating, and it begins preparations for launch.

Before launch, the company brings in an AI lawyer for startups to review their model. They point out that because the conversations used contain customer names, email addresses, account information, and confidential business details the company now faces serious questions about AI training data compliance. 

Namely, did the company have customer agreements in place that permitted using these conversations to train their AI model?

With the passage of California's AI training data transparency law, developers of covered generative AI systems now face new documentation and disclosure obligations around the data used to train their AI. 

For startups, understanding their AI legal compliance obligations can't wait until a product is ready to launch. They need to understand the legal risks of training AI models, so that compliance doesn't become an afterthought. 

What is the California AI Training Data Transparency Act?

California's AI training data law, AB 2013, went into effect on January 1, 2026. The act addresses data transparency in generative artificial intelligence by requiring developers to publish documentation about the datasets used to train their systems. 

For covered generative AI systems, what this means is that a developer must make information available to the public that documents the source or owners of the data used to develop the system. That includes a general description of the data, the purpose the data serves in developing the system, the number and types of data points, and whether the data includes copyright, trademark, or patent-protected materials. 

The California AI training data transparency act also requires developers to specify whether data was purchased or licensed, whether it contains personal information, how the datasets were collected, processed, or modified, and when they were first used. 

For startups, this creates a practical AI model training legal issue: they need to understand and accurately document what data was used to develop their model. That means practicing strong data governance as early as possible in the development cycle, rather than waiting until just before launch. 

Because AB 2013 specifically includes testing, validation, and fine-tuning in its broad definition of AI training, Startups need to account for these processes when maintaining the documentation required by the law. Developers should know what information their datasets contain, and maintain accurate records throughout development and fine-tuning. 

One last crucial detail: compliance with the California AI Training Data Transparency Act doesn't mean that all training data is legally permissible. AB 2013 is about transparency in the data used to train AI systems. It does not guarantee that an organization has the right to use copyrighted, personal, or confidential information. A company must still determine in-house whether they are legally entitled to use that data in the first place. Those are two separate questions.  

Eight Legal Risks of Training AI Models California Startups Must Address

Before launching an AI model or service, startups should consider these AI model training legal issues.

Risk #1: Copyright and Intellectual Property Issues

Just because content is publicly accessible on the internet does not mean it is automatically free to use to train AI systems. Fair use can apply in some circumstances, but it's not a given. Copyright owners can potentially object to unauthorized copying and use of their works in AI development. 

The U.S. Copyright Office has said that the law's application to AI training is very case and fact-specific. Depending on the specific circumstances, companies may need to lawfully acquire or license the data they intend to use for training. 

Before beginning training, startups should document where each dataset comes from, who owns it, whether it was licensed, and what the license permits. If copyrighted material is included, lawful acquisition may be required. 

Risk #2: Privacy and Personal-Data Violations

Depending on the source, training data can potentially hold personal information. A company that trains an AI on millions of customer reviews may have included names, email addresses, locations, medical and financial information, and other personal details in the dataset. 

Using personal information to train AI isn't always a legal risk, but if the startup is subject to California's CCPA, it must consider the law's requirements governing how consumers' protected information is collected, used, shared, and sold. 

A startup needs to consider how they will notify customers of their intended usage. If the company tells users “we use your information only to provide our services”, then uses customer conversations to train their commercial AI model, that can create legal risk. The FTC has warned AI companies that their privacy and confidentiality commitments can apply to AI training as well, and may face consequences for unlawful use of personal data to develop AI models. 

Risk #3: Confidential Information & Trade Secrets

For B2B AI startups, it's not just user personal data that presents a risk, but an enterprise customer's confidential information.

Take, for example, a company that introduces an AI coding assistant to businesses. Customers may upload proprietary source code, product designs, customer lists, manufacturing processes, business strategies, and internal financial information, all of which can contain confidential information or trade secrets. 

The company producing the AI system needs to understand whether its agreements with the client actually permit them to use customer information for model training. To ensure AI legal compliance, startups should review their terms of service, customer agreements, data-processing agreements, vendor agreements, and other contracts before launch.

Risk #4: Failing to Maintain Accurate Training-Data Records

Startups that don't keep accurate records of the training data they use can have a more difficult time staying compliant with AB 2013. Data provenance, the documented history of a piece of data, should identify its original source, created or owner, and whether it was licensed, collected internally, or synthesized. 

Keeping accurate records when development involves multiple vendors, scraped material, licensed datasets, and repeated fine-tuning requires diligence, but it can help the company respond to copyright, privacy, and contractual obligations. 

Risk #5: Bias and Discrimination

Training data has the potential to reproduce certain patterns or biases in AI output.  Biases are of particular concern when AI systems used to influence employment, housing, lending, insurance, healthcare, or other consequential decisions. California's Attorney General has emphasized that civil-rights and other existing laws can apply to AI-assisted decision-making. Depending on the use case and applicable laws, companies should consider testing for discriminatory outcomes and consider appropriate human oversight around AI-assisted decisions. 

Risk #6: Security Vulnerabilities and Data Leakage

Generative AI startups can face significant legal risks when a model can be manipulated into revealing information or behaving in a way that developers didn't anticipate. Prompt injection, where attackers use malicious inputs to manipulate AI systems into leaking sensitive information or taking unintended actions, is one example. Data leakage, model extraction, and exposure of confidential training data are other security risks that developers should consider in pre-launch testing. 

Risk #7: Misleading Claims About AI Capabilities

It's not just developers who can create unintentional risk. A company's marketing teams must be sure not to overstate claims of what their products can do. For example, statements like “100% accuracy”, “zero hiring bias”, or “our AI never hallucinates" cannot be difficult to substantiate, and may create consumer protection problems if they are false or misleading.

Risk #8: Third-Party Model, Dataset, and Software Licenses 

Many AI startups aren't creating their AI models from scratch. They might use an open source model, or license one from one of the major providers like OpenAI or Anthropic. They combine these off-the-shelf models with proprietary customer data, fine tune them, and create a specialized application. 

However, that creates another layer of legal risk. Startups need to understand what their upstream AI providers actually permit. Open source doesn't always mean “unrestricted”. Companies need to know if the model they rely on can be commercially used or fine tuned? What rights and restrictions apply to commercial use, fine-tuning, inputs, outputs, or particular use cases? AI legal compliance for startups requires keeping detailed inventory of every dataset, model, API, and vendor involved in development.

The Essential Legal Checklist for AI Startups

Before releasing an AI product, startups should be able to answer six basic questions.

What data was used?

AI startups need to accurately keep track of what data was used to train their models, the fine-tuning, testing, and validation methods used, and what customer or third-party datasets were used to train the model.

Do we have the right to use this data?

Data provenance is essential for AI training data compliance. You must document the source of each dataset, and confirm that the copyright, privacy, confidentiality, contractual, and licensing requirements have been met before launching the product.

What are we telling users?

Marketing claims should not exaggerate the model's capabilities. Likewise, privacy notices, terms of service, and disclosures should accurately describe how AI and user data is collected and used. 

What does California require?

Startups developing covered generative AI systems to be made available in California should determine whether AB 2013 applies and create a system for providing the required training data documentation before launch, and before any substantial modification or update to the model. 

What happens when the model is used?

Companies should test for foreseeable privacy, security, bias, accuracy and misuse risks. 

How will we stay compliant?

Assign ownership for monitoring legal requirements, updating documentation, and reviewing new training data. Changes or modifications to the model may require updating documentation to stay compliant. 

Unsure About the Legal Risks of Generative AI for Startups?

AI startups can move quickly without treating legal compliance as an afterthought. Compliance is not a one-time, check-the-box exercise. AI models, training data, and applicable laws can change throughout development and deployment. Addressing data rights, privacy, security, and California's transparency requirements early can help reduce costly problems later.

Need help navigating the legal risks of developing or launching an AI product? SVTech can help your startup build a practical legal strategy for AI development and deployment. 

Contact us today for an initial consultation on the legal issues of AI model training.