Building a Defensible AI Governance Program

Our initial sharing argued that artificial intelligence (“AI”) is not a separate body of law. Rather, it is a new factual setting in which longstanding legal doctrines—including fiduciary oversight, privilege, confidentiality, employment law, consumer protection, and corporate compliance—are increasingly applied.

The practical question for General Counsel therefore is not whether AI presents legal risk. It is whether the organization can demonstrate that it exercised reasonable judgment before those risks materialized. The distinction is critical because governance is almost always evaluated retrospectively, and that circumstance is not novel. When regulators investigate corporate misconduct, they examine the compliance program that existed before the alleged violation. When shareholders challenge board oversight, they examine the information directors received before the crisis. When juries evaluate corporate conduct, they ask what management knew and what management did before the injury occurred. Questions about how the company treated artificial intelligence should be expected to follow the same pattern.

In fact, the Department of Justice’s September 2024 revisions to the Evaluation of Corporate Compliance Programs (“ECCP”) directs federal prosecutors to examine precisely these issues. Prosecutors are instructed to consider whether a company has assessed the impact of AI and other emerging technologies on its ability to comply with criminal laws, whether AI-related risks are integrated into enterprise risk management, what governance exists over the company’s use of AI, and how the company mitigates deliberate or unintended misuse of those technologies.¹

Although the ECCP is guidance rather than binding law, it reflects the Department’s expectations regarding effective corporate compliance. For organizations operating in regulated industries—or those concerned with fraud, False Claims Act exposure, Foreign Corrupt Practices Act risk, or other white-collar matters—those expectations warrant careful attention.

Step One: Know Where AI Already Exists

Organizations frequently begin AI governance by drafting an AI policy. That is understandable. It is also, in many cases, not where to begin. Policies govern known activities. Governance begins by identifying those activities.

The DOJ’s updated ECCP asks prosecutors to determine what technologies Company A uses to conduct business, whether Company A has evaluated the associated compliance risks, and whether those risks have been incorporated into broader enterprise risk management.¹

Similarly, the National Institute of Standards and Technology’s Artificial Intelligence Risk Management Framework (AI RMF 1.0) begins not with technology controls, but with organizational understanding. The Framework’s “Core” points to four interrelated assessment functions (Govern, Map, Measure, and Manage) to emphasize establishing governance structures, identifying AI systems within the organization, understanding the context in which they operate, and evaluating the risks they may create.²

That translates into a practical legal question for General Counsel: Can the organization identify where AI is being used in ways that create legal risk?

The answer should extend beyond a software inventory. Legal departments should understand:

• Which business units employ AI;
• Whether AI systems are internally developed or procured from third parties;
• What categories of information are submitted to those systems;
• Whether AI influences customer-facing, employment, financial, or regulatory decisions;
• Whether AI interacts with confidential, proprietary, or regulated information; and
• Who is responsible for legal oversight of each material implementation.

Notice what this inventory does not require. While lawyers must have a basic understanding of the benefits and risks associated with AI, see Professional Ethics Rule 1.1, it does not require counsel to become machine-learning engineers. Rather, it enables legal departments to identify where established legal obligations intersect with AI-assisted business processes. That distinction is consistent with both the NIST AI RMF and the DOJ’s emphasis on risk assessment rather than technological sophistication.¹²

Practice Point: An AI inventory fundamentally should be a legal risk assessment—not merely a technology or software inventory.

Step Two: Assign Responsibility Before Problems Arise

The second recurring characteristic of mature compliance programs is accountability. Organizations routinely identify individuals responsible for cybersecurity, export controls, anti-corruption compliance, financial reporting, workplace safety, and privacy. AI frequently lacks comparable organizational ownership. And such an absence creates legal risk.

The DOJ’s ECCP repeatedly emphasizes responsibility, accountability, and management oversight as hallmarks of an effective compliance program. Similarly, ISO/IEC 42001 requires organizations implementing an Artificial Intelligence Management System to establish leadership responsibility, define organizational roles, and assign accountability for AI governance.³

Neither authority mandates a particular organizational chart, which makes sense. A multinational financial institution may require a dedicated AI governance committee. A regional manufacturer may reasonably assign responsibility through an existing enterprise risk management committee. The legal principle is therefore not organizational uniformity. It is organizational clarity.

If significant AI decisions are later scrutinized by regulators, insurers, shareholders, or juries, organizations should be able to identify who evaluated legal risks, who approved deployment, and who retained continuing oversight.

Step Three: Integrate AI Into Existing Compliance Systems

One misconception surrounding AI governance is that it requires constructing an entirely separate compliance infrastructure. Neither the DOJ nor the principal governance frameworks suggest that approach. Instead, they emphasize integrating new technological risks into existing governance systems. The DOJ specifically asks whether AI-related risks have been incorporated into broader enterprise risk management strategies.¹

That question has important practical consequences. Rather than creating an isolated “AI office,” many organizations may find greater success by integrating AI review into existing governance processes. For example:

• Procurement should evaluate AI vendors using existing third-party risk management procedures;
• Information Security should assess AI systems through existing cybersecurity review processes;
• Privacy officers should evaluate AI systems processing personal information under existing privacy governance;
• Employment counsel should review AI-assisted hiring tools through established employment law processes;
• Litigation counsel should evaluate records retention, privilege, and discovery implications before AI systems are implemented.

This integrated approach reflects an important governance principle. Artificial intelligence should become another enterprise risk managed through established governance structures—not an exception operating outside them.

Step Four: Policies Should Govern Judgment, Not Products

One of the quickest ways for an AI policy to become obsolete is to write it around today’s software. Technology will change; the underlying governance principles will not. Accordingly, organizations should avoid policies that simply identify approved or prohibited AI platforms. Instead, policies should establish decision-making principles that remain applicable regardless of technological evolution. Among other matters, policies should address:

• What categories of information may be submitted to external AI systems;
• When legal review is required before implementation;
• When procurement must negotiate AI-specific contractual protections;
• When human review of AI-generated outputs is mandatory;
• How AI-related incidents are reported and investigated; and
• How exceptions to policy are documented and approved.

There is no common answer across different entities. This principle is reflected in the NIST AI RMF’s emphasis on governance processes rather than technology-specific controls and in ISO/IEC 42001’s management-system approach, which is designed to support continual improvement as technologies evolve.² ³

Governance Is Only Half the Story

By this point, an organization should have identified where AI is deployed, assigned responsibility for governance, integrated AI into existing compliance structures, and adopted policies focused upon organizational judgment rather than particular software platforms. Those are necessary steps. They are not sufficient. The question every experienced trial lawyer eventually asks is different: if litigation began tomorrow morning, could the organization demonstrate that its AI governance program actually worked?

That question shifts the discussion from governance to defensibility. Courts do not evaluate written policies in the abstract. Regulators do not investigate organizational charts. Jurors rarely decide cases based upon mission statements. Instead, decision-makers evaluate evidence. Accordingly, mature AI governance should produce reliable documentation long before litigation begins.

That observation is consistent with the Department of Justice’s recurring emphasis that compliance programs must be effective “in practice,” not simply mellifluous or well written. Prosecutors are instructed to evaluate whether compliance programs actually function, whether they evolve over time, and whether organizations learn from experience.¹

Governance Must Produce Evidence

One of the recurring themes throughout modern corporate enforcement is documentation. The question is not just whether management considered a risk. It is whether the organization can demonstrate that it did. Accordingly, significant AI implementation decisions should ordinarily produce contemporaneous documentation reflecting:

• The proposed business purpose;
• Legal and regulatory issues considered;
• Participating reviewers;
• Significant contractual concerns;
• Identified risks;
• Mitigation measures adopted;
• Implementation conditions and/or documented testing;
• Responsible business owners; and
• Scheduled follow-up review.

These documents need not be lengthy. In fact, concise documentation often proves more useful than lengthy memoranda that no one reads. The objective is not creating paperwork. The goal is to create evidence of thoughtful governance before an event occurs. That distinction becomes particularly important if the organization later attempts to demonstrate good-faith compliance to prosecutors, regulators, insurers, or courts.

Records Retention And AI Considerations

AI introduces another practical issue that many organizations have not yet fully addressed: what constitutes the business record? Traditional document-retention programs typically address emails, contracts, financial records, electronically stored information, text messages, and collaboration platforms. Increasingly, they may also need to evaluate:

• AI prompts;
• AI-generated outputs incorporated into business decisions;
• Approval histories;
• AI-assisted drafting workflows;
• Audit logs;
• Governance committee records;
• AI incident reports; and
• Vendor communications relating to AI systems.

The answer will vary according to the organization’s business, applicable legal obligations, and the role AI plays within decision-making. Not every prompt warrants preservation. Not every output becomes a corporate record. However, organizations should make those determinations deliberately rather than accidentally.

The Federal Rules of Civil Procedure have long required preservation of relevant electronically stored information when litigation is reasonably anticipated. Those obligations are technology neutral. If AI-generated information becomes relevant evidence, existing preservation principles—not new AI rules—are going to govern. Fed. R. Civ. P. 26, 34, 37(e).

Internal Investigations Will Become More Complex

AI also changes how internal investigations are conducted. Historically, investigators sought emails, financial records, contracts, text messages, and witness interviews. Increasingly, investigations may also require examination of:

• Prompts submitted to AI systems;
• AI-generated recommendations;
• Model configuration settings;
• Human review processes;
• Approval chains;
• Vendor documentation;
• AI training materials;
• Governance committee records; and
• System access logs.

Importantly, investigators should resist treating AI as the subject of the investigation. Instead, AI frequently becomes another source of evidence concerning traditional misconduct; perhaps even a conduit. A False Claims Act investigation may examine AI-assisted billing. A securities investigation may examine AI-generated disclosures. An employment investigation may examine AI-assisted hiring. A commercial fraud investigation may examine AI-generated customer communications.  The legal theories remain familiar. The evidence and how to obtain it becomes a bit more technologically complex. Counsel should assess these considerations.

Conclusion

AI is unlikely to become the next isolated legal specialty. Instead, it is becoming another lens through which courts, regulators, prosecutors, insurers, shareholders, and juries evaluate organizational judgment. That reality should be reassuring rather than intimidating.

General Counsel already understand how to govern significant enterprise risk. They already understand privilege, investigations, contracts, discovery, and board reporting. AI does not require abandoning those disciplines. It does require applying them earlier, more deliberately, and across a broader range of business functions. Organizations that view AI solely as a technology initiative may eventually find themselves defending decisions they never realized they made. Organizations that view AI as an enterprise governance issue will be better positioned to demonstrate not only that they adopted innovative technologies, but that they did so responsibly.

Principal Authorities

1. U.S. Department of Justice, Criminal Division, Evaluation of Corporate Compliance Programs (Updated Sept. 23, 2024) at 10, (Instructing federal prosecutors to ask, in part, “[t]o To the extent that the company uses AI and similar technologies in its business or as part of its compliance program, are controls in place to monitor and ensure its trustworthiness, reliability, and use in compliance with applicable law and the company’s code of conduct?”).
2. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) at 20-33 (Jan. 2023), particularly the GOVERN and MAP functions addressing governance structures, organizational context, and AI risk management.
3. ISO/IEC 42001:2023, Information Technology—Artificial Intelligence—Management System, addressing leadership, governance, organizational roles, risk assessment, and continual improvement (consensus standard).
4. Responsibility, Formal Opinion 512: Generative Artificial Intelligence Tools (July 29, 2024), addressing competence, confidentiality, supervision, communication, and billing obligations when lawyers use generative AI.
5. Federal Trade Commission, Keep Your AI Claims in Check (Feb. 2023) and The Luring Test: AI and the Engineering of Consumer Trust (2024), emphasizing that businesses remain responsible for deceptive or unsupported AI-related representations.

The Robins Kaplan Emergency Envelope