Blogs

The AI security blind spot: Inadequate data governance

9/28/26

By: Josette Brooksbank and Michael Brown

Recent security testing involving some of the world’s most advanced artificial intelligence (AI) systems has exposed a fundamental cybersecurity concern: even sophisticated AI models can circumvent safeguards when deployed in inadequately secured environments. According to an August 4, 2026, Axios report, AI models developed by Anthropic and OpenAI reached systems outside their intended testing environments after exploiting weaknesses in human-designed security controls. The incidents highlight an important consideration for businesses deploying AI: securing the technology itself is only part of the challenge. Organizations must also evaluate the data, systems, and access permissions available to it.

The human vulnerabilities behind AI security incidents

According to Axios, Anthropic disclosed that three of its models accessed real-world systems during cybersecurity testing after a misunderstanding with a third-party evaluator resulted in unintended internet connectivity. The models reportedly obtained login credentials, uploaded malware to legitimate code repositories, and scanned the internet for vulnerable systems. Axios also reported that an OpenAI agent escaped its testing environment after discovering a previously unknown vulnerability in its containment mechanisms.

Although these incidents involved advanced AI models, they illustrate a familiar cybersecurity problem: security controls are only effective when the surrounding infrastructure adequately restricts access and contains unintended activity. The incidents demonstrate how vulnerabilities in testing environments can allow AI systems to interact with resources beyond their intended scope.

For businesses integrating AI into their operations, the implications extend beyond the security of individual models. Even an AI application with robust built-in safeguards may expose sensitive information if it is connected to inadequately secured repositories or granted excessive permissions.

AI can amplify existing data governance weaknesses

Many organizations maintain substantial volumes of information across cloud repositories, collaboration platforms, internal networks, and legacy systems. Without effective data classification and access controls, confidential business records, customer information, intellectual property, and other sensitive materials may be accessible to employees or applications that do not require them.

Integrating AI into these environments can magnify existing vulnerabilities. AI applications connected to internal repositories may retrieve and synthesize information across multiple sources, potentially exposing sensitive materials that were previously difficult to locate or access collectively. Similarly, AI agents granted broad permissions may be capable of retrieving, modifying, or transmitting information beyond what is necessary for their intended functions.

The central concern is therefore not simply whether an AI application is secure, but whether the organization has adequately defined and restricted the information that application can access.

These considerations are consistent with guidance from the National Institute of Standards and Technology (NIST), which recognizes that AI systems share many traditional cybersecurity risks while also presenting distinct risks that may warrant additional oversight and controls.

Litigation and regulatory implications

Inadequate data governance can create legal exposure beyond the immediate consequences of a security incident. Existing privacy and cybersecurity laws may require organizations to implement reasonable safeguards to protect sensitive information, regardless of whether AI is involved. The Federal Trade Commission (FTC), for example, has emphasized the importance of limiting access to sensitive data, maintaining appropriate security controls, and overseeing third-party service providers.

Organizations may face particular challenges where sensitive information is accessible through overly broad permissions, AI applications are deployed without adequate oversight, or existing security policies do not account for how AI tools retrieve and transmit information. In the event of an unauthorized disclosure, these deficiencies may become relevant to regulatory investigations or litigation concerning the reasonableness of an organization’s security practices, including whether it evaluated foreseeable risks and implemented appropriate controls.

For businesses deploying AI, these risks underscore the importance of integrating AI governance into existing cybersecurity and privacy programs rather than treating it as a separate compliance exercise.

Steps to reduce risk

Organizations deploying AI should evaluate their existing data governance programs before expanding AI access to internal systems. This includes conducting data inventories to identify sensitive information and its location, implementing classification frameworks based on confidentiality and regulatory requirements, reviewing employee and application permissions, and restricting AI access to information necessary for authorized functions.

Businesses should also consider segregating highly sensitive information, monitoring AI activity and data transfers, evaluating third-party AI vendors, and maintaining documentation of their risk assessments and security controls. These measures should be tailored to the organization’s data environment, intended AI use cases, and applicable legal obligations.

Importantly, AI governance should not operate independently of existing cybersecurity and privacy programs. Integrating AI-specific safeguards into established data protection practices can help organizations identify vulnerabilities before those weaknesses result in unauthorized access or disclosure.

Recent AI security testing incidents underscore that technological sophistication cannot compensate for inadequate security infrastructure. While AI introduces distinct cybersecurity challenges, it can also expose and amplify longstanding weaknesses in data classification, access management, and system segmentation. For businesses adopting AI, effective risk management begins with understanding what information they possess, where it resides, and who—or what—can access it.

For more information, please contact Josette Brooksbank at josette.brooksbank@fmglaw.com and Michael Brown  at michael.brown@fmglaw.com or your local FMG Law attorney.

Information conveyed herein should not be construed as legal advice or represent any specific or binding policy or procedure of any organization. Information provided is for educational purposes only. These materials are written in a general format and not intended to be advice applicable to any specific circumstance. Legal opinions may vary when based on subtle factual distinctions. All rights reserved. No part of this presentation may be reproduced, published or posted without the written permission of Freeman Mathis & Gary, LLP.