Enterprise AI adoption is accelerating, but security teams are right to be cautious. When you connect a large language model to your company’s data, you’re introducing a new category of risk that traditional security frameworks weren’t designed to address.

The good news: these risks are manageable with the right architecture. The bad news: most teams skip the security review because the AI demo was too exciting to slow down for governance. By the time the security team is looped in, a pilot is already in production and the conversation becomes a retrofit rather than a design review. Retrofits are always more expensive and less effective than building the controls in from the start.

Here’s the security checklist your AI deployment actually needs. Treat it as a living document — AI capabilities and the threats that come with them are evolving fast, and the controls that seem sufficient today will need to be revisited every few quarters.

Data Privacy

What data touches the model?

Map every data flow between your systems and the AI model. For each flow, classify the data sensitivity:

  • Public: Marketing copy, product documentation, public knowledge base
  • Internal: Team communications, project plans, internal processes
  • Confidential: Customer data, financial records, HR information
  • Restricted: Trade secrets, authentication credentials, PII

Rule: Never send Restricted data to a third-party model without explicit legal review and enterprise data agreements. The classification exercise sounds bureaucratic, but it’s the one step that prevents the most expensive class of mistake. A team that hasn’t explicitly categorised its data tends to default to “probably fine” when deciding what to send to a model — and “probably fine” is how customer PII ends up in a prompt log at a third-party provider.

Where is the data processed?

  • Third-party API (OpenAI, Anthropic, Google): Your data leaves your infrastructure. Check the provider’s data processing agreement, retention policies, and training data policies.
  • Cloud-hosted model (Azure OpenAI, AWS Bedrock): Data stays within your cloud provider’s environment. Better isolation, but still third-party infrastructure.
  • Self-hosted model (Llama, Mistral on your infrastructure): Maximum control. Data never leaves your environment. Higher operational cost.

Data retention

  • Does the AI provider retain your prompts and responses?
  • For how long?
  • Can they use your data to train future models?
  • Can you request deletion of your data?

Action: Get written confirmation of data retention and usage policies from every AI provider you use. Marketing language on a provider’s website isn’t enough — it changes, and the team that signs off on the deployment needs documented terms they can point to in an audit. A DPA or equivalent contract, countersigned, is the minimum artefact you’ll want on file.

Model Security

Prompt injection defense

Prompt injection is the SQL injection of AI. Users (or attackers) craft inputs that manipulate the model into ignoring its instructions, revealing system prompts, or performing unauthorized actions.

Defenses:

  • Separate system prompts from user input with clear delimiters
  • Validate and sanitize all user inputs before they reach the model
  • Implement output filtering for sensitive information patterns
  • Use allow-lists for actions the model can perform
  • Monitor for anomalous prompt patterns

It’s worth being honest that prompt injection is still an unsolved problem in the academic sense — there’s no cryptographic boundary between instructions and data in a language model the way there is in a traditional parser. The defences above reduce the attack surface but don’t eliminate it. That means architectural decisions matter: if your AI feature can trigger irreversible actions on untrusted input, you have a design problem that no amount of prompt hardening will fully fix.

Output validation

Never trust model output blindly. Treat it like untrusted user input:

  • Sanitize generated HTML/code before rendering
  • Validate data formats (JSON, SQL) before executing
  • Check generated URLs against allow-lists before redirecting
  • Implement content safety filters for user-facing outputs
  • Rate-limit AI-generated actions to prevent automated abuse

Model versioning

  • Pin model versions in production — don’t auto-upgrade
  • Test new model versions against your evaluation suite before deploying
  • Maintain rollback capability to previous model versions
  • Document model behavior changes across versions

Access Controls

Who can access the AI system?

  • Implement role-based access control (RBAC) for AI features
  • Different roles should have access to different data scopes
  • Admin access to prompt configuration should be restricted
  • API keys should be rotated regularly and scoped to minimum permissions

What can the AI do?

Define explicit boundaries:

  • Can it read data? Which data sources?
  • Can it write data? To which systems?
  • Can it trigger actions? Which ones?
  • Can it access external services? Which ones?

Principle of least privilege: The AI should have access to the minimum data and capabilities required for its function. The mistake we see most often is granting the AI broad read access “just in case” — pointing it at the whole database, the whole document store, the whole ticket system — to avoid having to revisit the permissions later. That shortcut turns every prompt injection into a potential data exfiltration event, and it’s the kind of decision that looks fine in a design review and terrible in an incident post-mortem.

Audit logging

Log every AI interaction:

  • Who made the request (user identity)
  • What was sent to the model (prompt content)
  • What the model returned (response content)
  • What actions were taken based on the response
  • Timestamp and session context

These logs are essential for incident investigation, compliance audits, and quality monitoring. Plan the retention policy on these logs carefully. They contain, by definition, the sensitive data you’ve been sending to the model. A verbose logging setup that stores every prompt and response for three years may create a secondary compliance exposure of its own, especially if the logs are held somewhere less defended than the source systems.

Compliance

SOC 2

If your organization is SOC 2 certified, your AI deployment needs to be included in the audit scope:

  • Document AI data flows in your system description
  • Include AI access controls in your control environment
  • Add AI-specific monitoring to your security operations
  • Ensure AI providers have their own SOC 2 certification

GDPR

If you process EU resident data:

  • Conduct a Data Protection Impact Assessment (DPIA) for AI processing
  • Document the legal basis for AI data processing
  • Implement data subject rights (access, deletion, portability) for AI-processed data
  • Ensure cross-border data transfer compliance if using US-based AI providers

Industry-Specific

  • Healthcare (HIPAA): AI processing PHI requires Business Associate Agreements with providers, encryption in transit and at rest, and access logging
  • Finance (SOX, PCI-DSS): AI in financial reporting or payment processing requires additional controls and audit trails
  • Government (FedRAMP): AI deployments in government contexts require FedRAMP-authorized infrastructure

The Enterprise AI Security Checklist

Before Deployment

  • [ ] Data classification completed for all AI data flows
  • [ ] AI provider data processing agreements reviewed by legal
  • [ ] Data retention and training policies confirmed in writing
  • [ ] Prompt injection defenses implemented and tested
  • [ ] Output validation and sanitization in place
  • [ ] RBAC configured for AI features
  • [ ] API keys scoped and stored securely
  • [ ] Audit logging enabled for all AI interactions
  • [ ] Model version pinned and documented
  • [ ] Compliance requirements mapped (SOC 2, GDPR, industry-specific)

At Deployment

  • [ ] Security review sign-off obtained
  • [ ] Monitoring dashboards for anomalous usage active
  • [ ] Incident response plan updated to include AI-specific scenarios
  • [ ] User training on AI usage policies completed
  • [ ] Rate limiting configured to prevent abuse

Ongoing

  • [ ] Monthly review of AI access logs
  • [ ] Quarterly prompt injection testing
  • [ ] Model version updates tested before deployment
  • [ ] Provider compliance certifications verified annually
  • [ ] Data retention audit conducted semi-annually

The Bottom Line

AI security isn’t about preventing AI from being used. It’s about ensuring AI is used safely, with appropriate controls, monitoring, and governance. The goal of the security team shouldn’t be to slow AI adoption, and the goal of the AI team shouldn’t be to work around the security team. Both outcomes produce worse results than genuine collaboration — engineers building with security controls in mind, security reviewing with an understanding of what the product is trying to achieve.

The organizations that deploy AI successfully treat it like any other system that handles sensitive data — with proper access controls, audit trails, and compliance oversight. The ones that skip these steps end up in breach notifications and regulatory investigations.

Security isn’t the enemy of AI innovation. It’s the foundation that makes AI innovation sustainable.