Most healthcare AI pilots fail compliance review not because the AI is dangerous, but because the IT team didn’t know what questions to ask before the vendor signed a BAA and data started flowing.
HIPAA ai compliance in the context of AI tools is genuinely different from traditional SaaS compliance. The rules haven’t changed, but the surface area has expanded dramatically. Inference logs, prompt histories, fine-tuning datasets, embedding stores — none of these existed in HIPAA’s original threat model, and most vendors aren’t thinking about them the way your legal team will.
This guide is for healthcare IT directors who need to move fast on AI adoption while keeping PHI inside the boundaries that regulators, attorneys, and your own security team will accept.
What Counts as PHI When AI Is Involved
The definition hasn’t changed: PHI is individually identifiable health information tied to past, present, or future health status, treatment, or payment. What has changed is how many new places that information ends up when AI enters the workflow.
When a clinician uses an AI assistant to summarize a patient note, the prompt itself may contain PHI. The model’s response may contain PHI. The inference log retained by the vendor almost certainly does. If the vendor uses interaction data to improve their model — even in aggregate — that’s a potential HIPAA exposure that most BAAs don’t address cleanly.
The practical checklist for your team:
- Prompt content: Does user input ever include patient names, MRNs, dates of service, or clinical details?
- Inference logs: Does the vendor retain input/output pairs? For how long? Under what access controls?
- Embeddings and vector stores: If the system uses retrieval-augmented generation against your clinical documents, where are those embeddings stored?
- Fine-tuning datasets: Has the vendor trained on customer data? Could your data have been included?
- Model outputs: Are responses cached or logged in vendor infrastructure?
The answer to most of these questions determines whether you have a technical hipaa ai compliance problem, a contractual one, or both.
What AI Vendor BAAs Actually Say — and What They Should
A BAA from an AI vendor is not the same as a BAA from your EHR hosting provider. The obligations look identical on paper. The technical architecture is completely different.
Most AI vendor BAAs you’ll see in the market do three things well: they acknowledge business associate status, they commit to breach notification timelines, and they prohibit selling PHI. That’s the floor. It’s not enough.
What a BAA for an AI vendor needs to address specifically:
- Training data prohibition: Explicit, unambiguous language that your data — including prompts, completions, and any derived representations — will not be used to train, fine-tune, or evaluate any model, including the vendor’s shared foundation models.
- Inference log retention limits: Specific retention periods with contractual deletion obligations, not just a reference to the vendor’s general data retention policy.
- Sub-processor disclosure: AI vendors almost always route through a foundation model provider (OpenAI, Anthropic, Google, etc.). That sub-processor relationship needs to be disclosed, and the sub-processor’s own obligations need to flow down contractually.
- Audit rights for AI-specific data stores: Your right to audit should extend to vector databases, embedding stores, and inference logs — not just traditional data repositories.
- Model isolation: If the vendor runs shared infrastructure, what technical controls prevent your PHI from being accessible in the context of another customer’s queries?
The gap between “we have a BAA” and “we have a BAA that actually covers AI” is where most healthcare organizations are currently exposed. Vendors know this. The pressure is on the buyer to ask for language that doesn’t appear in their standard template.
The Foundation Model Sub-Processor Problem
This is the issue that kills the most AI deals in healthcare, and it deserves its own section because it’s often invisible until late in the review cycle.
When your staff uses an AI tool that runs on GPT-4 or Claude or Gemini, your vendor is a business associate. The foundation model provider is a sub-processor of that business associate. For hipaa ai compliance to hold, that sub-processor needs to be operating under appropriate contractual obligations — and they need to actually be capable of meeting them technically.
OpenAI has an enterprise tier with a BAA. Anthropic has one. But the standard API terms that many AI vendors build on top of explicitly prohibit sending PHI. If your vendor is using the standard API tier — not the enterprise tier — you have a chain-of-custody problem that no BAA with the vendor can fix downstream.
Ask your vendor directly: Which tier of the foundation model API are you using, and can you provide documentation that PHI is permitted under those terms? The answer will tell you more about their compliance maturity than any questionnaire.
What On-Premises Deployment Actually Changes
The cleanest solution to the sub-processor problem is removing the external dependency entirely. On-premises or private cloud deployment — where the model runs inside the covered entity’s own infrastructure — eliminates the chain of external business associates and puts PHI where it belongs: under your direct control.
This is the architecture Peridot was built around. Rather than routing clinical data through vendor infrastructure and managing a cascade of BAAs across foundation model providers and their sub-processors, Peridot deploys inside your environment. The model runs in your data center or your private cloud tenant. Inference logs, if you choose to retain them, stay in your storage. No PHI leaves the covered entity’s infrastructure perimeter.
The compliance implications are concrete. You don’t need a BAA with a foundation model provider because the model isn’t being called externally. Your audit rights are trivially satisfied because you own the infrastructure being audited. Your security team’s network monitoring covers AI inference the same way it covers any other internal workload.
On-premises AI deployment also changes the breach notification calculus. If a model hosted externally is compromised or has a data exposure, you’re in the notification chain regardless of your vendor’s response timeline. If the model is running inside your network, a security incident is your incident — which sounds worse but is actually better, because you control the response.
For healthcare IT directors evaluating AI tools, the deployment architecture question should come before the feature comparison. The best AI capability in the market is not worth a consent decree. Peridot’s position is that hipaa ai compliance shouldn’t require trusting a vendor’s contractual commitments about infrastructure you can’t see — it should follow from architecture that keeps PHI where your policies already apply.
The market will eventually catch up to what regulated healthcare actually requires. Until it does, the burden is on IT teams to ask harder questions, read BAAs more carefully, and default to architectures that don’t create new regulatory exposure in exchange for marginal convenience.