Most of the software your teams use every day is running AI inference in the background — and your IT governance policy almost certainly hasn’t caught up.
The list of apps that use AI has grown faster than most enterprises can track. This isn’t about ChatGPT or Copilot. It’s about the tools already deployed across your organization — CRMs, IDEs, HR platforms, customer support suites — quietly sending data to foundation models and returning results your users treat as gospel. Understanding the mechanism matters as much as knowing the capability.
Productivity and Communication Tools
Gmail Smart Compose uses a sequence-to-sequence LSTM model trained on aggregate email data to predict your next phrase. Users think they’re getting autocomplete. They’re actually getting a probabilistic next-token engine shaped by billions of other people’s emails.
Notion AI routes user queries through OpenAI’s API, meaning every document you ask it to summarize leaves your Notion workspace and hits an external endpoint. Most users have no idea their content is in transit. Grammarly does the same — it sends full document context to cloud-based NLP models for tone and clarity scoring, not just local spell-check. Microsoft 365 Copilot integrates GPT-4 with your tenant’s Graph API, meaning it can see email, calendar, Teams messages, and SharePoint simultaneously when responding to a prompt. Slack now surfaces AI-generated channel summaries using their own fine-tuned model, though the training data provenance is not publicly disclosed. Zoom IQ transcribes and summarizes meetings using Zoom’s in-house ASR pipeline combined with an LLM summarization layer — audio leaves the call before the meeting ends.
Developer and Engineering Platforms
GitHub Copilot is the most widely deployed app that uses AI in enterprise engineering — it sends your open file, surrounding context, and recent edits to a Codex-derived model hosted by OpenAI on Microsoft’s infrastructure. The code you’re writing is the training signal for the prompt. JetBrains AI Assistant offers similar functionality with a choice of backend model, which is a meaningful architectural difference for compliance teams. Tabnine offers on-premise deployment, making it one of the few coding assistants that doesn’t route completion requests to an external API.
Datadog now uses anomaly detection models trained on your own time-series telemetry — this is local inference on customer data, which is architecturally different from the productivity tools above. PagerDuty‘s AIOps layer clusters similar incidents using embedding similarity, meaning it’s building a vector representation of your incidents and querying against it at alert time. Snyk uses ML to prioritize CVEs based on reachability analysis — it’s not just pattern matching, it’s context-aware static analysis powered by graph neural networks.
CRM, Sales, and Marketing Platforms
This category contains some of the most aggressive AI deployments in enterprise software, often with the least transparency. Salesforce Einstein runs predictive lead scoring using gradient boosted trees trained on your CRM data — but the model training happens in Salesforce’s cloud, not yours. HubSpot’s AI tools generate email copy and social content via OpenAI API calls, with your contact data and deal context included in the prompt payload. Gong transcribes every sales call, extracts topics and sentiment using a custom NLP pipeline, and stores structured representations of your customer conversations on Gong’s infrastructure indefinitely unless you explicitly configure retention limits.
Outreach uses a sequence model to recommend optimal send times and message variants — the model is trained on aggregate platform data across all customers, not just yours. Drift (now Salesloft) routes conversational AI responses through a combination of intent classification and retrieval-augmented generation, meaning it’s pulling from a knowledge base at inference time. Marketo Engage uses predictive content AI to serve dynamic content blocks — the personalization engine is scoring each visitor against a behavior model that updates in near real time.
Enterprise Infrastructure and the Control Problem
The apps listed above represent one architectural pattern: AI as a feature bolted onto an existing SaaS product, with inference running in the vendor’s environment. The data governance implications are significant. You don’t control the model version, the prompt construction, the data retention, or the logging behavior. When something goes wrong — a hallucinated contract clause, a leaked customer detail in a summary, a compliance violation surfaced in discovery — you’re debugging a black box you don’t own.
The other pattern is infrastructure-first AI: deploying models inside your own environment and building or integrating applications on top. This is where Peridot operates. Rather than connecting your data to a vendor’s AI layer, Peridot runs the AI control plane inside your infrastructure, so inference, access control, and audit logging stay within your perimeter. It’s the difference between using apps that use AI and owning the layer those apps run on.
Other platforms in this category include Weights & Biases for experiment tracking, Ray Serve for distributed model serving, BentoML for model packaging, and vLLM for high-throughput LLM inference. These are infrastructure components, not end-user applications — but they’re what serious enterprise AI deployments are built on. Peridot sits above this layer, adding governance, access policy, and observability that individual serving frameworks don’t provide.
The remaining categories — HR tech, legal tech, healthcare, finance, and customer support — follow the same two patterns. Workday uses ML for attrition prediction and skills matching, with model training running in Workday’s cloud. Eightfold AI builds talent intelligence graphs using transformer-based models fine-tuned on resume and job description corpora. Harvey runs legal research and drafting through GPT-4 with retrieval augmentation — powerful, but the data boundary question for law firms is severe.
The uncomfortable reality for IT directors is that the list of apps that use AI inside your organization is probably three times longer than your approved software registry. Shadow AI isn’t a future risk — it’s a current inventory problem. Every SaaS renewal conversation should now include a data flow audit for AI features, because the vendors are shipping them faster than your procurement process can evaluate them.
The organizations that come out ahead won’t be the ones that blocked all AI — they’ll be the ones that established a control layer before the sprawl became unmanageable.