A layperson's guide to 'private' AI | internet Research Lab

The use of large language models (LLMs) carries a myriad of privacy risks for users – ranging from simple retention and disclosure of personal information by LLM service providers, to more sophisticated leaks through model and inference attacks. In response to the former class of risks, “private AI” has emerged as a term referring to services that offer LLMs to users with some privacy guarantees, professedly beyond what popular LLM providers offer. This post breaks down the concrete security and privacy guarantees that different LLMs being marketed as such provide to users.

While there are privacy losses at the stage of training AI systems – notably through indiscriminate scraping of the web – the focus of this write-up is privacy guarantees at the inference stage, i.e. when a user submits a query to a LLM. These privacy promises include zero-data retention, unlinkability of queries to a particular user, encryption of conversations at rest, and guarantees being developed at the hardware-level. This post discusses the spectrum of these promises, and analyses which of these are enforced through server policy and which are through (somewhat) verifiable technical means.

Some AI models can be self-hosted, which remains the most private option for using LLMs, but this is not practical for most users. Note that modern LLMs also offer tools that add capabilities to use external resources (e.g., web search). For simplicity, this post assumes a user query that only interacts with a remote LLM server and employs no external tool.

The baseline: when data retention is the default

The privacy baseline for the most popular LLM service providers – ChatGPT and Claude, for instance – is the wholesale storage of user queries on their servers and subsequent use for multiple purposes. Claude and ChatGPT will, by default, retain conversations and use them to train their models. On the free and personal subscription tiers, a user needs to explicitly opt out of such use. Even after a user requests a conversation to be deleted, the scrubbing can take up to 30 days.

Each conversation is linked to a user’s account, and the entire history of all queries and responses stays available to the companies. This data is subject to the usual and very real privacy risks: intentional sharing with third parties, bad security practices leading to database leaks, and legal disclosure.

Encrypted at rest and in transit, plaintext at inference

Proton’s Lumo is an example that makes several promises: it keeps no logs, does not use the data for training, and implements “zero-access encryption.”

Zero-access encryption – a feature common across Proton services – is the guarantee that Proton cannot read the data that a user stores on the servers. Proton encrypts each user’s data with keys that can only be unlocked with the user’s password. Information is encrypted at rest on Proton’s server, and it is only decrypted on a user’s device after successful authentication. In the context of Lumo, it is important to note that this security guarantee only applies to conversation history after the LLM has processed the user query.

The actual processing of the query needs a separate treatment: the LLM has to decrypt a user’s query to see and answer it. In Proton Lumo’s case, user queries are encrypted throughout the flow – even Proton’s intermediate servers can’t read them. The query is in plaintext at the very end, i.e. the LLM server. Proton promises that the “LLM server forgets the request as soon as the response is generated.” Despite its utility, note that this guarantee is enforced through server policy rather than via verifiable technical or cryptographic means.

Privacy partitioning or “unlinkability”

Privacy partitioning or ‘decoupling’ is becoming a common design pattern in newer internet protocols that enables users to interact with a service without revealing some user-identifying information to the server. In its simplest form, an intermediary server is added between the user and the end-server. This intermediary forwards the request to the end-server on behalf of the user, so that the end-server does not have network traffic metadata associated with the user. The intermediary server only ends up seeing the user information, and the end-server only sees the request, with neither party being able to link the two – as long as they don’t collude.

For LLMs, a good example is Brave Leo’s offering when a user picks the open-weight GLM 5.3 Flash model, which the post describes in more detail below. Pertinently here, Brave acts as a relay between the user and the LLM. Brave’s relay receives the user’s request and can see the user’s IP address, but not the actual query. NEAR AI, the model’s host, never learns who sent it. Leo’s other models don’t work this way: Brave both relays and hosts them, so their privacy rests on Brave’s no-logging policy rather than on any trust split between parties.

Duck.ai uses a weaker, single-proxy version for the models it offers from OpenAI, Anthropic and Mistral. It strips a user’s IP address before forwarding the prompt. But DuckDuckGo itself sees both the IP and the prompt, and indeed does perform a limited range of analytics over plaintext user queries when they are in memory. Duck.ai has negotiated special agreements with these LLM providers, who in turn have provided an assurance that they will delete user prompts after generating a response, and not use them for training purposes.

Even if the conversations were intentionally or unwittingly retained by the LLM servers, one can be reasonably certain – if true privacy partitioning is implemented – that they will not be linked to a particular user, at least not without significant effort. Of course, this would be rendered useless if the user’s prompt or query itself includes personally-identifiable information.

Thankfully, usability need not take a huge hit: conversations are stored locally on the user’s device and can be accessed later. Moreover, Duck.ai combines this proxying with what they call secure Sync & Backup, which, similar to Lumo, stores encrypted conversation logs on DuckDuckGo’s servers that are only decrypted on user devices.

Hardware at play: confidential computing

Confidential computing is a class of technology that provides security and privacy through hardware design. A key innovation in the field is a trusted execution environment (TEE) – usually a virtual machine that has two key properties: first, its running memory is cryptographically isolated from its host machine, and second, its software cannot be modified in operation by other parties. In a TEE, the host machine cannot read what’s inside the virtual machine. An advance by Nvidia in 2023 extended confidential computing to GPUs, enabling its use in the context of LLMs. In this context, confidential computing usually means the model itself runs inside a TEE on the LLM server.

When it comes to previous case studies mentioned here (Lumo and some of Duck.ai’s offerings), the plaintext of the queries was available to the LLM servers. Users have to take companies at their word when they claim that they immediately delete the user query and the LLM response. LLM inference servers running in a securely-implemented TEE offer a technical guarantee that the host service itself does not have access to the user query and the LLM response.

Examples of this approach include Confer and Apple’s Private Cloud Compute (PCC) that launched in 2024. Apple uses PCC for Apple Intelligence requests that may be too computationally-intensive to run locally. Brave’s AI assistant, Leo, has run an open-weight model inside NEAR AI’s TEEs since late 2025. Duck.ai offers also offers some models by Tinfoil, which runs them inside TEEs. Since 2025, Meta has offered something similar with what it calls Private Processing, which is deployed selectively across products and not for all regular Meta AI conversations.

For (the very justified) haters of Meta Platforms and fans of meta questions, let’s extend our scrutiny of this trust chain further: how do we know that these servers claiming to run TEEs for LLM queries are actually doing so?

With remote attestation, users can ask the server to prove what it’s running. When the server boots, the chip records a fingerprint of every piece of software it loads. The user receives this report, signed by the chip with a key set at the time of manufacturing of the chip, along with a fresh encryption key generated inside the TEE. A user’s device checks that the signature traces back to a genuine chip from the manufacturer (say AMD, Intel or Nvidia); that the fingerprint matches a publicly published build; and that the encryption key came from inside the TEE. Only then does it send the prompt, encrypted so that only that TEE can read it. Overall, remote attestation verifiably tells someone what code they’re talking to.

Confer and Apple’s PCC provide remote attestation, so a user’s device runs this verification automatically when interacting with the LLM server. Apple publishes its software images; Confer goes further and publishes source code that anyone can rebuild into the exact same images. As Apple summarises, “user devices will be willing to send data only to PCC nodes that can cryptographically attest to running publicly listed software.”

These guarantees aren’t perfect: confidential computing for AI is new and tricky to implement. For instance, Trail of Bits found 28 issues with Meta’s Private Processing in its security audit, and revealed some vulnerabilities that could have let an attacker bypass its privacy guarantees entirely. Most critical issues were patched before Private Processing launched. TEEs are constantly improving and are now launched quite stably at a global scale, but a healthy skepticism around secure implementations is warranted.

Stacking the guarantees

LLM servers using TEEs provide undeniably significant privacy gains over the baseline that this post initially discussed. Privacy-enhancing techniques outlined here can be combined to even greater effect. Indeed, Brave Leo’s GLM option, Meta’s Private Processing and Apple’s PCC combine their use of TEEs with privacy partitioning. With many open-weight models now capable enough to perform most tasks for an average user, privacy-respecting use of modern LLMs through a diversity of service providers is a reality in grasp.

The fact that data-hungry tech giants have invested in these approaches is also a strong signal that privacy-respecting design, while still often an afterthought, holds notable weight as a market factor.

Needless to say, compromises in privacy are not the only danger posed by the increasing deployment of LLMs… by a mile, a mine, and a water tank. The deployment of privacy-respecting technologies in the space goes a tiny but welcome way in the recognition of the sober reality that internet users are relying on LLMs for sensitive work and personal advice – from legal counsel to psychological help. LLMs “invite confession”, and our confessions deserve protection.

This blogpost was written by Gurshabad Grover.