Handling PII and Sensitive Data

Prompts leave your infrastructure, logs persist, and models memorize. Three exposure paths and what to do about each.

Sending a support ticket to a model API means sending whatever that ticket contains — names, addresses, account numbers, medical details — to a third party.

Three distinct exposure paths, each needing a different response.

1 · Data in transit to the provider

Prompts leave your infrastructure. What happens next is a contractual question, not a technical one.

Retention. How long are requests stored? Most providers retain briefly for abuse monitoring; some offer zero-retention terms for eligible accounts.

Training use. Whether inputs may be used to train future models. Business and enterprise tiers commonly exclude this; consumer tiers frequently do not. This is the single most important term to check, and it differs between tiers of the same provider.

Sub-processors and residency. Who else touches the data, and in which jurisdictions. Determines whether you can use the provider at all under some regulations.

Human review. Whether staff may inspect flagged content.

Read the actual data processing terms for the tier you are on. Do not infer them from the provider’s general reputation, and do not assume the terms on a paid plan apply to a free one.

Where this cannot be satisfied, self-hosting an open-weight model is the alternative — data never leaves your infrastructure, at the cost of operating it.

2 · Data in your own logs

Frequently the larger exposure, and consistently underestimated.

Good observability means logging full prompts, which means your logs now contain every piece of sensitive data your users sent. Those logs are replicated, backed up, retained, and readable by whoever has log access.

For many systems this creates a bigger compliance surface than the API call itself, because logs persist far longer and are accessible to far more people.

Practical measures: redact before writing rather than after, set retention that matches your policy rather than your default, restrict log access as you would restrict database access, and log identifiers where you can instead of content. Sampling full prompts at a low rate while logging metadata for everything is a reasonable middle path.

3 · Memorization and leakage

Two separate concerns often conflated.

Training memorization. Models can reproduce verbatim sequences from training data, particularly rare strings repeated across documents — which is what an account number or API key looks like. This is why training on unredacted production data is risky in a way that is hard to undo.

Context leakage. More immediate: data in one request appearing in a response where it does not belong. If you put another customer’s record in context, the model may surface it. This is not a model defect — it is a retrieval and access-control defect.

The fix for the second is permission-filtered retrieval: filter by the requesting user’s permissions at query time, every time. A RAG system indexing documents with mixed access levels and no per-user filtering becomes a way to read things people cannot read directly. This is a common and serious mistake, and it is entirely preventable.

Reducing what you send

Redact before the call. Replace names, emails, and account numbers with placeholders, then restore them in the response if needed. Works well when the task does not depend on the actual values — classification, summarization, sentiment.

The limitation is that detection is imperfect. Structured identifiers are catchable by pattern; names and contextual identifiers are not, and free text defeats redaction regularly.

Send less. Often the task needs a field, not a record. Extract what is required and send only that.

Route by sensitivity. Non-sensitive traffic to a hosted API, sensitive traffic to a self-hosted model or a human. More engineering, and it resolves the tension cleanly.

Do not send it at all. Some data should not reach a third-party API regardless of terms. Decide this deliberately, and write it down where engineers will see it.

User-facing obligations

Disclose that a third-party model processes their data. Required in some jurisdictions, and reasonable everywhere.

Support deletion. If a user requests erasure, that must cover stored memories, logs, and retrieved indexes — not just your primary database. Design for this before you need it; retrofitting deletion across derived stores is painful.

Be careful with training on user data. Using customer content to fine-tune requires a legal basis and, usually, consent. A model trained on it cannot be unlearned selectively.

What to remember

• Three paths: transit to the provider, your own logs, and memorization or context leakage.
• Provider terms differ by tier — check retention, training use, residency, and human review for the tier you actually use.
• Logs are often the larger exposure: redact before writing, bound retention, restrict access.
• Filter retrieval by the requesting user’s permissions, or your RAG system becomes an access-control bypass.
• Redaction is imperfect on free text; sending less is more reliable than redacting more.
• Deletion requests must reach memories, logs, and indexes — design for it early.

Next: Red-Teaming AI Systems