Private LLM API for Document Processing & Extraction
Run contracts, medical records, and invoices through an OpenAI-compatible LLM API with attested TEE privacy, ZDR routes, and no prompt or output logs.
You do not need on-prem inference to keep your documents private.
Teams building pipelines over contracts, medical records, and invoices hit the same wall: the models are capable, but sending regulated documents to an opaque cloud endpoint stalls at security review. TrustedRouter is a document extraction API for LLM workloads with privacy you can check. Every prompt runs through an open-source, hardware-attested confidential-compute gateway. TrustedRouter never logs prompt or output content. Ordinary synchronous and streaming inference does not retain it. The opt-in Batch API temporarily retains enclave-encrypted artifacts for up to 30 days. Fetch the attestation yourself; the image digest matches the published build.
The API is OpenAI-compatible. Change base_url to https://api.trustedrouter.com/v1, keep your SDK, your model ids, and your extraction prompts. Structured data extraction, classification, and summarization over parsed PDFs run on the same code you have today, across 220+ model routes from 30+ providers, with automatic fallback when a provider goes down.
from openai import OpenAI
client = OpenAI(base_url="https://api.trustedrouter.com/v1",
api_key=key)
resp = client.chat.completions.create(
model="trustedrouter/auto",
messages=[
{"role": "system",
"content": "Extract vendor, date, line_items, total as JSON."},
{"role": "user", "content": invoice_text},
],
)
print(resp.choices[0].message.content)Attestation you can fetch
The prompt path runs inside a TEE. One curl to the attestation endpoint returns a JWT signed by the CPU vendor's root key, bound to your live TLS session so it cannot be replayed, with an image digest matching the published open-source gateway. Attestation beats a privacy policy.
Open, ZDR, or TEE per route
Choose the tier each pipeline needs: Open routes on attested infrastructure, Zero-Data-Retention routes where providers contractually keep nothing, or TEE routes where even the serving provider cannot read the document. That is a private document AI setup without racking GPUs. See zero-data-retention routes.
What your reviewers will ask for
Verifiable architecture plus readiness documentation your compliance team can assess: SOC 2 readiness, HIPAA readiness, a DPA, a BAA, and a current subprocessor list. Healthcare document pipelines can start at the HIPAA LLM API page.
Bring your parser. Swap the inference underneath.
TrustedRouter serves inference, and only inference. Keep Unstract, LlamaParse, or your own OCR and chunking stack, and send the parsed text through the API for LLM PDF data extraction. Routing, per-model pricing, and latency measured from real routed requests are public, so you can pick routes for a batch job with data rather than guesswork, or let trustedrouter/auto pick per request.
One anonymized data point: a document-processing company scaled from zero to over a billion tokens per week through the attested gateway within three weeks of onboarding. No architecture change, no on-prem buildout. Privacy you can check beats privacy you're promised.
Current routes, prices, privacy, and measured performance.
Catalog facts come from the routes currently configured in TrustedRouter. Performance uses the same cached metadata snapshot as the public leaderboard. Prompts and outputs are not part of these measurements.
| Model | Providers | Context | Input | Output | Privacy | Measured route |
|---|---|---|---|---|---|---|
Google: Gemini 3.1 Pro Previewgoogle/gemini-3.1-pro-preview |
+1
|
1,048,576 | $2.11/1M | $12.66/1M | ZDR 3 cited scores | 4801 ms TTFT google-ai-studio · 100.00% available · n=351 |
Google: Gemini 2.5 Flashgoogle/gemini-2.5-flash |
+1
|
1,048,576 | $0.3165/1M | $2.6375/1M | ZDR | 1707 ms TTFT google-ai-studio · 100.00% available · n=136 |
Google: Gemini 3 Flash Previewgoogle/gemini-3-flash-preview |
|
1,048,576 | $0.5275/1M | $3.165/1M | ZDR | 2066 ms TTFT google-ai-studio · 100.00% available · n=8 |
Google: Gemini 3.5 Flashgoogle/gemini-3.5-flash |
+1
|
1,048,576 | $1.5825/1M | $9.495/1M | ZDR | 1278 ms TTFT google-vertex · 107 tok/s · 100.00% available · n=7 |
Google: Gemini 3.1 Flash Litegoogle/gemini-3.1-flash-lite |
|
1,048,576 | $0.26375/1M | $1.5825/1M | ZDR | 890 ms TTFT google-ai-studio · 100.00% available · n=7 |
Gemini 2.5 Flash Litegoogle/gemini-2.5-flash-lite-preview-06-17 |
|
1,048,576 | $0.1055/1M | $0.422/1M | varies | 2137 ms TTFT lightning · 100.00% available · n=5 |
Browse every modelReview provider policiesOpen the full leaderboardSnapshot 2026-09-17T13:22:21.752Z
Questions
Can I use TrustedRouter for HIPAA-regulated documents?
TrustedRouter does not claim a HIPAA certification, and its regulatory pages are informational rather than legal advice. What it offers is verifiable architecture plus readiness documentation your compliance team can assess: a HIPAA readiness page, a BAA, a DPA, and a subprocessor list. The prompt path runs in a hardware-attested TEE and the attestation is checkable live. TrustedRouter never logs prompt or output content. Ordinary synchronous and streaming inference does not retain it. The opt-in Batch API temporarily retains enclave-encrypted artifacts for up to 30 days. Batch is excluded from PHI use unless expressly approved in a signed BAA amendment.
Does TrustedRouter parse PDFs or run OCR?
No. TrustedRouter serves inference only. Keep your existing OCR or PDF parsing stack, whether that is Unstract, LlamaParse, or your own pipeline, and send the extracted text through the API for structured data extraction. The API is OpenAI-compatible, so most document pipelines migrate by changing base_url while keeping the same SDK, model ids, and prompts.
How do I verify that my documents stay private?
Fetch a live attestation. A curl to the attestation endpoint with a nonce returns a JWT signed by the CPU vendor's root key, and the image digest in it matches the published open-source gateway build. The attestation binds to the live TLS session, so it cannot be replayed. TrustedRouter never logs prompt or output content. Ordinary synchronous and streaming inference does not retain it. The opt-in Batch API temporarily retains enclave-encrypted artifacts for up to 30 days. Zero-Data-Retention and TEE tiers are available per route.
Which models can I use for document extraction?
There are 220+ model routes across 30+ providers, including Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Llama, and Mistral. Chinese open-weight models are served by non-Chinese hosting providers on attested infrastructure, so prompts do not go to the model vendor. Meta-routes help batch pipelines: trustedrouter/auto picks a best-fit route per request, trustedrouter/cheap picks the cheapest capable route in a TEE, and automatic fallback covers provider outages.