The cost per document: making an LLM feature profitable.

01 — Where the money went

The first Docsee prototype did the obvious thing: send the whole scanned document to the model with the question and read back the answer. It worked beautifully in the demo. It also cost about $0.12 per document, and most people ask three questions per document, so a free user cost us $1.80 before they had decided whether to pay.

We measured before changing anything. Every call logged tokens in, tokens out, model, latency and the document it belonged to. A week of real usage gave us the table below.

78% of the cost was the document itself, sent again with every question. The question was a rounding error. That told us where to work.

02 — OCR on the device

Sending an image to a multimodal model is the most expensive way to read text. The phone can read text for free. iOS ships an OCR engine that handles most invoices and contracts well enough, and it runs offline in under a second.

So the pipeline became: capture, OCR on device, keep the layout (tables as tables, totals next to their labels), and send text instead of pixels. Cost per document dropped by about 4× before we touched the model at all. Privacy improved as a side effect: the scan never leaves the phone, only the text needed to answer does.

The cheapest token is the one you never send.

03 — Only the question goes to the model

A 40-page lease does not need all 40 pages to answer "when does the notice period end". We index each document once, retrieve the few passages that match the question, and send those. The model sees a page, not a book.

This is where the evaluation set earned its keep. We had 120 real questions with known answers. Every change to retrieval was scored against it, so "cheaper" never silently became "wrong". The pass rate stayed at 94% while the context shrank by 9×.

04 — Cache per document

People ask the same document several questions in a row. The parsed text, the index and the model's working context are all cached per document, so the second and third questions cost a fraction of the first. Repeat questions went from 14% of spend to 3%.

We also route by difficulty. "What is the total" goes to a small, fast model. "Summarise the termination clauses and their conditions" goes to a larger one. The evaluation set decides which questions are which.

05 — What it looks like now

Cost per document: $0.12 → $0.008. Answer quality on the evaluation set: unchanged at 94%. Median latency: 4.1s → 1.6s, because less text in means a faster answer out. The free tier is now a marketing cost we can calculate rather than a leak we hoped would stop.

None of this is specific to Docsee. If you are putting an LLM feature into a product, the same four moves apply: measure per call, move what you can to the edge, send only what the question needs, cache what repeats. We do this for clients under AI integration.