LLM pipelines look cheap at demo scale. In regulated decisioning, the real cost is compounding error, human review and the mistakes that escape to customers. Here is the maths, for a lender processing 50,000 applications a month.
Chain enough 90% steps together and most of your files come out wrong.
End-to-end accuracy at 90% per step. Benchmarks quoted as pass@1 typically overstate production reality by 20 to 40 points.
About $3.6M a year saved at this volume. Assumes 90% per-step accuracy, 90% detection, $50 per escaped error, 25k-token prompts.
GPU inference changes with batch size, because reduction kernels add numbers in different orders. Batch-invariant kernels fix it and cost you throughput. Determinism can be bought on a large model. It is priced in throughput. Ours ship that way by default, on CPU.
Cost per token is a vendor's metric. Cost per correct, accepted, unreviewed output is yours. The question was always what a wrong answer costs.
The model proposes. The rules dispose.