Frontier LLMs treated as 'god' are expensive and slow
Frontier LLMs are increasingly treated like a 'god'—used for any fuzzy-input task—but this generality incurs high cost and latency on every call, even when only a binary decision is needed.
Supporting evidence
Building Production Agents with Jev and LangGraph
Original excerpt
As frontier LLMs have gotten more capable, we've started treating them like god, reaching for them for any task with fuzzy inputs: prose generation, document extraction, search, ranking, research, classification, and more. Unfortunately, god is expensive, and slow. You pay for that generality on every call, even when all you needed was a yes or no.