Model choice involves cost and task accuracy
Wade Foster says the leading model on Zapier’s Automation Bench completed about 40% of its automation tasks accurately. He describes cheaper models as costing less while performing reasonably well, and argues that organizations should be able to choose models for different workflows.
Supporting evidence
Original excerpt
It performs about 40% of the tasks accurately, which is the highest that there is. Now, it's more expensive than, say, something like Gemini 3.7, which is, does pretty good but does it at a fraction of the cost.No Code Is Code: Zapier CEO Wade Foster on Headless Tools, Zapier MCP & Automation Bench
Additional context is not included in this excerpt. Read the original conversation.
Start time comes from the supplied transcript. Playback alignment is awaiting review.
