Slack reports better offline results with fewer tokens
In a customer testimonial published by Anthropic, Slack’s Curtis Allen reports that Sonnet 5.5 outperformed Sonnet 5 on almost all of their offline Slackbot evaluations without prompt changes, using fewer steps and about 14% fewer output tokens.
Supporting evidence
Customer testimonials published by Anthropic for Claude Sonnet 5.5
Original excerpt
“Without changing any of our prompts, Claude Sonnet 5.5 did better than Sonnet 5 on almost all of our offline Slackbot evals, in fewer steps and with about 14% fewer output tokens. When someone gives Slackbot a task, quality and speed are what matter most, and Sonnet 5.5 allows Slackbot to deliver better outcomes for users, faster.”