Token exhaustion in a pelican-SVG test
In Willison’s pelican-SVG test, Sonnet 5.5 at “max” thinking effort used 128,000 tokens, costing $1.28, before running out of tokens without producing an SVG.
Supporting evidence
Original excerpt
the "max" thinking effort pelican thought for 128,000 tokens (at a cost of $1.28) before running out of tokens and failing to produce an SVG.Claude Sonnet 5.5
Context
Here are some pelicans riding bicycles . Sonnet 5.5 suffered from the same bug as Opus 5.5 :
Text location from the supplied original; confirm wording and attribution in context.
Share insightCheck this claim