Stamirowska says reasoning shouldn't just be linguistic
Stamirowska says, 'We realized that reasoning shouldn't just be linguistic.' She says verbalizing things in chain-of-thought is not optimal: it involves 'gluing it on top of a transformer,' and transformers are naturally ill-suited to dealing with time and learning from experience.
Supporting evidence
What Comes After Transformers?
Original excerpt
Specifically, we’re talking about continual learning, memory, and reasoning first. And as we moved through it, we realized that reasoning shouldn’t just be linguistic. Verbalizing things in chain of thought is not the optimal way to do reasoning. With chain of thought, we’re essentially gluing it on top of a transformer, and transformers are naturally ill-suited to dealing with time and learning from experience.
Share insightCheck this claim