CONNECTED THINKING
Knowledge atlas
Follow people, viewpoints and their original evidence.
1 people · 1 sources · 1 viewpoints
IN CONTEXT
post-transformer architecture motivation
Choose a viewpoint. Follow it back to the conversation.
RAG, context, and memory were bottlenecks in 2023–2024
Equal sectors are reading positions, not rankings.
Showing 1–1 of 1 viewpoints · Newest sources first
1 / 1
Selected viewpoint
RAG, context, and memory were bottlenecks in 2023–2024
Stamirowska says the team saw RAG, context, and memory as bottlenecks in 2023 and 2024. They were not efficient or sufficient for what she describes as 'AI with no ceiling, if you wish.'
These are individual perspectives, not a measure of consensus. Source material stays in its original language.
Supporting evidence
What Comes After Transformers?
Original excerpt
But very concretely, what prompted us to start Pathway and to start working on post-transformer architectures was that we were looking forward to reasoning becoming a thing. We have a very strong background in dealing with the dimension of time in machine learning—any sort of online learning. And we saw RAG, context, and memory, and we’re talking about 2023 and 2024, as bottlenecks. They were not efficient or sufficient to unlock AI as we would like—AI with no ceiling, if you wish.
Publication dates describe the sources, not changes in belief.