RAG, context, and memory were bottlenecks in 2023–2024
Stamirowska says the team saw RAG, context, and memory as bottlenecks in 2023 and 2024. They were not efficient or sufficient for what she describes as 'AI with no ceiling, if you wish.'
Supporting evidence
What Comes After Transformers?
Original excerpt
But very concretely, what prompted us to start Pathway and to start working on post-transformer architectures was that we were looking forward to reasoning becoming a thing. We have a very strong background in dealing with the dimension of time in machine learning—any sort of online learning. And we saw RAG, context, and memory, and we’re talking about 2023 and 2024, as bottlenecks. They were not efficient or sufficient to unlock AI as we would like—AI with no ceiling, if you wish.
Share insightCheck this claim