nafyi.

OPINIONS EXPRIMÉES EN PUBLIC

Zuzanna Stamirowska

Source participant

1 sources · 3 points de vue · 3 sujets

Contenu mis à jour:

Zuzanna Stamirowska sur latent reasoning benefits, post-transformer architecture motivation, reasoning modality. Explorez 3 points de vue par thème, avec des éléments tirés de 1 source.

Explorer les liens

Points de vue par sujet

Points de vue attribués, classés par date de publication de la source. Un aperçu de ces échanges, sans prétendre définir toutes les convictions de la personne.

Les traductions sont destinées à la lecture ; les extraits originaux restent la source evidence.

post-transformer architecture motivation

Voir ce sujet
post-transformer architecture motivation

RAG, context, and memory were bottlenecks in 2023–2024

Stamirowska says the team saw RAG, context, and memory as bottlenecks in 2023 and 2024. They were not efficient or sufficient for what she describes as 'AI with no ceiling, if you wish.'

Éléments favorables

Zuzanna Stamirowska·

What Comes After Transformers?

Extrait original

But very concretely, what prompted us to start Pathway and to start working on post-transformer architectures was that we were looking forward to reasoning becoming a thing. We have a very strong background in dealing with the dimension of time in machine learning—any sort of online learning. And we saw RAG, context, and memory, and we’re talking about 2023 and 2024, as bottlenecks. They were not efficient or sufficient to unlock AI as we would like—AI with no ceiling, if you wish.
Ouvrir l’original · Publisher transcript paragraph 7 (s’ouvre dans un nouvel onglet)Source du texte original (s’ouvre dans un nouvel onglet)
Zuzanna Stamirowska
Partager un aperçu

reasoning modality

Voir ce sujet
reasoning modality

Stamirowska says reasoning shouldn't just be linguistic

Stamirowska says, 'We realized that reasoning shouldn't just be linguistic.' She says verbalizing things in chain-of-thought is not optimal: it involves 'gluing it on top of a transformer,' and transformers are naturally ill-suited to dealing with time and learning from experience.

Éléments favorables

Zuzanna Stamirowska·

What Comes After Transformers?

Extrait original

Specifically, we’re talking about continual learning, memory, and reasoning first. And as we moved through it, we realized that reasoning shouldn’t just be linguistic. Verbalizing things in chain of thought is not the optimal way to do reasoning. With chain of thought, we’re essentially gluing it on top of a transformer, and transformers are naturally ill-suited to dealing with time and learning from experience.
Ouvrir l’original · Publisher transcript paragraph 8 (s’ouvre dans un nouvel onglet)Source du texte original (s’ouvre dans un nouvel onglet)
Zuzanna Stamirowska
Partager un aperçu

latent reasoning benefits

Voir ce sujet
latent reasoning benefits

Latent thinking can help solve constraint-satisfaction and math problems

Stamirowska says latent thinking can help solve constraint-satisfaction problems and math with 'tiny, tiny models.'

Éléments favorables

Zuzanna Stamirowska·

What Comes After Transformers?

Extrait original

Latent thinking has huge benefits. It can help us solve constraint-satisfaction problems natively, like what we’re showing. We’re solving constraint-satisfaction problems and math with tiny, tiny models. And then there’s cost efficiency. There are just insane cost gains coming from latent reasoning.
Ouvrir l’original · Publisher transcript paragraph 37 (s’ouvre dans un nouvel onglet)Source du texte original (s’ouvre dans un nouvel onglet)
Zuzanna Stamirowska
Partager un aperçu

Propos par date de source1

Les propos sont classés par date de publication de la source originale ; une différence de formulation ne prouve pas un changement de position.