The Data Exchange ·
Richard Ho describes OpenAI’s chip work as an effort to lower inference and compute costs. He explains the memory-compute locality built into the architecture and how one device can operate along the Pareto curve between high throughput and low latency. Lisez 3 points de vue avec leurs éléments à l’appui et les liens vers les sources.
The mission was to lower inference and compute costs—making intelligence faster and cheaper for users.
Lire le moment probant · Publisher transcript paragraph 10The only key hardwired architectural feature is tight memory-compute locality—specifically, affinity between HBM and compute cores—to improve performance, power efficiency, and reduce data movement.
Lire le moment probant · Publisher transcript paragraph 31Ho says a single device can operate at either end of the Pareto curve between high throughput and low latency, or at points between them. He connects this capability to lowering infrastructure costs.
Lire le moment probant · Publisher transcript paragraph 43Extraits courts et attribués, avec le contexte permettant de les vérifier. La conversation complète reste chez son éditeur.
Extrait original
They wanted to look really hard at what we could do to lower the cost of inference and lower the cost of compute.
I think this project really kicked off when Sam and Greg decided that infrastructure was going to be a key differentiator and one of the key drivers of AI and intelligence to users.
Extrait original
The one thing that is really unique about our architecture, that is kind of hardwired in, is the locality and affinity between memory and compute. That’s the key architectural difference of this device.
Extrait original
What we have in a single device—and I think this is really critical for our infrastructure and being able to lower the cost of the infrastructure—is a single device that can operate at both points, or at any point along that curve. We call it the Pareto curve between high throughput and low latency.
Ces points de vue renvoient à leurs sources originales. Les reformulations sont signalées et ne sont pas des citations mot à mot.
Ouvrir la transcription ou les documents sources (s’ouvre dans un nouvel onglet)Signaler un problème