The Data Exchange ·
Richard Ho describes OpenAI’s chip work as an effort to lower inference and compute costs. He explains the memory-compute locality built into the architecture and how one device can operate along the Pareto curve between high throughput and low latency. Lee 3 puntos de vista con sus evidencias y enlaces a las fuentes.
The mission was to lower inference and compute costs—making intelligence faster and cheaper for users.
Ver el momento de apoyo · Publisher transcript paragraph 10The only key hardwired architectural feature is tight memory-compute locality—specifically, affinity between HBM and compute cores—to improve performance, power efficiency, and reduce data movement.
Ver el momento de apoyo · Publisher transcript paragraph 31Ho says a single device can operate at either end of the Pareto curve between high throughput and low latency, or at points between them. He connects this capability to lowering infrastructure costs.
Ver el momento de apoyo · Publisher transcript paragraph 43Fragmentos breves y atribuidos, con el contexto necesario para verificarlos. La conversación completa permanece con su editor.
Extracto original
They wanted to look really hard at what we could do to lower the cost of inference and lower the cost of compute.
I think this project really kicked off when Sam and Greg decided that infrastructure was going to be a key differentiator and one of the key drivers of AI and intelligence to users.
Extracto original
The one thing that is really unique about our architecture, that is kind of hardwired in, is the locality and affinity between memory and compute. That’s the key architectural difference of this device.
Extracto original
What we have in a single device—and I think this is really critical for our infrastructure and being able to lower the cost of the infrastructure—is a single device that can operate at both points, or at any point along that curve. We call it the Pareto curve between high throughput and low latency.
Estas perspectivas enlazan a sus fuentes originales. Las paráfrasis están identificadas y no son citas textuales.
Abrir transcripción o material de origen (se abre en una pestaña nueva)Reportar un problema