nafyi.

A TOPIC, IN CONTEXT

KernelBench GPU kernel evaluation

Judgments in this source concerning KernelBench GPU kernel evaluation. Explore 1 viewpoint with evidence from 1 source.

1 people · 1 sources · 1 viewpoints

Content updated:

Explore connections ↗

Viewpoint map

Explore by person. Select two or three to compare.

1 people · 1 sources · 1 viewpoints

Lilian Weng

KernelBench measures correctness and speed of generated GPU kernels

KernelBench evaluates LLMs on 250 PyTorch tasks to assess their ability to generate correct and fast GPU kernels, using fast_p—the percentage of generated kernels that are both correct and faster than the baseline—as its metric.

Harness Engineering for Self-Improvement

Supporting evidence

Lilian Weng·

Harness Engineering for Self-Improvement

Original excerpt

KernelBench : evaluate correctness and speed for generated GPU kernels. 250 PyTorch tasks to evaluate whether LLM can write fast and correct kernels.
Context

The evaluation metric fast_p = the percentage of generated kernels that are correct and faster than baseline.

Open original · Paragraph 143 (opens in a new tab)Original text source (opens in a new tab)
Share insightCheck this claim

These are individual perspectives, not a measure of consensus. Source material stays in its original language.

Original conversations1

ARTICLE

Harness Engineering for Self-Improvement

Lil'Log