Hugging Face ships 207 WebGPU kernels and a 2.57x speed claim

Hugging Face has put 207 WebGPU kernels on its Hub as individually versioned packages — each one its own repository holding a manifest, WGSL shader templates, correctness tests, benchmarks and a runnable example — and shipped a JavaScript loader, @huggingface/kernels, that pulls a kernel into an app by repository ID and contract version. On an Apple M4 GPU the company measured them against ORT WebGPU and reported a 2.57x geometric-mean speedup across 809 comparable cases. The speed number will get the attention. The more consequential thing here is that a GPU shader now has a package name, a version and a test suite.

A model running in a browser eventually reduces to a sequence of GPU operations: matrix multiplications, normalizations, convolutions, attention primitives, quantization ops, layout transforms. WebGPU exposes those through a portable API in modern browsers, and WGSL is the common language for the shaders that execute. Portability, though, buys correctness, not speed. Two shaders can compute the same operation and return identical results while behaving completely differently on different accelerators, because performance depends on workgroup sizes, memory access patterns, vectorization, data types and how operations are fused. The best choice also shifts with the shape of the input, the device, the browser and which WebGPU features are available.

That is the argument for treating kernels as the base layer of browser inference. A high-level runtime can only be as efficient as the operations it calls. Break those operations out into separate artifacts that can be searched, tested, benchmarked and versioned, and the bottom of the stack can improve on its own schedule while the contract above it stays fixed.

Every kernel gets a card describing what the operation means, its inputs, outputs, attributes, supported data types and source files, plus an example ready to run. Take ai.onnx.Add, elementwise addition with multidirectional broadcasting — about as plain as a neural network operation gets, used in residual connections and in bias addition. Its card lists two inputs, the result shape after broadcasting, the supported data types, and the variants for different shapes and devices.

Behind the card sits the full repository. The effect is to turn a shader into a reusable software artifact: the interface can be read without reading any WGSL, the tests live next to the implementation, and published versions can be loaded explicitly instead of through an unversioned file URL. Hugging Face also expects them to serve as reference implementations for people writing their own WebGPU kernels or building runtimes that integrate these operations. Running any of it requires a browser with WebGPU, whose availability depends on browser, operating system, GPU and driver; the check in JavaScript is "gpu" in navigator.

The library itself is thin. A developer calls getKernel with a Hub repository ID and a contract version, then runs the returned function with typed inputs and tensor shapes. The published example is a bias add where the second input broadcasts along the first dimension, so the result takes shape [2, 3]; the loader derives that shape and the logical data type from the manifest contract and the inputs, then allocates memory for the output automatically. Adding six floating-point numbers is deliberately the smallest possible case — at that size, moving data to the GPU costs far more than the arithmetic — and the point is that the call pattern does not change for the heavy operations where optimized kernels actually pay, such as ai.onnx.MatMul. Only the repository ID and the inputs change.

Even that trivial operation ships in four variants: a direct vectorized path for equal shapes, a vectorized broadcast with its own indexing logic, a scalar path, and a general broadcast for everything else. The runtime can pick per call and per device without the application seeing any difference. The version parameter is its own axis: version: 1 selects the first published contract for the kernel, separate from the ONNX operator set, the operator's since_version field, and the model version. An application pins a stable JavaScript contract while the implementation underneath is free to churn.

Now the benchmark, which is worth reading slowly. Hugging Face ran 1,756 test cases across all 207 operations against ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a, then kept the 809 cases where both sides produced matching results and timing was reliable. Across those: 2.57x by geometric mean, 1.90x by median, 629 wins, 176 losses, 4 ties. Individual operations went much further. A bilinear Einsum with indices i,ij,j at size 4096 took 0.136 ms against 1,396 ms for ORT WebGPU — more than 10,000 times slower. A row-wise CumSum over shape [256, 4096] came in 301 times faster, 0.016 ms against 4.784 ms. Hugging Face says plainly that these are individual cases and not the speed to expect from every operation, and that they show what a specialized kernel can do when a general implementation lands on a slow path.

The caveats are unusually honest, and they matter. Only GPU execution was timed: kernel loading, session creation, input transfer, shader compilation and reading results back were all excluded. Very short tasks are hard to measure, and small cases can be flattered by GPU cache. These are individual operations, not whole models.

My reading is that the median is the number to carry, not the geometric mean. 1.90x is a genuinely good result for a first release; 2.57x is the mean being pulled by a tail that includes a 10,000x outlier. The 176 losses are more interesting than either — a fifth of comparable cases where the hand-tuned kernel was beaten by a general-purpose runtime, which is roughly what you would expect from a collection this young and is the strongest argument for publishing the benchmarks alongside the code. And there is an internal tension the post does not quite resolve: the whole case for Fleet is that one machine's numbers tell you very little about WebGPU performance, yet the headline comparison is one machine's numbers, from an Apple M4.

The question nobody is asking is what happened to the other 947 cases. The filter criteria are stated — matching results and reliable timing — but not the split. How many were dropped because the timing was too noisy to trust, and how many because the two implementations disagreed on the answer, are very different facts. One is a measurement limitation. The other is a correctness gap in a library whose entire premise is that correctness tests ship with the kernel.

Fleet is the intended remedy. It lets anyone run the correctness and performance checks in their own browser and see how the kernels behave on their hardware, and with the user's consent each run contributes private data back — enough to surface device-specific bugs, compare kernel variants and improve the rules for choosing between them. The goal is a picture assembled from real hardware no ordinary lab can cover. Hugging Face is also working with the ONNX Runtime team to upstream the improvements into ONNX Runtime Web, which is the right instinct: the value of this collection never depended on ORT staying slow.

The kernels sit on the Hub's Kernels page alongside CUDA, ROCm, Metal and other platforms, filterable and sortable like any other artifact.

Hugging Face calls 207 a starting point and plans to connect the kernels to higher-level model tooling, widen operation coverage and make fast local inference easier across the WebAI ecosystem. The structural consequence is already visible: until now, the speed of a model in a browser was a property of whichever runtime you chose, and improving it meant waiting for that runtime's maintainers. Unbundling the operations moves that decision outside the runtime — and puts whoever maintains the kernel collection underneath everyone else's stack.