Exe Heavy Metal: custom Metal kernels for ggml, video up to 7.7× faster on Apple Silicon

[exe-hq]

Hello, I'm Chris, a solo developer from Germany. I'm quite new to GitHub, so please tell me if I'm doing something wrong here.

I use stable-diffusion.cpp every day on my Mac, and video was just too slow for me. So I spent the last months on the Metal side of ggml, on the ops that image and video models spend their time on. Today I made it public:

https://github.com/exe-ai-productions/exe-heavy-metal

stable-diffusion.cpp runs on top unmodified: same command line, same model files. Every change lives in ggml's Metal backend (convolutions, attention, GROUP_NORM, rotary embeddings, copies), and most kernels can be switched off at run time, so one build measures both paths.

Whole runs on an M2 Max (38-core GPU, 64 GB), from process start to the written file, against the official build master-1000-95ff762:

Model	Size	Stock	Heavy Metal	Speedup
LTX-2.3 22B Distilled, video, Q8_0	1280×704, 121 frames	2,715.0 s	350.8 s	7.7×
Qwen-Image 2.1, F16, 30 steps	1024×1024	1,035.4 s	479.7 s	2.2×
FLUX.3 Klein Base, Q8_0, 20 steps	1024×1024	465.0 s	171.0 s	2.7×
SDXL, F16, 20 steps	1024×1024	112.5 s	37.8 s	3.0×
SD 1.5, F16, 20 steps	512×512	28.5 s	7.0 s	4.1×

The median over 19 whole runs of 10 models is 2.2×. The LTX-2.3 video decode alone goes from 2,278.0 s to 20.6 s.

One finding that may matter here even without the fork: in stock ggml, Metal GROUP_NORM normalizes only the first image of a batch, so SDXL and SD 1.5 with batched CFG write a white image. --batched-cfg off avoids it on stock.

What I checked: ggml's test-backend-ops passes 17,940 of 17,940 cases on Metal against the CPU reference, and the README lists the PSNR of every run against the stock output.

Limits: measured on one M2 Max only. Weights in F16 or Q8_0. The FlashAttention kernel runs on M1 and M2; other chips keep ggml's attention kernel.

The method and all numbers are written up in seven short reports: https://exe-hq.net/research/

It is MIT, like ggml and stable-diffusion.cpp. It builds on leejet's work here, on ggml, and on liuliu's ccv, whose 3D convolution and Metal FlashAttention generators are vendored.

I only have this one Mac. If you try it on another chip, I would love to see your numbers. And thank you, leejet, for this project.

Chris