Repository navigation

ggml-cpu: add Q1_0 AVX2 fast path - #2

Closed

elusznik wants to merge 106 commits into

Commits on Mar 30, 2026

server: wrap headers for mcp proxy (#21072)

authored
abf9a62
ci : bump ty to 0.0.26 (#21156)
CISC
authored
e2eb39e
llama-model-loader: print warning when using overrides with mmap (#20978)
am17an
authored
278521c
webui: Fix branching logic on edit message (#21175)
allozaur
authored
389c7d4
rpc : fix misleading error log (#21184)
rgerganov
authored
cad2d38
CUDA : Fix CUB's argsort when nrows % block_size == 0 CCCL < 3.1 (#21181)
ORippler
authored
64ac9ab
jinja : handle empty expressions correctly (#20913)

zeph1912
and
CISC
authored
ead417f
CI : Enable CUDA and Vulkan ARM64 runners and fix CI/CD (#21122)

authored
84ae843
opencl: add q4_K gemm and gemv kernels for Adreno (#20919)

shaofeiqi
and
lhez
authored
08f2145

Commits on Mar 31, 2026

common : Disable backend sampling if reasoning budget is enabled (#21209)
Galunid
authored
5ce013c
vendor : update BoringSSL to 0.20260327.0 (#21211)
angt
authored
26dac84
server/webui: cleanup dual representation approach, simplify to openai-compat (#21090)

pwilkin
and
allozaur
authored
4453e77
fix: include API key in CORS proxy requests for MCP connections (#21193)
satishkc7
authored
fcc2d59
common: add bounds check in common_init_result::sampler to prevent segfault on failed model load (#21082)
mtmcp
authored
90aa83c
sycl : enhance fattn perf (#21185)
arthw
authored
62278ce
common : move up common_init() and fix Windows UTF-8 logs (#21176)
angt
authored
41361c8
ggml : bump version to 0.9.9 (ggml/1449)
ggerganov
committed
0be6c7c
sync : ggml
ggerganov
committed
9281dd1
CI: Enable CPU and Vulkan ARM64 Release (#21207)
ehfd
authored
eec6f85
fix: correct misspellings in code comments (#21217)
lainon1
authored
0b6ff47
common : gpt-oss handle builtin and unsolicited tool calls (#21213)
aldehir
authored
624733d
server: (webui) no more gzip compression (#21073)

Xuan-Son Nguyen
and
allozaur
authored
4a00bbf
CANN: fix multi-thread set_tensor race conditions (#20151)
hipudding
authored
632219a
common : cleanup logs and modernize the progress bar (#21215)
angt
authored
6307ec0
fix: Use lower-case proxy headers naming (#21235)
allozaur
authored
0fcb376
ggml-webgpu: port all AOT operators to JIT (#20728)
abhijitramesh
authored
825eb91

Commits on Apr 1, 2026

ggml webgpu: quantized buffers to u32 + wider browser/device support (#21046)
reeselevine
authored
82764c3
llama : refactor llama_model_quantize_params to expose a pure C interface (#20346)

EAddario
and
ggerganov
authored
4951250
CUDA: Add Flash Attention Support for Head Dimension 512 (#20998)

anavp-nvidia
and
JohannesGaessler
authored
8845816
ggml-cpu: fix fallback for RVV kernels without zvfh (#21157)
taimur-10x
authored
2b86e5c
ggml : fix RWKV ops thread assignment (#21226)
ggerganov
authored
d43375f
CUDA/HIP: Fix kernel slection for mmvq mmid kernel to align host selection with device launch bounds (#21238)
IMbackK
authored
88d5f8f
memory: respect unified KV cache in hybrid memory for eval tasks (#21224)
mudler
authored
e1cb817
ggml-cuda: Add generic NVFP4 MMQ kernel (#21074)

michaelw9999
and
JohannesGaessler
authored
84f82e8
sycl : support nvfp4 type in mul_mat (#21227)
arthw
authored
6b949d1
ggml : bump version to 0.9.10 (ggml/1454)
ggerganov
committed
296bc05
sync : ggml
ggerganov
committed
6422036
scripts: add function call test script (#21234)
Xuan-Son Nguyen
authored
0356e33
llama : rotate activations for better quantization (#21038)
ggerganov
authored
744c0c7
fix: tool call parsing for LFM2 and LFM2.5 models (#21242)
jbuchananr
authored
1d6d4cf
hexagon: improve RMS_NORM and DIV accuracy (#21251)

aparmp-quic
and
srikris-sridhar
authored
8710e5f
Update Dawn version in WebGPU CI (#20784)
nikhilJain17
authored
5a0ed51
kleidiai: add CPU feature detection to CI run script (#20394)
martin-klacer-arm
authored
6de97b9
CUDA: fix FA kernel selection logic (#21271)
JohannesGaessler
authored
86221cf
server: Bypass API Key validation for WebUI static bundle assets (#21269)
allozaur
authored
12dbf1d
opencl: fix leak in Adreno q8_0 path (#21212)
lhez
authored
95a6eba
contrib : rewrite AGENTS.md, make it more clear about project values (#21270)
Xuan-Son Nguyen
authored
c30e012

Commits on Apr 2, 2026

hexagon : add cumsum op support (#21246)

tboinovski1
and
max-krasnyansky
authored
fbd441c
sycl : fix llama_kv_cache hang when kv_cache is huge: 5GB (#21283)
arthw
authored
4888137
ggml : bump version to 0.9.11 (ggml/1456)
ggerganov
committed
bc07d55
sync : ggml
ggerganov
committed
dae2bf4
Ignore Transfer-Encoding header. (#20269)
crmky
authored
d6dac92
kv-cache : do not quantize SWA KV cache (#21277)
ggerganov
authored
17193cc
chat : add Granite 4.0 chat template with correct tool_call role mapping (#20804)

jesus-talavera-ibm
and
claude
authored
6137c32
Relax prefill parser to allow space. (#21240)
pwilkin
authored
e15efe0
common : add commentary rules for gpt-oss-20b (#21286)
aldehir
authored
2233737
model, mtmd: fix gguf conversion for audio/vision mmproj (#21309)
Xuan-Son Nguyen
authored
63f8fe0
tests: allow exporting graph ops from HF file without downloading weights (#21182)
0cc4m
authored
5803c8d
ggml-webgpu: add vectorized flash attention (#20709)

ArberSephirotheca
and
reeselevine
authored
a1cfb64
tests : add unit test coverage for llama_tensor_get_type (#20112)

authored
7992aa7
fix: gemma 4 template (#21326)
pwilkin
authored
5208e2d
[HIP] Bump ROCm version to 7.2.1 (#21066)

slojosic-amd
and
CISC
authored
7c7d6ce

Commits on Apr 3, 2026

ci : add AMD ZenDNN label to PR labeler (#21345)

z-vishal
and
taronaeo
authored
f49e917
(revert) kv-cache : do not quantize SWA KV cache (#21332)
ggerganov
authored
39b27f0
chat : avoid including json in chat.h (#21306)
ggerganov
authored
57ace0d
rpc : reuse compute graph buffers (#21299)
rgerganov
authored
0c58ba3
vocab: fix Gemma4 tokenizer (#21343)

authored
b069b10
ggml-zendnn : add MUL_MAT_ID op support for MoE models (#21315)

z-vishal
and
taronaeo
authored
f1ac841
fix: add openssl to nix dependencies (#21353) (#21355)
Tillerino
authored
f851fa5
HIP: build eatch ci build test for a different architecture (#21337)
IMbackK
authored
43a4ee4
fix: remove stale assert (#21369)
pwilkin
authored
d3416a4
ci: add more binary checks (#21349)
taronaeo
authored
887535c
jinja: coerce input for string-specific filters (#21370)
CISC
authored
1f34806
docs: Update build.md: HSA_OVERRIDE_GFX_VERSION clarification (#21331)
jeromew
authored
384c007
docker : bump cuda12 to 12.9.1 (#20920)

authored
277ff5f
common : fix tool call type detection for nullable and enum schemas (#21327)
sacredvoid
authored
af5c138
common/parser: fix call ID detection (Mistral parser mostly) + atomicity for tag-json parsers (#21230)

pwilkin
and
CISC
authored
f1f793a
server: save and clear idle slots on new task (`--clear-idle`) (#20993)

yychyo
and
ggerganov
authored
50e0ad0
ci: Add Windows Vulkan backend testing on Intel (#21292)
rillomas
authored
e439700
ggml-webgpu: move from parameter buffer pool to single buffer with offsets (#21278)
reeselevine
authored
d006858

Commits on Apr 4, 2026

llama: add custom newline split for Gemma 4 (#21406)
am17an
authored
b7ad48e
llama-model: read final_logit_softcapping for Gemma 4 (#21390)
ssam18
authored
650bf14
common : respect specified tag, only fallback when tag is empty (#21413)
angt
authored
d01f627
server: Fix undefined timing measurement errors in server context (#21201)

thedanhoffman
and
Dan Hoffman
authored
9c69907
common : add gemma 4 specialized parser (#21418)
aldehir
authored
b863507

Commits on Apr 5, 2026

ci: fix vulkan workflow referencing non-existent action (#21442)
nisparks
authored
661e9ac
ci: lower cuda12 floor to 12.8.1 for broader host compatibility (#21438)

M1DNYT3
and
M1DNYT3
authored
c08d28d
server : fix logging of build + system info (#21460)
ddh0
authored
5d3a4a7
ci : use default RISE RISC-V Runners (#21263)
luhenry
authored
761797f
model : add HunyuanOCR support (#21395)

authored
af76639
llama : correct platform-independent loading of BOOL metadata (#21428)
anchortense
authored
58190cc

Commits on Apr 6, 2026

hexagon: slight optimization for argosrt output init (#21463)
YardenTal44
authored
25eec6f
sycl : handle other FA case (#21377)
arthw
authored
f51fd36
convert : set "add bos" == True for Gemma 4 (#21500)
ggerganov
authored
400ac8e
docs: add hunyuan-ocr gguf, also add test [no ci] (#21490)
Xuan-Son Nguyen
authored
3979f2b
server : handle unsuccessful sink.write in chunked stream provider (#21478)
lainon1
authored
482d862
convert : fix block_ff_dim retrieval for lfm2 (#21508)
CISC
authored
941146b
vocab : add byte token handling to BPE detokenizer for Gemma4 (#21488)
aldehir
authored
4aa962e
llama-bench: add `-fitc` and `-fitt` to arguments (#21304)
am17an
authored
94ca829
[CUDA ] Write an optimized flash_attn_stream_k_fixup kernel (#21159)
gaugarg-nv
authored
15f786e
cli: fix stripping of \n in multiline input (#21485)

bipinyadav3175
and
CISC
authored
506200c
ggml: add Q1_0 1-bit quantization support (CPU) (#21273)

khosravipasha
and
CISC
authored
2e1f0a8
ggml-webgpu: Add the support of `MUL_MAT_ID` (#21147)

yomaytk
and
reeselevine
authored
d0a6dfe

Commits on Apr 7, 2026