furyhawk/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-07-21 10:15:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

GLM-DSA: add ability to use quantized indexer cache (#2075)

Browse Source 
* GLM-DSA: improve TG performance even more

* Fix crash when using MTP

* GLM-DSA: much better PP performance (CPU-only)

* GLM-DSA: allow for quantized indexer cache
This commit is contained in:
Kawrakow
2026-07-06 12:04:15 +02:00
committed by GitHub
parent eacd450d2b
commit a0859ce5ed
4 changed files with 17 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
include/llama.h
+2
View File
@@ -378,6 +378,7 @@ extern "C" {
enum ggml_type type_k;
enum ggml_type type_v;
enum ggml_type idx_type_k;
uint32_t max_ctx_size;
int32_t n_seq_max;
int32_t n_ubatch;
@@ -465,6 +466,7 @@ extern "C" {
enum ggml_type type_k; // data type for K cache [EXPERIMENTAL]
enum ggml_type type_v; // data type for V cache [EXPERIMENTAL]
enum ggml_type idx_type_k; // data type for indexer K cache [EXPERIMENTAL]
enum ggml_type type_reduce; // data type for reduce operations
enum ggml_type type_graph_attn; // flash-attn precision under -sm graph
enum ggml_type type_k_first;