furyhawk/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-07-21 02:05:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: token "corruption" due to wrong RoPE type for GLM-DSA (#2099)

Browse Source 
The DSA lightning indexer hardcoded LLAMA_ROPE_TYPE_NEOX for indexer Q/K
positional encoding, but the GLM-5.2 model config.json explicitly sets
"indexer_rope_interleave": true, meaning the indexer uses interleaved
RoPE (LLAMA_ROPE_TYPE_NORM).

GLM-5.1 config.json also sets "rope_interleave": true.

The mismatch caused incorrect indexer scores, wrong top-k key selection,
and single-character token errors and other corruption that could grow
worse with context length.

Here the setup is changed to use the model's given rope_type instead of
hardcoding NEOX. This fixes GLM-5.2, GLM-5.1, and should be less brittle
for future GLM variants.
This commit is contained in:
usrlocalben
2026-07-09 08:49:21 +03:00
committed by GitHub
parent 6198a356a8
commit 9647246458
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/graphs/build_deepseek2.cpp
+2 -2
View File
@@ -396,7 +396,7 @@ ggml_tensor * llm_build_context::build_deepseek2_dsa_indexer(
ggml_row_size(indexer_q->type, rope_dim));
indexer_q_pe = ggml_rope_ext(ctx0, indexer_q_pe, inp_pos, nullptr, n_rot,
LLAMA_ROPE_TYPE_NEOX, n_ctx_orig, freq_base, freq_scale,
rope_type, n_ctx_orig, freq_base, freq_scale,
ext_factor, attn_factor, beta_fast, beta_slow);
// {head_size, n_ihead, n_tokens}
@@ -419,7 +419,7 @@ ggml_tensor * llm_build_context::build_deepseek2_dsa_indexer(
ggml_row_size(indexer_k->type, rope_dim));
indexer_k_pe = ggml_rope_ext(ctx0, indexer_k_pe, inp_pos, nullptr, n_rot,
LLAMA_ROPE_TYPE_NEOX, n_ctx_orig, freq_base, freq_scale,
rope_type, n_ctx_orig, freq_base, freq_scale,
ext_factor, attn_factor, beta_fast, beta_slow);
// {head_size, 1, n_tokens}