mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-07-21 10:15:35 +00:00
Fused indexer top_k (CPU only) (#2098)
* Indexer topk op - CPU only We do save memory, but it is somehow much slower than what we have on main. * Make it a command line option
This commit is contained in:
@@ -492,6 +492,7 @@ extern "C" {
|
||||
bool rope_cache; // whether to use RoPE cache [EXPERIMENTAL]
|
||||
bool graph_reuse; // whether to reuse graphs when possible [EXPERIMENTAL]
|
||||
bool dsa; // enable GLM DSA sparse attention (off by default) [EXPERIMENTAL]
|
||||
bool fused_idx_topk; // enable the fused indexer topk op (off by default) [EXPERIMENTAL]
|
||||
int dsa_top_k; // DSA top-k override (<0 => model's configured indexer_top_k) [EXPERIMENTAL]
|
||||
int min_experts;
|
||||
float thresh_experts;
|
||||
|
||||
Reference in New Issue
Block a user