furyhawk/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-07-21 10:15:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec : disacard last drafted token with low prob (#1820)

Browse Source 
* spec : disacard last drafted token with low prob

* Apply suggestion from @ikawrakow

Co-authored-by: Kawrakow <iwankawrakow@gmail.com>

---------

Co-authored-by: firecoperana <firecoperana>
Co-authored-by: Kawrakow <iwankawrakow@gmail.com>
This commit is contained in:
firecoperana
2026-05-19 08:35:35 +03:00
committed by GitHub
co-authored by firecoperana <firecoperana> Kawrakow
parent f43a9f1cf6
commit 104846ddee
2 changed files with 9 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/server/server-context.cpp
+1 -1
View File
@@ -5032,7 +5032,7 @@ void server_context::update_slots() {
// start populating the batch for this iteration
common_batch_clear(batch);
// frist, add sampled tokens from any ongoing sequences
// first, add sampled tokens from any ongoing sequences
add_sampled_tokens(); // Prepare batch for inference
// process in chunks of params.n_batch