Thireus ☠ and GitHub
6d78a87c4c
perplexity: signal-driven hot-swap mode for persistent per-tensor PPL/KLD benchmarking (extends #1989 ) ( #2131 )
...
* perplexity: add signal-driven hot-swap mode for persistent KLD/PPL benchmarking
llama-perplexity can now stay resident and be driven through control/status files (reload/compute/exit): it reloads only the tensors that changed on disk and recomputes PPL/KLD without ever reloading the full model. File-based signalling works on Windows, macOS and Linux. The reload returning-to-original path now refreshes tensor data from disk instead of
reattaching stale weights.
* Not Cygwin specific
2026-07-14 12:56:03 +03:00
Thireus ☠ and GitHub
86597623a5
Port of Qwen3-VL support from mainline ( #883 )
...
* Port of Qwen3-VL for latest ik_llama.cpp
- convert_hf_to_gguf.py - Not touched, use llama.cpp to convert model instead
- sysl and metal support for imrope not added
- Vulkan support for imrope not tested
- Code not tested
* Bugfix n_embd was declared multiple times
https://github.com/ikawrakow/ik_llama.cpp/pull/883#issuecomment-3471179655
* Fix n_embd issue with qwen3vl
* model.output tensor not required
https://github.com/ikawrakow/ik_llama.cpp/pull/883#discussion_r2480388389
* Improved logic for qkv combined tensors
https://github.com/ikawrakow/ik_llama.cpp/pull/883/files/59ceaf8fcbaf28b29c96d7afb04c526e5c82c2fd#r2480395800
https://github.com/ikawrakow/ik_llama.cpp/pull/883/files/59ceaf8fcbaf28b29c96d7afb04c526e5c82c2fd#r2480398187
* Fix n_embd for merge_qkv() + cleaner code
https://github.com/ikawrakow/ik_llama.cpp/pull/883#discussion_r2481227395
* Revert TENSOR_NOT_REQUIRED
2025-11-04 19:20:54 +02:00
Thireus ☠ and GitHub
d65d5fe29e
Add support for GLM-4.5 models ( #668 )
...
* GLM-4.5
* GLM-4.5
* GLM-4.5
* convert_hf_to_gguf.py compatibility bugfix with GLM-4.5
From @ubergarm - https://github.com/ikawrakow/ik_llama.cpp/pull/668#issuecomment-3145913701
* Add ubergarm comments + my own
* Revert to llama.cpp script version that produced good BF16
See: https://github.com/ikawrakow/ik_llama.cpp/pull/668#issuecomment-3147374559
* Support for jinja chat templates
See https://github.com/ikawrakow/ik_llama.cpp/pull/668#issuecomment-3148109962
* GLM-4.5 llama.cpp final port
* Handle TENSOR_SKIP
Ported the hanges from:
https://github.com/sammcj/llama.cpp/commit/f129567dc0232272358ea71c5017486554b2abd3
https://github.com/sammcj/llama.cpp/commit/dcbbd2cb057a6c6e907e0195395a74201ef19e1b
Except op info since ik_llama.cpp doesn't support this operation.
* Bugfix for TENSOR_SKIP
skip loading if a tensor has the TENSOR_SKIP flag - @ubergarm via https://github.com/ikawrakow/ik_llama.cpp/pull/668#issuecomment-3155297198
* Update llama.cpp
Restore original GGLM_ASSERT
* Fix chat template detection
Changes suggested by @ubergarm - https://github.com/ikawrakow/ik_llama.cpp/pull/668#issuecomment-3155927840
* Revert to original GGML_ASSERT
2025-08-07 07:55:00 +03:00
Thireus ☠ and GitHub
cc51044e72
Add GGML_MAX_CONTEXTS definition in CMakeLists.txt ( #622 )
...
* Add GGML_MAX_CONTEXTS definition in CMakeLists.txt
If this entry is missing, GGML_MAX_CONTEXTS is ignored
* Update CMakeLists.txt
add_compile_definitions for GGML_MAX_CONTEXTS
2025-07-17 08:50:42 +02:00
Thireus ☠ and GitHub
eddeaac009
Bump Windows max open files from 512 to 2048 ( #620 )
...
* Bump windows max open files from 512 to 2048
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmaxstdio?view=msvc-160
* Make _GGML_STDIO_TARGET dependent of GGML_MAX_CONTEXTS for Windows
2025-07-17 08:50:26 +02:00
Thireus ☠ and GitHub
da38486de5
Bump GGML_MAX_CONTEXTS to allow loading more shards ( #611 )
...
* Bump GGML_MAX_CONTEXTS to allow loading more shards
This var prevents more than 64 shards from being loaded - Specifically relevant for large models such as DeepSeek R1.
* https://github.com/ikawrakow/ik_llama.cpp/pull/611#issuecomment-3072175559
2025-07-16 14:11:19 +02:00
