MTP: option to use re-quantized output tensor for better TG performance (#1809)

* Option to use re-quantized output tensor for MTP

* Remove quantize extra output option

* Handle interleaved types
This commit is contained in:
Kawrakow
2026-05-16 14:40:18 +03:00
committed by GitHub
parent 5cc0d86c76
commit 3e573cfea6
10 changed files with 312 additions and 77 deletions
+2
View File
@@ -394,6 +394,8 @@ extern "C" {
int32_t n_v_first;
int32_t n_v_last;
enum ggml_type extra_output_type;
// proportion of the model (layers or rows) to offload to each GPU, size: llama_max_devices()
const float * tensor_split;