furyhawk/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-07-21 10:15:53 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,075 Commits 618 Branches 6,870 Tags
master
Commit Graph
59 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
lhezandGitHub 86a9c79f86 opencl: load and use kernel_gemm_moe_q6_k_f32_ns from bin kernel lib (#25797) 2026-07-17 15:29:29 -07:00
lhezandGitHub 931ca30bef opencl: fix potential crash in aos reconstruct (#25383) 2026-07-07 20:34:29 -07:00
lhezandGitHub 4fc4ec5541 opencl: allow loading precompiled binary kernels from library (#23042) 
* opencl: allow loading binary kernel

* opencl: add libdl.h

* ggml-backend-dl is in ggml, which depends backend libs, thus
  ggml-opencl cannot depend on ggml-backend-dl
* add libdl.h to break cyclic dep

* opencl: allow loading bin kernel lib

* opencl: load `gemm_moe_mxfp4_f32_ns` from kernel lib if available

* opencl: load q8_0 gemm from kernel lib

* opencl: load q4_0 moe gemm from kernel lib

* opencl: load q4_1 moe gemm from kernel lib

* opencl: load q4_k moe gemm from kernel lib

* opencl: always declare `get_adreno_bin_kernel_func_t`

* opencl: rephrase message

* opencl: fix for rebase

* opencl: update doc
 2026-07-01 10:29:22 -07:00
lhezandGitHub fd1a05791d opencl: initial q1_0 support (#25160) 
* opencl: general q1_0 support

* opencl: add Adreno GEMM/GEMV for q1_0
 2026-06-30 21:43:20 -07:00
lhezandGitHub fdb2c11c70 opencl: support non-contig rows in norm (#24965) 2026-06-24 19:21:25 -07:00
lhezandGitHub 51571722aa opencl: optimize mul_mat_f16_f32_l4 for decode (#24504) 2026-06-16 23:21:26 -07:00
lhezandGitHub 308f61c31f opencl: improve get_rows, cpy, concat and q6_k flat gemv (#24160) 
* opencl: allow multiple workgroups for large rows

* opencl: improve small cpy

* opencl: packed concat for small input

* opencl: tweak flat q6_K gemv, increase N_DST and remap threads
 2026-06-05 13:45:25 -07:00
lhezandGitHub 63e66fdd23 opencl: use flat variants of q4_K and q6_K gemv for very large M (#24006) 2026-06-02 14:16:17 -07:00
lhezandGitHub 210a6570ce opencl: fix compiler warnings for non-adreno path (#23922) 
* opencl: fix compiler warnings for non-adreno path

* opencl: fix const cast warning
 2026-06-01 19:15:09 -07:00
lhezandGitHub d6588daa80 opencl: support bf16 by converting to f16 (#23839) 2026-05-30 10:17:47 -07:00
lhezandGitHub 408ae2b9e5 opencl: move backend info printing into its own function (#23702) 
* opencl: move backend info print into its own function

* opencl: move new log line

* opencl: fix for non adreno path
 2026-05-28 11:05:42 -07:00
lhezandGitHub 3a6db741a8 opencl: refactor backend initilization (#23318) 
* opencl: refactor initialization

* opencl: refactor GPU identification

* opencl: rename for consistency

* opencl: cache global mem size in dev_ctx

* opencl: adjust log level

* opencl: load argsort and flash_attn kernels in supports_op

* argsort kernel must be built for supports_op for querying the max
  workgroups
* flash_attn kernel has many variants, only load them when needed
 2026-05-20 09:57:36 -07:00
lhezandGitHub 1e4579fbb8 opencl: fix crash when warming up MoE on Adreno (#22876) 2026-05-13 11:24:33 -07:00
lhezandGitHub ff806a110d opencl: refactor Adreno q4_0 (#22335) 
* opencl: refactor adreno q4_0 gemm/gemv dispatch

* opencl: refactor q4_0 gemm/gemv loading, use consistent names

* opencl: use consistent name for adreno q8_0 gemm/gemv

* opencl: use consistent names for adreno q4_0 gemm/gemv

* opencl: simplify adreno q4_0 set_tensor

* opencl: refactor q4_0 get_tensor
 2026-05-05 09:38:57 -07:00
lhezandGitHub f454bd7eb8 opencl: add iq4_nl support (#22272) 
* opencl: add general support for iq4_nl

* opencl: add iq4_nl gemm/gemv for adreno

* opencl: pack 2 lut entries into a uint
 2026-04-25 21:21:58 -07:00
lhezandGitHub 5e6c0e18b6 opencl: refactor q8_0 set_tensor and mul_mat host side dispatch for Adreno (#21938) 
* opencl: refactor q8_0 gemm/gemv Adreno dispatch

* opencl: refactor q8_0 set_tensor

* opencl: fix whitespace
 2026-04-16 22:28:33 -07:00
lhezandGitHub 95a6ebabb2 opencl: fix leak in Adreno q8_0 path (#21212) 2026-04-01 12:54:58 -07:00
lhezandGitHub ded446b34c opencl: allow large buffer for adreno (#20997) 2026-03-26 08:52:21 -07:00
lhezandGitHub 1772701f99 opencl: add q6_K gemm and gemv kernels for Adreno (#20089) 
* opencl: add q6_K noshuffle kernels, initial q6_K gemv, some host code

* opencl: add q6_K transpose

* opencl: fix cvt kernel name

* opencl: add call to q6_K gemv

* opencl: fix q6_K scale transpose

* opencl: fix loading for gemv q6_K, refactor

* opencl: fix transpose_8_buf kernel assignment, refactor

* opencl: refactor q6_K transpose

* opencl: add gemm_noshuffle_q6_k_f32

* opencl: fix qh loading

* opencl: refactor q6_K gemv host side, release bufs and imgs

* opencl: refactor

* opencl: fix q6_K dequant and scale selection

* opencl: workaround compiler bug, fix dump_tensor

* opencl: refactor q6_K convert kernels

* opencl: unpack transformed q6_K in get_tensor

* opencl: refactor, handle non-uniform workgroups

* opencl: support non-vector subgroup bcast
 2026-03-23 12:44:18 -07:00
lhezandGitHub 3b439504ba opencl: fix l2_norm (#20480) 2026-03-13 22:18:52 -07:00
lhezandGitHub 0516e04bf9 opencl: use larger workgroup size for get_rows (#20316) 2026-03-11 22:03:27 -07:00
lhezandGitHub 6fce5c6a7d opencl: add l2_norm (#20160) 2026-03-06 18:03:05 -08:00
lhezandGitHub 6c97bffd65 opencl: add neg, exp and diag (#20127) 
* opencl: add `neg`

* opencl: add `exp`

* opencl: add `diag`
 2026-03-05 21:16:39 -08:00
lhezandGitHub 69fd345335 opencl: add SET, support i32 for CPY, minor refactor for cpy (#20101) 2026-03-04 21:32:26 -08:00
lhezandGitHub 79cc0f2daf opencl: add basic support for q4_1 (#19534) 
* opencl: add q4_1 mv

* opencl: clean up

* opencl: add flattened q4_1 mv

* opencl: clean up

* opencl: add basic q4_1 mm

* opencl: fix whitespace

* opencl: add general q4_0 mm
 2026-02-12 14:52:37 -08:00
lhezandGitHub 4d3daf80f8 opencl: add general Q6_K mm and Q4_K mv (#19347) 
* opencl: add general q6_k mm

* opencl: refine condition for q6_K mm

* opencl: add general q4_K mv

* opencl: fix whitespace
 2026-02-11 10:33:13 -08:00
lhezandGitHub 91ea44e89b opencl: refactor some ops, concat, repeat, tanh and scale (#19226) 
* opencl: refactor concat

* opencl: refactor repeat

* opencl: refactor tanh

* opencl: enable fp16 for tanh

* opencl: refactor scale

* opencl: fix unused variables
 2026-02-02 15:54:43 -08:00
lhezandGitHub 94eeb5967c opencl: add flattened q6_K mv (#19054) 
* opencl: flatten `q6_K` and add `kernel_mul_mv_q6_K_f32_flat`

* opencl: clean up

* opencl: refactor q6_K mv - put loop body in `block_q_6_K_dot_y_flat`

* opencl: tweak the workgroup size a bit

* opencl: output 4 values per subgroup for `kernel_mul_mv_q6_K_f32_flat`

* opencl: proper alignment for q6_K

* opencl: boundary handling for flattened q6_K mv

* opencl: rename q6_K mv kernel file

* opencl: put flattened q6_K mv in its own file

* opencl: use lower k in file name

* opencl: use K in variable names
 2026-01-26 19:36:24 -08:00
lhezandGitHub 9c96465f99 opencl: enable the general fp mm for non-cont input and as a fallback for specialized kqv kernel for adreno (#18970) 
* opencl: add `copy_to_contiguous` and utilize mm kernels

* opencl: only copy to cont for f32 and f16 tensors

* opencl: use cont mm for fallback when dst is large

* opencl: use nb local to copy-to-cont

* opencl: use local offset as well
 2026-01-22 10:29:25 -08:00
lhezandGitHub d1b4757ded opencl: fix q6_K mv for m=1 (#18893) 2026-01-17 13:50:32 -08:00
lhezandGitHub 08566977a7 opencl: allow resizing transpose buffers (#18384) 
* opencl: allow resizing transpose buffers instead of using fixed sizes

* opencl: remove commented code
 2025-12-27 15:51:14 -08:00
lhezandGitHub eb492bf43f opencl: unpack q4_0 for adreno in get_tensor (#18278) 2025-12-22 10:19:01 -08:00
lhezandGitHub 2d2e1030e3 docs : update opencl ops (#17904) 2025-12-10 15:20:00 +01:00
lhezandGitHub 7cba58bbea opencl: add sqr, sqrt, mean and ssm_conv (#17476) 
* opencl: add sqr

* opencl: add sqrt

* opencl: add mean

* opencl: add ssm_conv

* opencl: add missing cl_khr_fp16

* opencl: do sqrt in f32 then convert to f16 for better precision
 2025-11-26 13:29:58 -08:00
lhezandGitHub 8e9ddba610 opencl: refine condition for kqv mm (#17392) 2025-11-21 14:34:48 -08:00
lhezandGitHub 52e5d421f1 opencl: fix rms_norm_mul (#17250) 
* opencl: use subgrroup reduce for reduction in rms_norm_mul

* opencl: add comment about workgroup size
 2025-11-15 17:40:14 -08:00
lhezandGitHub ece0f5c177 opencl: add fastdiv and use it in set_rows, ported from cuda (#17090) 
* opencl: add fastdiv for mm q8_0

* opencl: use uint4 for fastdiv vals

* opencl: use fastdiv for set_rows

* opencl: do not use fastdiv for q8_0 mm
 2025-11-10 15:00:13 -08:00
lhezandGitHub 5e90233bdb opencl: update doc (#17011) 
* opencl: update docs

* opencl: update docs

* opencl: fix link

* opencl: update doc
 2025-11-04 16:02:36 -08:00
lhezandGitHub c5023daf60 opencl: support imrope (#16914) 
* opencl: support imrope

* opencl: fix whitespace
 2025-11-03 11:47:57 -08:00
lhezandGitHub 9984cbb61d opencl: fix boundary handling for mul_mm (#16875) 2025-10-30 16:00:20 -07:00
lhezandGitHub 6ea37f5739 opencl: fix warnings and clean up profiling (#16688) 
* opencl: remove unused headers, fix warnings

* opencl: clean up profiling, only keep kernel time
 2025-10-20 22:26:17 -07:00
lhezandGitHub 0cb7a0683b opencl: add q8_0 mm support (#16469) 
* opencl: add mm_q8_0_f32

* opencl: fix data loading for incomplete tile

* opencl: use q8_0 mm for larger matrix

* opencl: add some tests to cover the path
 2025-10-15 10:51:04 -07:00
lhezandGitHub d93f8439b0 opencl: fix FA for f32 (#16584) 2025-10-15 10:48:28 -07:00
lhezandGitHub 5016b72862 opencl: fix build targeting CL 2 (#16554) 2025-10-13 11:50:37 -07:00
lhezandGitHub 7c156df414 opencl: support pad_ext (#15888) 2025-09-30 10:45:45 -07:00
lhezandGitHub d1c84a662d opencl: support ne3 in get_rows (#15866) 2025-09-30 09:55:13 -07:00
lhezandGitHub de41f2b7bf codeowners: add codeowners for opencl backend (#16344) 2025-09-30 08:30:16 +03:00
lhezandGitHub 51f5a45fbe opencl: fix concat crash on win arm64 with Adreno (#15944) 2025-09-21 16:42:10 -07:00
lhezandGitHub c4510dc937 opencl: initial q8_0 mv support (#15732) 2025-09-21 14:48:44 -07:00
lhezandGitHub f7207b0415 opencl: fix support ops condition for rms_norm (#15560) 2025-08-25 14:18:09 -07:00
lhezandGitHub fb22dd07a6 opencl: mark argsort unsupported if cols exceed workgroup limit (#15375) 2025-08-19 11:25:51 -07:00
lhezGitHubshawngu-quic
 		e2c1bfff53 opencl: add initial mxfp4 support via mv (#15270) 
* opencl: add reference `mul_mv_mxfp4_f32`

* opencl: add reference `mul_mv_id` for mxfp4

* Q4_0 tranpose fix for Adreno

---------

Co-authored-by: shawngu-quic <shawngu@qti.qualcomm.com>
 2025-08-15 09:52:14 -07:00
lhezandGitHub aaa3d07ae7 opencl: support sink in soft_max (attn sinks) (#15152) 2025-08-07 21:47:03 -07:00
lhezandGitHub e725a1a982 opencl: add swiglu_oai and add_id (#15121) 
* opencl: add `swiglu-oai`

* opencl: add `add_id`

* opencl: add missing `add_id.cl`
 2025-08-06 12:12:17 -07:00
lhezandGitHub 5c0eb5ef54 opencl: fix adreno compiler detection logic (#15029) 2025-08-02 19:51:18 +02:00
lhezandGitHub 6e6725459a opencl: add mul_mat_f32_f32_l4_lm and mul_mat_f16_f32_l4_lm (#14809) 2025-07-30 14:56:55 -07:00
lhezandGitHub 8ad7b3e65b opencl : add ops docs (#14910) 2025-07-28 18:50:17 +02:00
lhezandGitHub ce111d39d6 opencl: add fused rms_norm_mul (#14841) 
* opencl: add fused `rms_norm` + `mul`

* opencl: improve workgroup size for `rms_norm_mul`
 2025-07-25 17:12:13 +02:00
lhezandGitHub 8e6f8bc875 opencl: remove unreachable return (#14806) 2025-07-22 08:53:30 +02:00