mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-07-21 02:05:35 +00:00
fix(rpc): update ggml_backend_cuda_init to 3-arg signature (#2084)
Commit 75a5f6d0 (Per model CUDA contexts) added a third parameter
"model" to ggml_backend_cuda_init, but rpc-server was missed.
The RPC server creates backends at startup before any model is
loaded, so passing nullptr for the model parameter is correct: the
RPC server acts as a pure compute proxy and does not host models
locally. The nullptr serves as the context grouping key in
all_ctx, which is sufficient since all RPC backends in the same
process share the same group for multi-GPU reduce operations.
Fixes: cmake build failure with GGML_CUDA=ON + GGML_RPC=ON
This commit is contained in:
@@ -263,7 +263,7 @@ static ggml_backend_t create_gpu_backend(const rpc_server_params& params, uint32
|
||||
ggml_backend_t backend = NULL;
|
||||
#ifdef GGML_USE_CUDA
|
||||
fprintf(stderr, "%s: using CUDA backend: CUDA%d\n", __func__, device);
|
||||
backend = ggml_backend_cuda_init(device, nullptr); // init device
|
||||
backend = ggml_backend_cuda_init(device, nullptr, nullptr); // init device
|
||||
if (!backend) {
|
||||
fprintf(stderr, "%s: ggml_backend_cuda_init() failed\n", __func__);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user