furyhawk/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-07-20 09:45:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update docker (#2123)

Browse Source 
* llama-swap latest version

- Sample config still works

* llama-swap latest version ik_llama-cuda.Containerfile

- Sample config still works

* Update README.md

- The prebuilt docker images are fully working, move their instructions to the front
This commit is contained in:
Marian M.
2026-07-13 13:27:47 +03:00
committed by GitHub
parent a3836a5de0
commit 5d766f536a
3 changed files with 18 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+15 -2
View File
@@ -77,10 +77,23 @@ Start the server with one of the commands (CPU or GPU):
./build/bin/llama-server --model /my_local_files/gguf/Qwen_Qwen3-0.6B-IQ4_NL.gguf --ctx-size 4096 -ngl 999
```
That's all! Open [http://127.0.0.1:8080](http://127.0.0.1:8080) in Browser start chatting.
That's all! Open [http://127.0.0.1:8080](http://127.0.0.1:8080) in Browser and start chatting, or use the available API endpoins in your program/harness.
### Run in Docker or Podman
### [Step by step guide](./docker/README.md) for ik_llama.cpp in podman/docker container including llama-swap
Pull one of the available images from `ghcr.io`. [View all tags](https://github.com/ikawrakow/ik_llama.cpp/pkgs/container/ik-llama-cpp/versions?filters%5Bversion_type%5D=tagged)
```bash
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cpu-swap
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cpu-server
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cpu-full
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cu12-swap
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cu12-server
docker pull ghcr.io/ikawrakow/ik-llama-cpp:cu12-full
```
Check [Step by step guide](./docker/README.md) for image customization and other details.
### [Common parameters and options](./docs/parameters.md)
docker/ik_llama-cpu.Containerfile
+2 -2
View File
@@ -84,11 +84,11 @@ ENTRYPOINT [ "/app/llama-server" ]
# Stage 5: Swap
FROM server AS swap
ARG LS_REPO=mostlygeek/llama-swap
ARG LS_VER=199
ARG LS_VER=239
RUN curl -sSL "https://github.com/${LS_REPO}/releases/download/v${LS_VER}/llama-swap_${LS_VER}_linux_amd64.tar.gz" \
| tar -xz
COPY --from=build /app/docker/ik_llama-cpu-swap.config.yaml /app/config.yaml
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD [ "curl", "-f", "http://localhost:8080"]
ENTRYPOINT [ "/app/llama-swap", "-config", "/app/config.yaml" ]
ENTRYPOINT [ "/app/llama-swap", "-config", "/app/config.yaml" ]
docker/ik_llama-cuda.Containerfile
+1 -1
View File
@@ -86,7 +86,7 @@ ENTRYPOINT [ "/app/llama-server" ]
# Stage 5: Swap
FROM server AS swap
ARG LS_REPO=mostlygeek/llama-swap
ARG LS_VER=199
ARG LS_VER=239
RUN curl -sSL "https://github.com/${LS_REPO}/releases/download/v${LS_VER}/llama-swap_${LS_VER}_linux_amd64.tar.gz" \
| tar -xz