mirror of
https://github.com/furyhawk/home_stack.git
synced 2026-07-21 10:16:47 +00:00
furyhawk
e9dc288cef
- Created README.md for Ollama Docker setup with instructions for running and using models. - Added docker-compose.yml for Ollama service configuration. - Introduced scripts for model setup, including pulling models from Hugging Face and loading them into Ollama. - Implemented AI launcher script to choose between Ollama and LlamaCPP backends. - Added GPU support scripts for LlamaCPP with detailed logging and error handling. - Included simple model pull script for easier model management. - Established a structured approach for model management and integration with Ollama API.
2 lines
298 B
Bash
2 lines
298 B
Bash
podman build -t furyhawk/llama.cpp:full-cuda --target full -f .devops/cuda.Dockerfile .
|
|
podman run --name llamacpp-server --gpus all -v ~/models:/models localhost/furyhawk/llama.cpp:full-cuda -b -m /models/DeepSeek-R1-0528-Qwen3-8B-UD-Q4_K_XL.gguf --port 8000 --host 0.0.0.0 -n 512 --n-gpu-layers 1 |