mirror of
https://github.com/furyhawk/home_stack.git
synced 2026-07-21 18:26:54 +00:00
furyhawk
3c75694260
- Introduced a new Jupyter notebook `unsloth_nb.ipynb` for utilizing the Unsloth library. - Implemented code to load and patch FastLanguageModel and FastQwen2Model for improved training speed. - Configured models to support 4-bit quantization for reduced memory usage. - Included detailed logging for model loading and system specifications.
4 lines
296 B
Bash
Executable File
4 lines
296 B
Bash
Executable File
#!/bin/bash
|
|
|
|
# Start Llama.cpp service using Podman with GPU support
|
|
podman run -d --name llamacpp-server -p 8000:8000 --gpus all -v $HOME/models:/models localhost/local/llama.cpp:full-cuda -s -m /models/DeepSeek-R1-0528-Qwen3-8B-UD-Q4_K_XL.gguf --port 8000 --host 0.0.0.0 -n 512 --n-gpu-layers 1 |