Add Ollama Docker setup and model management scripts

- Created README.md for Ollama Docker setup with instructions for running and using models.
- Added docker-compose.yml for Ollama service configuration.
- Introduced scripts for model setup, including pulling models from Hugging Face and loading them into Ollama.
- Implemented AI launcher script to choose between Ollama and LlamaCPP backends.
- Added GPU support scripts for LlamaCPP with detailed logging and error handling.
- Included simple model pull script for easier model management.
- Established a structured approach for model management and integration with Ollama API.

ai_stack/llamacpp/.dockerignore

@@ -0,0 +1,8 @@
+# Python
+__pycache__
+app.egg-info
+*.pyc
+.mypy_cache
+.coverage
+htmlcov
+.venv

ai_stack/llamacpp/Dockerfile

@@ -0,0 +1,30 @@
+FROM ubuntu:latest
+
+# Install dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    cmake \
+    g++ \
+    wget \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Clone llama.cpp
+RUN git clone https://github.com/ggerganov/llama.cpp.git
+
+# Build llama.cpp with server support
+WORKDIR /llama.cpp
+RUN mkdir build && cd build && cmake .. -DLLAMA_SERVER=ON && make -j$(nproc)
+
+# Create models directory
+RUN mkdir -p /models
+
+# Expose the server port
+EXPOSE 8080
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8080/health || exit 1
+
+# Default command - will be overridden by docker-compose
+CMD ["/llama.cpp/llama-server", "--host", "0.0.0.0", "--port", "8080"]

ai_stack/llamacpp/README.md

@@ -0,0 +1,272 @@
+# LlamaCPP Model Setup
+
+Scripts and configuration for running DeepSeek-R1-0528-Qwen3-8B model with LlamaCPP server.
+
+## Available Scripts
+
+### 1. `pull-deepseek-llamacpp.sh`
+Downloads the DeepSeek-R1 GGUF model from Hugging Face and prepares it for LlamaCPP.
+
+**Features:**
+- Downloads the Q4_K_XL quantized version (optimal balance of quality and size)
+- Verifies download integrity
+- Tests server connectivity
+- Provides usage examples
+
+**Usage:**
+```bash
+# Using Makefile (recommended)
+make setup-llamacpp         # Start LlamaCPP and pull model automatically
+make pull-deepseek-llamacpp # Pull model only (requires server running)
+
+# Or run directly
+./scripts/pull-deepseek-llamacpp.sh
+```
+
+### 2. `llamacpp-gpu.sh`
+Starts GPU-accelerated LlamaCPP server with CUDA support for better performance.
+
+**Features:**
+- Automatic GPU detection
+- CUDA-optimized build
+- GPU layer offloading (35 layers for 8B model)
+- Enhanced performance parameters
+
+**Usage:**
+```bash
+make llamacpp-gpu           # Start GPU-accelerated server
+```
+
+## Makefile Targets
+
+### Basic Operations
+- `make llamacpp-up` - Start LlamaCPP container
+- `make llamacpp-down` - Stop LlamaCPP container
+- `make llamacpp-logs` - View container logs
+- `make pull-deepseek-llamacpp` - Download DeepSeek model
+- `make setup-llamacpp` - Complete setup (start + pull model)
+
+### GPU Operations
+- `make llamacpp-gpu` - Start GPU-accelerated server
+
+## Model Information
+
+**DeepSeek-R1-0528-Qwen3-8B-GGUF**
+- **Source:** [unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF)
+- **Quantization:** Q4_K_XL (4-bit quantization, extra large)
+- **Size:** ~5.5GB
+- **Context Length:** 32,768 tokens
+- **Performance:** Optimized for reasoning and conversation
+
+## Server Configuration
+
+### Default Parameters
+- **Host:** 0.0.0.0 (accessible externally)
+- **Port:** 8080
+- **Context Size:** 32,768 tokens
+- **Max Predict:** 2,048 tokens
+- **Batch Size:** 512
+- **Parallel Requests:** 2 (4 for GPU)
+- **Temperature:** 0.7
+- **Top-P:** 0.9
+- **Top-K:** 40
+
+### GPU Configuration
+- **GPU Layers:** 35 (automatically offloaded to GPU)
+- **CUDA Support:** Enabled
+- **Flash Attention:** Enabled
+- **Parallel Requests:** 4
+
+## Quick Start
+
+### CPU-Only Setup
+```bash
+# 1. Start LlamaCPP and download model
+make setup-llamacpp
+
+# 2. Test the server
+curl http://localhost:8080/health
+```
+
+### GPU-Accelerated Setup
+```bash
+# 1. Ensure model is downloaded
+make pull-deepseek-llamacpp
+
+# 2. Start GPU server
+make llamacpp-gpu
+
+# 3. Verify GPU usage
+nvidia-smi
+```
+
+## API Usage
+
+### Health Check
+```bash
+curl http://localhost:8080/health
+```
+
+### Text Completion
+```bash
+curl -X POST http://localhost:8080/completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "Hello, can you explain quantum computing?",
+    "max_tokens": 200,
+    "temperature": 0.7,
+    "stream": false
+  }'
+```
+
+### Streaming Completion
+```bash
+curl -X POST http://localhost:8080/completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "Write a Python function to sort a list:",
+    "max_tokens": 150,
+    "stream": true
+  }'
+```
+
+### Chat Completion (OpenAI-compatible)
+```bash
+curl -X POST http://localhost:8080/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "DeepSeek-R1",
+    "messages": [
+      {"role": "user", "content": "Explain machine learning in simple terms"}
+    ],
+    "max_tokens": 200,
+    "temperature": 0.7
+  }'
+```
+
+## Performance Optimization
+
+### CPU Optimization
+- **Threads:** Adjust based on CPU cores (default: 4)
+- **Batch Size:** Increase for better throughput (512-1024)
+- **Context Size:** Reduce if memory is limited
+
+### GPU Optimization
+- **GPU Layers:** Increase if you have more VRAM (max: 35 for 8B model)
+- **Batch Size:** Increase for GPU (1024-2048)
+- **Parallel Requests:** Increase based on VRAM capacity
+
+### Memory Requirements
+- **CPU Only:** ~8GB RAM minimum, 16GB recommended
+- **GPU:** ~6GB VRAM for full model, 4GB minimum with reduced layers
+
+## Troubleshooting
+
+### Server Won't Start
+```bash
+# Check logs
+make llamacpp-logs
+
+# Verify model file exists
+ls -la ai_stack/llamacpp/models/
+
+# Test with minimal configuration
+docker run --rm -v $(pwd)/ai_stack/llamacpp/models:/models \
+  llamacpp /llama.cpp/llama-server \
+  --model /models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_XL.gguf \
+  --host 0.0.0.0 --port 8080 --ctx-size 2048
+```
+
+### GPU Issues
+```bash
+# Check GPU availability
+nvidia-smi
+
+# Verify CUDA installation
+nvcc --version
+
+# Test GPU container
+docker run --rm --gpus all nvidia/cuda:12.1-base-ubuntu22.04 nvidia-smi
+```
+
+### Performance Issues
+```bash
+# Monitor resource usage
+docker stats llama-cpp-server
+
+# Check server metrics
+curl http://localhost:8080/metrics
+
+# Reduce context size for better performance
+# Edit docker-compose.yml: --ctx-size 8192
+```
+
+### API Errors
+```bash
+# Test basic connectivity
+curl -f http://localhost:8080/health
+
+# Check server response
+curl -v http://localhost:8080/completion \
+  -H "Content-Type: application/json" \
+  -d '{"prompt": "test", "max_tokens": 5}'
+```
+
+## File Locations
+
+- **Models:** `./ai_stack/llamacpp/models/`
+- **Dockerfile:** `./ai_stack/llamacpp/Dockerfile`
+- **Compose:** `./ai_stack/llamacpp/docker-compose.yml`
+- **Scripts:** `./scripts/pull-deepseek-llamacpp.sh`, `./scripts/llamacpp-gpu.sh`
+
+## Integration
+
+### With Frontend Applications
+```javascript
+// Example frontend integration
+const response = await fetch('http://localhost:8080/completion', {
+  method: 'POST',
+  headers: { 'Content-Type': 'application/json' },
+  body: JSON.stringify({
+    prompt: userInput,
+    max_tokens: 200,
+    temperature: 0.7,
+    stream: false
+  })
+});
+
+const result = await response.json();
+console.log(result.content);
+```
+
+### With Python
+```python
+import requests
+
+def query_llama(prompt, max_tokens=200):
+    response = requests.post('http://localhost:8080/completion', 
+        json={
+            'prompt': prompt,
+            'max_tokens': max_tokens,
+            'temperature': 0.7,
+            'stream': False
+        })
+    return response.json()['content']
+
+# Usage
+result = query_llama("Explain neural networks")
+print(result)
+```
+
+## Monitoring
+
+### Server Health
+- Health endpoint: `http://localhost:8080/health`
+- Metrics endpoint: `http://localhost:8080/metrics`
+- Logs: `make llamacpp-logs`
+
+### Performance Metrics
+- Request latency and throughput
+- Token generation speed
+- Memory and GPU utilization
+- Queue depth and processing time

ai_stack/llamacpp/docker-compose.yml

@@ -0,0 +1,38 @@
+version: '3.8'
+services:
+  llama-cpp-server:
+    build: .
+    ports:
+      - "8080:8080" # Map host port 8080 to container port 8080
+    volumes:
+      - ./models:/models # Mount a local directory for models
+    container_name: llama-cpp-server
+    environment:
+      - LLAMA_ARG_HOST=0.0.0.0
+      - LLAMA_ARG_PORT=8080
+    restart: unless-stopped
+    command: [
+      "/llama.cpp/llama-server",
+      "--model", "/models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_XL.gguf",
+      "--host", "0.0.0.0",
+      "--port", "8080",
+      "--ctx-size", "32768",
+      "--n-predict", "2048",
+      "--threads", "4",
+      "--batch-size", "512",
+      "--cont-batching",
+      "--parallel", "2",
+      "--flash-attn",
+      "--temperature", "0.7",
+      "--top-p", "0.9",
+      "--top-k", "40",
+      "--repeat-penalty", "1.1"
+    ]
+    # Uncomment for GPU support
+    # deploy:
+    #   resources:
+    #     reservations:
+    #       devices:
+    #         - driver: nvidia
+    #           count: all
+    #           capabilities: [gpu]

ai_stack/ollama/.dockerignore

@@ -0,0 +1,8 @@
+# Python
+__pycache__
+app.egg-info
+*.pyc
+.mypy_cache
+.coverage
+htmlcov
+.venv

ai_stack/ollama/Dockerfile

@@ -0,0 +1,39 @@
+# Use Ubuntu as base image for better compatibility with Ollama
+FROM ubuntu:22.04
+
+# Set environment variables
+ENV DEBIAN_FRONTEND=noninteractive
+ENV OLLAMA_HOST=0.0.0.0
+
+# Install dependencies
+RUN apt-get update && apt-get install -y \
+    curl \
+    ca-certificates \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install Ollama
+RUN curl -fsSL https://ollama.com/install.sh | sh
+
+# Create ollama user and directories
+RUN useradd -r -s /bin/false -m -d /usr/share/ollama ollama && \
+    mkdir -p /usr/share/ollama/.ollama && \
+    chown -R ollama:ollama /usr/share/ollama
+
+# Create directory for models
+RUN mkdir -p /models && chown -R ollama:ollama /models
+
+# Switch to ollama user
+USER ollama
+
+# Set working directory
+WORKDIR /usr/share/ollama
+
+# Expose the default Ollama port
+EXPOSE 11434
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:11434/api/version || exit 1
+
+# Start Ollama server
+CMD ["ollama", "serve"]

ai_stack/ollama/README.md

@@ -0,0 +1,309 @@
+# Ollama Docker Setup
+
+This directory contains the Docker configuration for running [Ollama](https://ollama.com/), a lightweight and extensible framework for running large language models locally.
+
+## Overview
+
+Ollama allows you to run large language models like Llama 2, Code Llama, and other models locally with a simple API interface. This Docker setup provides an easy way to deploy and manage Ollama in a containerized environment.
+
+## Quick Start
+
+### Prerequisites
+
+- Docker and Docker Compose installed
+- At least 8GB of RAM (16GB+ recommended for larger models)
+- Optional: NVIDIA GPU with Docker GPU support for better performance
+
+### Running Ollama
+
+1. **Start the service:**
+   ```bash
+   docker-compose up -d
+   ```
+
+2. **Check if the service is running:**
+   ```bash
+   docker-compose ps
+   ```
+
+3. **View logs:**
+   ```bash
+   docker-compose logs -f ollama
+   ```
+
+## Using Ollama
+
+### Pulling Models
+
+Once Ollama is running, you can pull models using the following commands:
+
+```bash
+# Pull Llama 2 (7B parameters)
+docker exec ollama-server ollama pull llama2
+
+# Pull Code Llama for code generation
+docker exec ollama-server ollama pull codellama
+
+# Pull Mistral 7B
+docker exec ollama-server ollama pull mistral
+
+# Pull other available models
+docker exec ollama-server ollama pull llama2:13b
+docker exec ollama-server ollama pull llama2:70b
+```
+
+### List Available Models
+
+```bash
+docker exec ollama-server ollama list
+```
+
+### Running Models
+
+```bash
+# Run interactive chat with a model
+docker exec -it ollama-server ollama run llama2
+
+# Run with specific prompt
+docker exec ollama-server ollama run llama2 "Explain quantum computing"
+```
+
+### API Usage
+
+Ollama provides a REST API accessible at `http://localhost:11434`. Here are some examples:
+
+#### Generate Text
+
+```bash
+curl http://localhost:11434/api/generate -d '{
+  "model": "llama2",
+  "prompt": "Why is the sky blue?",
+  "stream": false
+}'
+```
+
+#### Chat API
+
+```bash
+curl http://localhost:11434/api/chat -d '{
+  "model": "llama2",
+  "messages": [
+    {
+      "role": "user",
+      "content": "Why is the sky blue?"
+    }
+  ],
+  "stream": false
+}'
+```
+
+#### List Models via API
+
+```bash
+curl http://localhost:11434/api/tags
+```
+
+## Configuration
+
+### Environment Variables
+
+- `OLLAMA_HOST`: Host binding (default: `0.0.0.0`)
+- `OLLAMA_PORT`: Port number (default: `11434`)
+
+### Volumes
+
+- `./models:/models`: Local directory for storing model files
+- `ollama_data:/usr/share/ollama/.ollama`: Persistent storage for Ollama data and models
+
+### GPU Support
+
+To enable GPU acceleration (NVIDIA only), uncomment the GPU configuration in `docker-compose.yml`:
+
+```yaml
+deploy:
+  resources:
+    reservations:
+      devices:
+        - driver: nvidia
+          count: all
+          capabilities: [gpu]
+```
+
+**Prerequisites for GPU support:**
+- NVIDIA GPU with CUDA support
+- NVIDIA Container Toolkit installed
+- Docker configured for GPU access
+
+## Model Management
+
+### Popular Models
+
+| Model | Size | Use Case | Pull Command |
+|-------|------|----------|--------------|
+| Llama 2 | 7B | General purpose | `ollama pull llama2` |
+| Llama 2 | 13B | Better quality | `ollama pull llama2:13b` |
+| Code Llama | 7B | Code generation | `ollama pull codellama` |
+| Mistral | 7B | Fast inference | `ollama pull mistral` |
+| Phi-2 | 2.7B | Lightweight | `ollama pull phi` |
+
+### Custom Models
+
+You can also create custom models using a Modelfile:
+
+1. Create a `Modelfile`:
+   ```
+   FROM llama2
+   PARAMETER temperature 0.8
+   PARAMETER top_p 0.9
+   ```
+
+2. Create the model:
+   ```bash
+   docker exec ollama-server ollama create mymodel -f Modelfile
+   ```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Service won't start:**
+   - Check Docker logs: `docker-compose logs ollama`
+   - Ensure port 11434 is available
+   - Verify sufficient system resources
+
+2. **Models fail to download:**
+   - Check internet connectivity
+   - Ensure sufficient disk space
+   - Try smaller models first
+
+3. **Performance issues:**
+   - Consider enabling GPU support
+   - Increase Docker memory limits
+   - Use smaller models for testing
+
+### Health Checks
+
+The container includes a health check that verifies the API is responding:
+
+```bash
+# Check container health
+docker inspect ollama-server --format='{{.State.Health.Status}}'
+```
+
+### Resource Usage
+
+Monitor resource usage:
+
+```bash
+# Container stats
+docker stats ollama-server
+
+# Disk usage
+docker exec ollama-server du -sh /usr/share/ollama/.ollama
+```
+
+## Integration Examples
+
+### Python Integration
+
+```python
+import requests
+import json
+
+def chat_with_ollama(message, model="llama2"):
+    url = "http://localhost:11434/api/chat"
+    data = {
+        "model": model,
+        "messages": [{"role": "user", "content": message}],
+        "stream": False
+    }
+    
+    response = requests.post(url, json=data)
+    return response.json()["message"]["content"]
+
+# Example usage
+response = chat_with_ollama("Explain machine learning")
+print(response)
+```
+
+### JavaScript Integration
+
+```javascript
+async function chatWithOllama(message, model = "llama2") {
+    const response = await fetch("http://localhost:11434/api/chat", {
+        method: "POST",
+        headers: {
+            "Content-Type": "application/json",
+        },
+        body: JSON.stringify({
+            model: model,
+            messages: [{ role: "user", content: message }],
+            stream: false,
+        }),
+    });
+    
+    const data = await response.json();
+    return data.message.content;
+}
+
+// Example usage
+chatWithOllama("What is the capital of France?").then(console.log);
+```
+
+## Security Considerations
+
+- Ollama runs as a non-root user inside the container
+- API is bound to all interfaces (0.0.0.0) for container access
+- Consider using a reverse proxy (nginx, traefik) for production deployments
+- Implement authentication/authorization for production use
+
+## Performance Optimization
+
+### Memory Management
+
+- Allocate sufficient RAM based on model size:
+  - 7B models: 8GB+ RAM
+  - 13B models: 16GB+ RAM
+  - 70B models: 64GB+ RAM
+
+### GPU Acceleration
+
+- NVIDIA GPUs significantly improve inference speed
+- Ensure GPU memory is sufficient for the model
+- Use mixed precision for better GPU utilization
+
+## Maintenance
+
+### Updates
+
+Update Ollama to the latest version:
+
+```bash
+# Rebuild with latest Ollama
+docker-compose down
+docker-compose build --no-cache
+docker-compose up -d
+```
+
+### Cleanup
+
+Remove unused models and data:
+
+```bash
+# Remove specific model
+docker exec ollama-server ollama rm model_name
+
+# Clean up unused Docker resources
+docker system prune -f
+```
+
+## Links
+
+- [Ollama Official Website](https://ollama.com/)
+- [Ollama GitHub Repository](https://github.com/ollama/ollama)
+- [Available Models](https://ollama.com/library)
+- [API Documentation](https://github.com/ollama/ollama/blob/main/docs/api.md)
+
+## Support
+
+For issues specific to this Docker setup, check the container logs and ensure all prerequisites are met. For Ollama-specific issues, refer to the official Ollama documentation and GitHub issues.

ai_stack/ollama/docker-compose.yml

@@ -0,0 +1,24 @@
+version: '3.8'
+services:
+  ollama:
+    build: .
+    ports:
+      - "11434:11434" # Map host port 11434 to container port 11434 (Ollama default)
+    volumes:
+      - ./models:/models # Mount a local directory for models
+      - ollama_data:/usr/share/ollama/.ollama # Persistent storage for Ollama data
+    container_name: ollama-server
+    environment:
+      - OLLAMA_HOST=0.0.0.0
+    restart: unless-stopped
+    # Uncomment the following lines if you have NVIDIA GPU support
+    # deploy:
+    #   resources:
+    #     reservations:
+    #       devices:
+    #         - driver: nvidia
+    #           count: all
+    #           capabilities: [gpu]
+
+volumes:
+  ollama_data:

makefile

@@ -12,7 +12,7 @@ include $(ENV_FILE)
 export
 
 # Targets
-.PHONY: up down restart logs build reset network
+.PHONY: up down restart logs build reset network pull-deepseek-model ollama-up ollama-down ollama-logs llamacpp-up llamacpp-down llamacpp-logs pull-deepseek-llamacpp llamacpp-gpu ai-launcher
 
 network:
 	@podman network exists traefik-public || podman network create traefik-public
@@ -80,19 +80,42 @@ help:
 	@echo "Makefile for managing Podman Compose"
 	@echo ""
 	@echo "Usage:"
-	@echo "  make network    Create the traefik-public network if it doesn't exist"
-	@echo "  make up         Start the containers in detached mode"
-	@echo "  make down       Stop the containers"
-	@echo "  make logs       View the logs of the containers"
-	@echo "  make build      Build the containers"
-	@echo "  make restart    Restart the containers"
-	@echo "  make reset      Completely reset Podman (containers, volumes, networks)"
-	@echo "  make backup     Create a backup of the database volume"
-	@echo "  make restore    Restore the database volume from a backup"
-	@echo "  make clean      Remove all containers and volumes"
-	@echo "  make prune      Remove all stopped containers and unused volumes"
-	@echo "  make info       Display detailed information about Podman state"
-	@echo "  make help       View this help message"
+	@echo "  make network              Create the traefik-public network if it doesn't exist"
+	@echo "  make up                   Start the containers in detached mode"
+	@echo "  make down                 Stop the containers"
+	@echo "  make logs                 View the logs of the containers"
+	@echo "  make build                Build the containers"
+	@echo "  make restart              Restart the containers"
+	@echo "  make reset                Completely reset Podman (containers, volumes, networks)"
+	@echo "  make backup               Create a backup of the database volume"
+	@echo "  make restore              Restore the database volume from a backup"
+	@echo "  make clean                Remove all containers and volumes"
+	@echo "  make prune                Remove all stopped containers and unused volumes"
+	@echo "  make info                 Display detailed information about Podman state"
+	@echo "  make help                 View this help message"
+	@echo "  make ai-launcher          Interactive AI backend launcher (recommended for new users)"
+	@echo ""
+	@echo "Ollama-specific targets:"
+	@echo "  make ollama-up            Start only the Ollama container"
+	@echo "  make ollama-down          Stop only the Ollama container"
+	@echo "  make ollama-logs          View Ollama container logs"
+	@echo "  make pull-deepseek-model  Download and load DeepSeek-R1 model into Ollama"
+	@echo "  make setup-ollama         Start Ollama and automatically pull DeepSeek model"
+	@echo ""
+	@echo "LlamaCPP-specific targets:"
+	@echo "  make llamacpp-up          Start only the LlamaCPP container"
+	@echo "  make llamacpp-down        Stop only the LlamaCPP container"
+	@echo "  make llamacpp-logs        View LlamaCPP container logs"
+	@echo "  make pull-deepseek-llamacpp  Download DeepSeek-R1 model for LlamaCPP"
+	@echo "  make setup-llamacpp       Start LlamaCPP and automatically pull DeepSeek model"
+	@echo "  make llamacpp-gpu         Start GPU-accelerated LlamaCPP with DeepSeek model"
+	@echo ""
+	@echo "LlamaCPP-specific targets:"
+	@echo "  make llamacpp-up          Start only the LlamaCPP container"
+	@echo "  make llamacpp-down        Stop only the LlamaCPP container"
+	@echo "  make llamacpp-logs        View LlamaCPP container logs"
+	@echo "  make pull-deepseek-llamacpp  Download and load DeepSeek-R1 model into LlamaCPP"
+	@echo "  make setup-llamacpp       Start LlamaCPP and automatically pull DeepSeek model"
 	@echo ""
 
 info:
@@ -108,3 +131,58 @@ info:
 	podman images
 	@echo "\n===== Podman Compose Config ====="
 	podman compose --env-file $(ENV_FILE) -f docker-compose.yml -f docker-compose.override.yml config
+
+# Ollama-specific targets
+ollama-up:
+	cd ai_stack/ollama && podman compose up -d
+	@echo "Ollama container started. Use 'make pull-deepseek-model' to download and load the DeepSeek model."
+
+ollama-down:
+	cd ai_stack/ollama && podman compose down
+	@echo "Ollama container stopped."
+
+ollama-logs:
+	cd ai_stack/ollama && podman compose logs -f
+
+pull-deepseek-model:
+	@echo "Pulling DeepSeek-R1 model from Hugging Face..."
+	./scripts/pull-deepseek-model.sh
+	@echo "Model setup completed. You can now use the model with Ollama."
+
+# Combined target to start Ollama and pull the model
+setup-ollama: ollama-up
+	@echo "Waiting for Ollama to be ready..."
+	@sleep 10
+	@$(MAKE) pull-deepseek-model
+
+# LlamaCPP-specific targets
+llamacpp-up:
+	cd ai_stack/llamacpp && podman compose up -d
+	@echo "LlamaCPP container started. Use 'make pull-deepseek-llamacpp' to download the DeepSeek model."
+
+llamacpp-down:
+	cd ai_stack/llamacpp && podman compose down
+	@echo "LlamaCPP container stopped."
+
+llamacpp-logs:
+	cd ai_stack/llamacpp && podman compose logs -f
+
+pull-deepseek-llamacpp:
+	@echo "Pulling DeepSeek-R1 model from Hugging Face for LlamaCPP..."
+	./scripts/pull-deepseek-llamacpp.sh
+	@echo "Model setup completed. You can now use the model with LlamaCPP."
+
+# Combined target to start LlamaCPP and pull the model
+setup-llamacpp: llamacpp-up
+	@echo "Waiting for LlamaCPP to be ready..."
+	@sleep 15
+	@$(MAKE) pull-deepseek-llamacpp
+
+# GPU-accelerated LlamaCPP
+llamacpp-gpu:
+	@echo "Starting GPU-accelerated LlamaCPP with DeepSeek model..."
+	./scripts/llamacpp-gpu.sh
+
+# Interactive AI backend launcher
+ai-launcher:
+	@./scripts/ai-launcher.sh

notebooks/test.ipynb

@@ -0,0 +1,25 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d4ed83af",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.12.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

scripts/README-models.md

@@ -0,0 +1,115 @@
+# Model Setup Scripts
+
+This directory contains scripts to download and set up AI models for the Ollama service.
+
+## Available Scripts
+
+### 1. `pull-deepseek-model.sh`
+Main script that downloads the DeepSeek-R1-0528-Qwen3-8B model in GGUF format from Hugging Face and loads it into Ollama.
+
+**Features:**
+- Downloads the Q4_K_XL quantized version (optimal balance of quality and size)
+- Creates a proper Modelfile with appropriate templates and parameters
+- Loads the model into Ollama with the name `deepseek-r1-qwen3-8b`
+- Includes verification and testing steps
+
+**Requirements:**
+- `huggingface-hub` Python package (will be installed automatically if missing)
+- Ollama container running
+- Internet connection
+
+**Usage:**
+```bash
+# Using Makefile (recommended)
+make setup-ollama          # Start Ollama and pull model automatically
+make pull-deepseek-model   # Pull model only (requires Ollama to be running)
+
+# Or run directly
+./scripts/pull-deepseek-model.sh
+```
+
+### 2. `pull-model-simple.sh`
+Alternative script that attempts to pull models using `ollama pull` command.
+
+**Usage:**
+```bash
+./scripts/pull-model-simple.sh
+```
+
+## Makefile Targets
+
+The main Makefile includes these Ollama-related targets:
+
+- `make ollama-up` - Start only the Ollama container
+- `make ollama-down` - Stop only the Ollama container  
+- `make ollama-logs` - View Ollama container logs
+- `make pull-deepseek-model` - Download and load DeepSeek model
+- `make setup-ollama` - Complete setup (start Ollama + pull model)
+
+## Model Information
+
+**DeepSeek-R1-0528-Qwen3-8B-GGUF**
+- **Source:** [unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF)
+- **Quantization:** Q4_K_XL (4-bit quantization, extra large)
+- **Size:** ~5.5GB
+- **Context Length:** 32,768 tokens
+- **Use Case:** General conversation, reasoning, coding assistance
+
+## Quick Start
+
+1. **Start the entire stack:**
+   ```bash
+   make up
+   ```
+
+2. **Or start just Ollama and set up the model:**
+   ```bash
+   make setup-ollama
+   ```
+
+3. **Use the model:**
+   ```bash
+   # Via Docker
+   docker exec ollama-server ollama run deepseek-r1-qwen3-8b "Hello, how are you?"
+   
+   # Via API
+   curl http://localhost:11434/api/generate -d '{
+     "model": "deepseek-r1-qwen3-8b",
+     "prompt": "Hello, how are you?",
+     "stream": false
+   }'
+   ```
+
+## Troubleshooting
+
+1. **Script fails with "Ollama not running":**
+   - Make sure Ollama container is started: `make ollama-up`
+   - Wait a few seconds for the service to be ready
+
+2. **Download fails:**
+   - Check internet connection
+   - Verify Hugging Face Hub access
+   - Try installing huggingface-hub manually: `pip install huggingface-hub`
+
+3. **Model loading fails:**
+   - Check if the GGUF file was downloaded completely
+   - Verify the Modelfile syntax
+   - Check Ollama container logs: `make ollama-logs`
+
+4. **Out of disk space:**
+   - The model file is ~5.5GB, ensure sufficient free space
+   - Consider using a smaller quantization (modify the script)
+
+## Customization
+
+You can modify the scripts to download different models or quantizations:
+
+1. **Change model variant:** Edit `MODEL_FILE` in `pull-deepseek-model.sh`
+2. **Adjust parameters:** Modify the Modelfile template in the script
+3. **Use different model:** Change `MODEL_REPO` to any GGUF model on Hugging Face
+
+## File Locations
+
+- **Models:** `./ai_stack/ollama/models/`
+- **Modelfiles:** `./ai_stack/ollama/models/Modelfile.*`
+- **Ollama data:** Docker volume `ollama_data`

scripts/ai-launcher.sh

@@ -0,0 +1,244 @@
+#!/usr/bin/env bash
+
+# AI Stack Launcher - Choose your AI backend
+# This script helps users choose between Ollama and LlamaCPP for DeepSeek-R1
+
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+PURPLE='\033[0;35m'
+CYAN='\033[0;36m'
+NC='\033[0m' # No Color
+
+# Print colored output
+print_header() {
+    echo -e "${CYAN}================================${NC}"
+    echo -e "${CYAN}     AI Stack DeepSeek-R1       ${NC}"
+    echo -e "${CYAN}================================${NC}"
+    echo
+}
+
+print_info() {
+    echo -e "${GREEN}ℹ️  $1${NC}"
+}
+
+print_warn() {
+    echo -e "${YELLOW}⚠️  $1${NC}"
+}
+
+print_error() {
+    echo -e "${RED}❌ $1${NC}"
+}
+
+print_option() {
+    echo -e "${BLUE}$1${NC} $2"
+}
+
+# Check if GPU is available
+check_gpu() {
+    if command -v nvidia-smi &> /dev/null && nvidia-smi &> /dev/null; then
+        return 0
+    else
+        return 1
+    fi
+}
+
+# Display system information
+show_system_info() {
+    echo -e "${PURPLE}System Information:${NC}"
+    echo "  CPU: $(nproc) cores"
+    echo "  RAM: $(free -h | awk '/^Mem:/ {print $2}')"
+    
+    if check_gpu; then
+        echo "  GPU: $(nvidia-smi --query-gpu=name --format=csv,noheader | head -1)"
+        echo "  VRAM: $(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits | head -1) MB"
+    else
+        echo "  GPU: Not available"
+    fi
+    echo
+}
+
+# Show backend options
+show_options() {
+    echo -e "${PURPLE}Available Backends:${NC}"
+    echo
+    
+    print_option "1)" "Ollama (Recommended for beginners)"
+    echo "     • Easy to use and manage"
+    echo "     • Web UI available"
+    echo "     • Good performance"
+    echo "     • Port: 11434"
+    echo
+    
+    print_option "2)" "LlamaCPP (CPU-only)"
+    echo "     • Direct GGUF model loading"
+    echo "     • Lightweight and fast"
+    echo "     • OpenAI-compatible API"
+    echo "     • Port: 8080"
+    echo
+    
+    if check_gpu; then
+        print_option "3)" "LlamaCPP (GPU-accelerated)"
+        echo "     • CUDA-optimized performance"
+        echo "     • Best speed and efficiency"
+        echo "     • Requires NVIDIA GPU"
+        echo "     • Port: 8080"
+        echo
+    fi
+    
+    print_option "0)" "Exit"
+    echo
+}
+
+# Launch Ollama
+launch_ollama() {
+    print_info "Launching Ollama with DeepSeek-R1..."
+    echo
+    
+    print_info "Starting Ollama container..."
+    make ollama-up
+    
+    echo
+    print_info "Downloading and loading DeepSeek-R1 model..."
+    make pull-deepseek-model
+    
+    echo
+    print_info "✅ Ollama setup complete!"
+    echo "   🌐 Access at: http://localhost:11434"
+    echo "   📚 Model: deepseek-r1-qwen3-8b"
+    echo "   💬 Chat: docker exec ollama-server ollama run deepseek-r1-qwen3-8b"
+}
+
+# Launch LlamaCPP CPU
+launch_llamacpp_cpu() {
+    print_info "Launching LlamaCPP (CPU) with DeepSeek-R1..."
+    echo
+    
+    print_info "Starting LlamaCPP container..."
+    make llamacpp-up
+    
+    echo
+    print_info "Downloading DeepSeek-R1 model..."
+    make pull-deepseek-llamacpp
+    
+    echo
+    print_info "✅ LlamaCPP (CPU) setup complete!"
+    echo "   🌐 Web UI: http://localhost:8080"
+    echo "   📡 API: http://localhost:8080/completion"
+    echo "   📊 Health: http://localhost:8080/health"
+}
+
+# Launch LlamaCPP GPU
+launch_llamacpp_gpu() {
+    print_info "Launching LlamaCPP (GPU) with DeepSeek-R1..."
+    echo
+    
+    print_info "Downloading model if needed..."
+    make pull-deepseek-llamacpp
+    
+    echo
+    print_info "Starting GPU-accelerated LlamaCPP..."
+    make llamacpp-gpu
+    
+    echo
+    print_info "✅ LlamaCPP (GPU) setup complete!"
+    echo "   🌐 Web UI: http://localhost:8080"
+    echo "   📡 API: http://localhost:8080/completion"
+    echo "   📊 Health: http://localhost:8080/health"
+    echo "   🚀 GPU-accelerated for best performance!"
+}
+
+# Test the selected backend
+test_backend() {
+    local backend=$1
+    
+    echo
+    print_info "Would you like to test the ${backend} backend? (y/N)"
+    read -r response
+    
+    if [[ $response =~ ^[Yy]$ ]]; then
+        echo
+        print_info "Testing ${backend}..."
+        
+        case $backend in
+            "Ollama")
+                if docker exec ollama-server ollama run deepseek-r1-qwen3-8b "Hello, introduce yourself briefly" 2>/dev/null; then
+                    print_info "✅ Ollama test successful!"
+                else
+                    print_warn "⚠️  Test failed, but server should be running"
+                fi
+                ;;
+            "LlamaCPP")
+                sleep 5  # Give server time to start
+                if curl -s -X POST http://localhost:8080/completion \
+                    -H "Content-Type: application/json" \
+                    -d '{"prompt": "Hello!", "max_tokens": 20}' | grep -q "content"; then
+                    print_info "✅ LlamaCPP test successful!"
+                else
+                    print_warn "⚠️  Test failed, server might still be starting up"
+                fi
+                ;;
+        esac
+    fi
+}
+
+# Show management commands
+show_management() {
+    echo
+    echo -e "${PURPLE}Management Commands:${NC}"
+    echo "  make help                 # Show all available commands"
+    echo "  make ollama-logs          # View Ollama logs"
+    echo "  make llamacpp-logs        # View LlamaCPP logs"
+    echo "  make ollama-down          # Stop Ollama"
+    echo "  make llamacpp-down        # Stop LlamaCPP"
+    echo "  docker ps                 # See running containers"
+    echo
+}
+
+# Main menu
+main_menu() {
+    print_header
+    show_system_info
+    show_options
+    
+    echo -n "Choose your AI backend (1-3, 0 to exit): "
+    read -r choice
+    
+    case $choice in
+        1)
+            launch_ollama
+            test_backend "Ollama"
+            ;;
+        2)
+            launch_llamacpp_cpu
+            test_backend "LlamaCPP"
+            ;;
+        3)
+            if check_gpu; then
+                launch_llamacpp_gpu
+                test_backend "LlamaCPP GPU"
+            else
+                print_error "GPU not available! Please choose option 1 or 2."
+                return 1
+            fi
+            ;;
+        0)
+            print_info "Goodbye!"
+            exit 0
+            ;;
+        *)
+            print_error "Invalid choice. Please try again."
+            return 1
+            ;;
+    esac
+    
+    show_management
+    print_info "🎉 Setup complete! Your AI backend is ready to use."
+}
+
+# Run main menu
+main_menu

scripts/build-llamacpp.sh

@@ -0,0 +1,2 @@
+podman build -t furyhawk/llama.cpp:full-cuda --target full -f .devops/cuda.Dockerfile .
+podman run --name llamacpp-server --gpus all -v ~/models:/models localhost/furyhawk/llama.cpp:full-cuda -b -m /models/DeepSeek-R1-0528-Qwen3-8B-UD-Q4_K_XL.gguf --port 8000 --host 0.0.0.0 -n 512 --n-gpu-layers 1

scripts/llamacpp-gpu.sh

@@ -0,0 +1,224 @@
+#!/usr/bin/env bash
+
+# Script to start LlamaCPP with GPU support for DeepSeek-R1 model
+# This script enables CUDA/GPU acceleration for better performance
+
+set -e
+
+# Configuration
+MODEL_FILE="DeepSeek-R1-0528-Qwen3-8B-Q4_K_XL.gguf"
+MODELS_DIR="./ai_stack/llamacpp/models"
+CONTAINER_NAME="llama-cpp-server-gpu"
+HOST_PORT="8080"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log_info() {
+    echo -e "${GREEN}[INFO]${NC} $1"
+}
+
+log_warn() {
+    echo -e "${YELLOW}[WARN]${NC} $1"
+}
+
+log_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# Check if NVIDIA GPU is available
+check_gpu() {
+    log_info "Checking for NVIDIA GPU..."
+    
+    if command -v nvidia-smi &> /dev/null; then
+        if nvidia-smi &> /dev/null; then
+            log_info "✅ NVIDIA GPU detected:"
+            nvidia-smi --query-gpu=name,memory.total --format=csv,noheader,nounits
+            return 0
+        else
+            log_warn "nvidia-smi found but failed to query GPU"
+            return 1
+        fi
+    else
+        log_warn "nvidia-smi not found - no NVIDIA GPU support"
+        return 1
+    fi
+}
+
+# Check if model exists
+check_model() {
+    local model_path="${MODELS_DIR}/${MODEL_FILE}"
+    
+    if [ ! -f "${model_path}" ]; then
+        log_error "Model file not found at ${model_path}"
+        log_error "Please run: make pull-deepseek-llamacpp"
+        exit 1
+    fi
+    
+    log_info "✅ Model file found: ${model_path}"
+}
+
+# Stop existing container if running
+stop_existing() {
+    if docker ps -q -f name="${CONTAINER_NAME}" | grep -q .; then
+        log_info "Stopping existing container: ${CONTAINER_NAME}"
+        docker stop "${CONTAINER_NAME}"
+        docker rm "${CONTAINER_NAME}"
+    fi
+}
+
+# Start LlamaCPP with GPU support
+start_gpu_server() {
+    log_info "Starting LlamaCPP server with GPU acceleration..."
+    
+    # Create absolute path for models directory
+    local abs_models_dir=$(realpath "${MODELS_DIR}")
+    
+    docker run -d \
+        --name "${CONTAINER_NAME}" \
+        --gpus all \
+        -p "${HOST_PORT}:8080" \
+        -v "${abs_models_dir}:/models" \
+        -e CUDA_VISIBLE_DEVICES=0 \
+        llama-cpp-cuda:latest \
+        /llama.cpp/llama-server \
+        --model "/models/${MODEL_FILE}" \
+        --host "0.0.0.0" \
+        --port "8080" \
+        --ctx-size "32768" \
+        --n-predict "2048" \
+        --threads "4" \
+        --batch-size "512" \
+        --n-gpu-layers "35" \
+        --cont-batching \
+        --parallel "4" \
+        --flash-attn \
+        --temperature "0.7" \
+        --top-p "0.9" \
+        --top-k "40" \
+        --repeat-penalty "1.1"
+    
+    log_info "LlamaCPP GPU server started on port ${HOST_PORT}"
+}
+
+# Build GPU-enabled image if needed
+build_gpu_image() {
+    log_info "Checking for GPU-enabled LlamaCPP image..."
+    
+    if ! docker images | grep -q "llama-cpp-cuda"; then
+        log_info "Building GPU-enabled LlamaCPP image..."
+        
+        # Create temporary Dockerfile for GPU build
+        cat > /tmp/Dockerfile.llamacpp-gpu << 'EOF'
+FROM nvidia/cuda:12.1-devel-ubuntu22.04
+
+# Install dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    cmake \
+    g++ \
+    wget \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Clone llama.cpp
+RUN git clone https://github.com/ggerganov/llama.cpp.git
+
+# Build llama.cpp with CUDA support
+WORKDIR /llama.cpp
+RUN mkdir build && cd build && \
+    cmake .. -DLLAMA_SERVER=ON -DLLAMA_CUDA=ON && \
+    make -j$(nproc)
+
+# Create models directory
+RUN mkdir -p /models
+
+# Expose the server port
+EXPOSE 8080
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8080/health || exit 1
+
+# Default command
+CMD ["/llama.cpp/llama-server", "--host", "0.0.0.0", "--port", "8080"]
+EOF
+
+        docker build -f /tmp/Dockerfile.llamacpp-gpu -t llama-cpp-cuda:latest .
+        rm /tmp/Dockerfile.llamacpp-gpu
+        
+        log_info "✅ GPU-enabled image built successfully"
+    else
+        log_info "✅ GPU-enabled image already exists"
+    fi
+}
+
+# Wait for server to be ready
+wait_for_server() {
+    log_info "Waiting for server to be ready..."
+    local max_attempts=30
+    local attempt=1
+    
+    while [ $attempt -le $max_attempts ]; do
+        if curl -s -f "http://localhost:${HOST_PORT}/health" >/dev/null 2>&1; then
+            log_info "✅ Server is ready!"
+            return 0
+        fi
+        
+        echo -n "."
+        sleep 2
+        ((attempt++))
+    done
+    
+    log_error "❌ Server failed to start within expected time"
+    return 1
+}
+
+# Show usage information
+show_usage() {
+    log_info "GPU-accelerated LlamaCPP server is running!"
+    echo
+    echo "🌐 Web Interface: http://localhost:${HOST_PORT}"
+    echo "📡 API Endpoint: http://localhost:${HOST_PORT}/completion"
+    echo "🔍 Health Check: http://localhost:${HOST_PORT}/health"
+    echo "📊 Metrics: http://localhost:${HOST_PORT}/metrics"
+    echo
+    echo "Container: ${CONTAINER_NAME}"
+    echo "GPU Layers: 35 (adjust based on your GPU memory)"
+    echo
+    echo "Management commands:"
+    echo "  docker logs ${CONTAINER_NAME}     # View logs"
+    echo "  docker stop ${CONTAINER_NAME}     # Stop server"
+    echo "  docker stats ${CONTAINER_NAME}    # View resource usage"
+}
+
+# Main execution
+main() {
+    log_info "Starting GPU-accelerated LlamaCPP setup for DeepSeek-R1..."
+    
+    # Check prerequisites
+    if ! check_gpu; then
+        log_error "No GPU support detected. Use regular LlamaCPP setup instead."
+        log_error "Run: make llamacpp-up"
+        exit 1
+    fi
+    
+    check_model
+    stop_existing
+    build_gpu_image
+    start_gpu_server
+    
+    if wait_for_server; then
+        show_usage
+        log_info "✅ GPU-accelerated DeepSeek-R1 setup completed successfully!"
+    else
+        log_error "❌ Server setup failed. Check logs: docker logs ${CONTAINER_NAME}"
+        exit 1
+    fi
+}
+
+# Run main function
+main "$@"

scripts/pull-deepseek-llamacpp.sh

@@ -0,0 +1,172 @@
+#!/usr/bin/env bash
+
+# Script to pull DeepSeek-R1 model from Hugging Face for LlamaCPP
+# This script downloads the model and prepares it for LlamaCPP server
+
+set -e
+
+# Configuration
+MODEL_REPO="unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF"
+MODEL_FILE="DeepSeek-R1-0528-Qwen3-8B-Q4_K_XL.gguf"
+MODELS_DIR="./ai_stack/llamacpp/models"
+LLAMACPP_HOST="http://localhost:8080"
+CONTAINER_NAME="llama-cpp-server"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log_info() {
+    echo -e "${GREEN}[INFO]${NC} $1"
+}
+
+log_warn() {
+    echo -e "${YELLOW}[WARN]${NC} $1"
+}
+
+log_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# Create models directory if it doesn't exist
+create_models_dir() {
+    if [ ! -d "${MODELS_DIR}" ]; then
+        log_info "Creating models directory: ${MODELS_DIR}"
+        mkdir -p "${MODELS_DIR}"
+    fi
+}
+
+# Download model from Hugging Face
+download_model() {
+    log_info "Downloading model from Hugging Face..."
+    
+    # Check if huggingface-hub is installed
+    if ! command -v huggingface-cli &> /dev/null; then
+        log_warn "huggingface-cli not found. Installing huggingface-hub..."
+        pip install huggingface-hub
+    fi
+    
+    # Download the specific GGUF file
+    log_info "Downloading ${MODEL_FILE} from ${MODEL_REPO}..."
+    huggingface-cli download \
+        "${MODEL_REPO}" \
+        "${MODEL_FILE}" \
+        --local-dir "${MODELS_DIR}" \
+        --local-dir-use-symlinks False
+    
+    log_info "Model downloaded to ${MODELS_DIR}/${MODEL_FILE}"
+}
+
+# Verify model file exists and get info
+verify_model() {
+    local model_path="${MODELS_DIR}/${MODEL_FILE}"
+    
+    if [ ! -f "${model_path}" ]; then
+        log_error "Model file not found at ${model_path}"
+        exit 1
+    fi
+    
+    local file_size=$(du -h "${model_path}" | cut -f1)
+    log_info "✅ Model file verified: ${model_path} (${file_size})"
+}
+
+# Check if LlamaCPP container is running
+check_llamacpp() {
+    log_info "Checking if LlamaCPP container is running..."
+    if docker ps | grep -q "${CONTAINER_NAME}"; then
+        log_info "LlamaCPP container is running"
+        
+        # Test if server is responding
+        if curl -f "${LLAMACPP_HOST}/health" >/dev/null 2>&1; then
+            log_info "LlamaCPP server is responding"
+            return 0
+        else
+            log_warn "LlamaCPP container is running but server is not responding"
+            return 1
+        fi
+    else
+        log_warn "LlamaCPP container is not running"
+        return 1
+    fi
+}
+
+# Test the model with a simple prompt
+test_model() {
+    log_info "Testing model with a simple prompt..."
+    
+    local prompt="Hello, can you introduce yourself?"
+    log_info "Sending test prompt: '${prompt}'"
+    
+    # Test via API
+    curl -s -X POST "${LLAMACPP_HOST}/completion" \
+        -H "Content-Type: application/json" \
+        -d "{
+            \"prompt\": \"${prompt}\",
+            \"max_tokens\": 100,
+            \"temperature\": 0.7,
+            \"stop\": [\"\\n\\n\"]
+        }" | jq -r '.content' 2>/dev/null || {
+        log_warn "Model test failed or jq not available. Server might still be starting up."
+        log_info "You can test manually at: ${LLAMACPP_HOST}"
+    }
+}
+
+# Display usage information
+show_usage() {
+    log_info "Model setup completed! Here's how to use it:"
+    echo
+    echo "🌐 Web Interface: ${LLAMACPP_HOST}"
+    echo "📡 API Endpoint: ${LLAMACPP_HOST}/completion"
+    echo
+    echo "Example API call:"
+    echo "curl -X POST ${LLAMACPP_HOST}/completion \\"
+    echo "  -H 'Content-Type: application/json' \\"
+    echo "  -d '{\"prompt\": \"Hello!\", \"max_tokens\": 50}'"
+    echo
+    echo "Container management:"
+    echo "  Start: make llamacpp-up"
+    echo "  Stop:  make llamacpp-down"
+    echo "  Logs:  make llamacpp-logs"
+}
+
+# Main execution
+main() {
+    log_info "Starting DeepSeek-R1 model setup for LlamaCPP..."
+    
+    create_models_dir
+    
+    # Check if model file already exists
+    if [ -f "${MODELS_DIR}/${MODEL_FILE}" ]; then
+        log_warn "Model file already exists at ${MODELS_DIR}/${MODEL_FILE}"
+        read -p "Do you want to re-download it? (y/N): " -n 1 -r
+        echo
+        if [[ $REPLY =~ ^[Yy]$ ]]; then
+            download_model
+        else
+            log_info "Skipping download, using existing model file"
+        fi
+    else
+        download_model
+    fi
+    
+    verify_model
+    
+    # Check if container is running and offer to test
+    if check_llamacpp; then
+        read -p "LlamaCPP server is running. Do you want to test the model? (Y/n): " -n 1 -r
+        echo
+        if [[ ! $REPLY =~ ^[Nn]$ ]]; then
+            test_model
+        fi
+    else
+        log_info "LlamaCPP container is not running. Start it with: make llamacpp-up"
+    fi
+    
+    show_usage
+    log_info "✅ DeepSeek-R1 model setup completed successfully!"
+}
+
+# Run main function
+main "$@"

scripts/pull-deepseek-model.sh

@@ -0,0 +1,190 @@
+#!/usr/bin/env bash
+
+# Script to pull DeepSeek-R1 model from Hugging Face and load it into Ollama
+# This script should be run after Docker Compose is up and running
+
+set -e
+
+# Configuration
+MODEL_REPO="unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF"
+MODEL_FILE="DeepSeek-R1-0528-Qwen3-8B-Q4_K_XL.gguf"
+OLLAMA_HOST="http://localhost:11434"
+MODEL_NAME="deepseek-r1-qwen3-8b"
+MODELS_DIR="./ai_stack/ollama/models"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log_info() {
+    echo -e "${GREEN}[INFO]${NC} $1"
+}
+
+log_warn() {
+    echo -e "${YELLOW}[WARN]${NC} $1"
+}
+
+log_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# Check if Ollama is running
+check_ollama() {
+    log_info "Checking if Ollama is running..."
+    if ! curl -f "${OLLAMA_HOST}/api/version" >/dev/null 2>&1; then
+        log_error "Ollama is not running or not accessible at ${OLLAMA_HOST}"
+        log_error "Please make sure Docker Compose is running with: make up"
+        exit 1
+    fi
+    log_info "Ollama is running"
+}
+
+# Create models directory if it doesn't exist
+create_models_dir() {
+    if [ ! -d "${MODELS_DIR}" ]; then
+        log_info "Creating models directory: ${MODELS_DIR}"
+        mkdir -p "${MODELS_DIR}"
+    fi
+}
+
+# Download model from Hugging Face using huggingface-hub
+download_model() {
+    log_info "Downloading model from Hugging Face..."
+    
+    # Check if huggingface-hub is installed
+    if ! command -v huggingface-cli &> /dev/null; then
+        log_warn "huggingface-cli not found. Installing huggingface-hub..."
+        pip install huggingface-hub
+    fi
+    
+    # Download the specific GGUF file
+    log_info "Downloading ${MODEL_FILE} from ${MODEL_REPO}..."
+    huggingface-cli download \
+        "${MODEL_REPO}" \
+        "${MODEL_FILE}" \
+        --local-dir "${MODELS_DIR}" \
+        --local-dir-use-symlinks False
+    
+    log_info "Model downloaded to ${MODELS_DIR}/${MODEL_FILE}"
+}
+
+# Create Modelfile for Ollama
+create_modelfile() {
+    local modelfile_path="${MODELS_DIR}/Modelfile.${MODEL_NAME}"
+    
+    log_info "Creating Modelfile at ${modelfile_path}"
+    
+    cat > "${modelfile_path}" << EOF
+FROM ./${MODEL_FILE}
+
+TEMPLATE """{{ if .System }}<|im_start|>system
+{{ .System }}<|im_end|>
+{{ end }}{{ if .Prompt }}<|im_start|>user
+{{ .Prompt }}<|im_end|>
+{{ end }}<|im_start|>assistant
+{{ .Response }}<|im_end|>
+"""
+
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+PARAMETER num_ctx 32768
+PARAMETER num_predict 8192
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER top_k 40
+PARAMETER repeat_penalty 1.1
+
+SYSTEM """You are DeepSeek-R1, a helpful and harmless AI assistant developed by DeepSeek. You should think step-by-step and provide detailed, accurate responses."""
+EOF
+
+    log_info "Modelfile created successfully"
+}
+
+# Load model into Ollama
+load_model() {
+    log_info "Loading model into Ollama..."
+    
+    # Change to models directory for relative path in Modelfile
+    cd "${MODELS_DIR}"
+    
+    # Create the model in Ollama
+    docker exec ollama-server ollama create "${MODEL_NAME}" -f "Modelfile.${MODEL_NAME}"
+    
+    # Go back to original directory
+    cd - > /dev/null
+    
+    log_info "Model '${MODEL_NAME}' loaded successfully into Ollama"
+}
+
+# Verify model is loaded
+verify_model() {
+    log_info "Verifying model is available in Ollama..."
+    
+    if docker exec ollama-server ollama list | grep -q "${MODEL_NAME}"; then
+        log_info "✅ Model '${MODEL_NAME}' is available in Ollama"
+        
+        # Show model details
+        log_info "Model details:"
+        docker exec ollama-server ollama list | grep "${MODEL_NAME}" || true
+        
+        log_info "You can now use the model with: ollama run ${MODEL_NAME}"
+    else
+        log_error "❌ Model '${MODEL_NAME}' was not found in Ollama"
+        exit 1
+    fi
+}
+
+# Test the model with a simple prompt
+test_model() {
+    log_info "Testing model with a simple prompt..."
+    
+    echo "Testing model response:"
+    echo "Prompt: 'Hello, can you introduce yourself?'"
+    echo "Response:"
+    
+    docker exec ollama-server ollama run "${MODEL_NAME}" "Hello, can you introduce yourself?" || {
+        log_warn "Model test failed, but model is loaded. You can test it manually."
+    }
+}
+
+# Main execution
+main() {
+    log_info "Starting DeepSeek-R1 model setup for Ollama..."
+    
+    check_ollama
+    create_models_dir
+    
+    # Check if model file already exists
+    if [ -f "${MODELS_DIR}/${MODEL_FILE}" ]; then
+        log_warn "Model file already exists at ${MODELS_DIR}/${MODEL_FILE}"
+        read -p "Do you want to re-download it? (y/N): " -n 1 -r
+        echo
+        if [[ $REPLY =~ ^[Yy]$ ]]; then
+            download_model
+        else
+            log_info "Skipping download, using existing model file"
+        fi
+    else
+        download_model
+    fi
+    
+    create_modelfile
+    load_model
+    verify_model
+    
+    # Ask if user wants to test the model
+    read -p "Do you want to test the model? (Y/n): " -n 1 -r
+    echo
+    if [[ ! $REPLY =~ ^[Nn]$ ]]; then
+        test_model
+    fi
+    
+    log_info "✅ DeepSeek-R1 model setup completed successfully!"
+    log_info "You can now use the model with: docker exec ollama-server ollama run ${MODEL_NAME}"
+    log_info "Or via API at: ${OLLAMA_HOST}/api/generate"
+}
+
+# Run main function
+main "$@"

scripts/pull-model-simple.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+
+# Simple script to pull DeepSeek-R1 model using Ollama pull command
+# This is an alternative if the model becomes available in Ollama's registry
+
+set -e
+
+MODEL_NAME="deepseek-r1:8b-q4"
+OLLAMA_HOST="http://localhost:11434"
+CONTAINER_NAME="ollama-server"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+log_info() {
+    echo -e "${GREEN}[INFO]${NC} $1"
+}
+
+log_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# Check if Ollama container is running
+check_ollama() {
+    log_info "Checking if Ollama container is running..."
+    if ! docker ps | grep -q "${CONTAINER_NAME}"; then
+        log_error "Ollama container '${CONTAINER_NAME}' is not running"
+        log_error "Please start it with: make ollama-up"
+        exit 1
+    fi
+    log_info "Ollama container is running"
+}
+
+# Pull model using ollama pull
+pull_model() {
+    log_info "Attempting to pull model: ${MODEL_NAME}"
+    
+    # Try to pull the model - this will work if the model is in Ollama's registry
+    if docker exec "${CONTAINER_NAME}" ollama pull "${MODEL_NAME}"; then
+        log_info "✅ Model '${MODEL_NAME}' pulled successfully"
+    else
+        log_error "❌ Failed to pull model '${MODEL_NAME}'"
+        log_error "The model might not be available in Ollama's registry."
+        log_error "Use the pull-deepseek-model.sh script to download from Hugging Face instead."
+        exit 1
+    fi
+}
+
+# Verify model is available
+verify_model() {
+    log_info "Verifying model is available..."
+    
+    if docker exec "${CONTAINER_NAME}" ollama list | grep -q "${MODEL_NAME}"; then
+        log_info "✅ Model '${MODEL_NAME}' is available"
+        docker exec "${CONTAINER_NAME}" ollama list | grep "${MODEL_NAME}"
+    else
+        log_error "❌ Model verification failed"
+        exit 1
+    fi
+}
+
+main() {
+    log_info "Starting simple model pull for ${MODEL_NAME}..."
+    
+    check_ollama
+    pull_model
+    verify_model
+    
+    log_info "✅ Model setup completed!"
+    log_info "You can now use: docker exec ${CONTAINER_NAME} ollama run ${MODEL_NAME}"
+}
+
+main "$@"