home_stack/notebooks/llamacpp_rag.ipynb

In [1]:

!pip install llama-cpp-python
!pip install langchain langchain-community sentence-transformers chromadb
!pip install pypdf requests pydantic tqdm

Collecting llama-cpp-python
  Downloading llama_cpp_python-0.3.9.tar.gz (67.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.9/67.9 MB 52.3 MB/s eta 0:00:00 MB/s eta 0:00:01:01
[?25h  Installing build dependencies ... [?25ldone
[?25h  Getting requirements to build wheel ... [?25ldone
[?25h  Preparing metadata (pyproject.toml) ... [?25ldone
[?25hCollecting typing-extensions>=4.5.0 (from llama-cpp-python)
  Using cached typing_extensions-4.14.0-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from llama-cpp-python)
  Downloading numpy-2.3.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (62 kB)
Collecting diskcache>=5.6.1 (from llama-cpp-python)
  Using cached diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from llama-cpp-python)
  Using cached jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python)
  Using cached MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
Using cached jinja2-3.1.6-py3-none-any.whl (134 kB)
Downloading numpy-2.3.0-cp312-cp312-manylinux_2_28_x86_64.whl (16.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.6/16.6 MB 55.8 MB/s eta 0:00:0031m61.9 MB/s eta 0:00:01
[?25hDownloading typing_extensions-4.14.0-py3-none-any.whl (43 kB)
Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23 kB)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... [?25ldone
[?25h  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.3.9-cp312-cp312-linux_x86_64.whl size=4205333 sha256=68aac9f8fa6d6d82c6b489528cad19ff4f1e2a90522d62eb578d5f1f7a9dbfd6
  Stored in directory: /home/user/.cache/pip/wheels/e9/22/42/98dca29f6195951fae2aa548582827a45306350e282ab30617
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
Successfully installed MarkupSafe-3.0.2 diskcache-5.6.3 jinja2-3.1.6 llama-cpp-python-0.3.9 numpy-2.3.0 typing-extensions-4.14.0

[notice] A new release of pip is available: 24.3.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip
Collecting langchain
  Downloading langchain-0.3.25-py3-none-any.whl.metadata (7.8 kB)
Collecting langchain-community
  Downloading langchain_community-0.3.25-py3-none-any.whl.metadata (2.9 kB)
Collecting sentence-transformers
  Downloading sentence_transformers-4.1.0-py3-none-any.whl.metadata (13 kB)
Collecting chromadb
  Downloading chromadb-1.0.12-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.9 kB)
Collecting langchain-core<1.0.0,>=0.3.58 (from langchain)
  Downloading langchain_core-0.3.65-py3-none-any.whl.metadata (5.8 kB)
Collecting langchain-text-splitters<1.0.0,>=0.3.8 (from langchain)
  Downloading langchain_text_splitters-0.3.8-py3-none-any.whl.metadata (1.9 kB)
Collecting langsmith<0.4,>=0.1.17 (from langchain)
  Downloading langsmith-0.3.45-py3-none-any.whl.metadata (15 kB)
Collecting pydantic<3.0.0,>=2.7.4 (from langchain)
  Downloading pydantic-2.11.7-py3-none-any.whl.metadata (67 kB)
Collecting SQLAlchemy<3,>=1.4 (from langchain)
  Downloading sqlalchemy-2.0.41-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.6 kB)
Requirement already satisfied: requests<3,>=2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from langchain) (2.32.4)
Collecting PyYAML>=5.3 (from langchain)
  Using cached PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting aiohttp<4.0.0,>=3.8.3 (from langchain-community)
  Downloading aiohttp-3.12.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.6 kB)
Collecting tenacity!=8.4.0,<10,>=8.1.0 (from langchain-community)
  Downloading tenacity-9.1.2-py3-none-any.whl.metadata (1.2 kB)
Collecting dataclasses-json<0.7,>=0.5.7 (from langchain-community)
  Downloading dataclasses_json-0.6.7-py3-none-any.whl.metadata (25 kB)
Collecting pydantic-settings<3.0.0,>=2.4.0 (from langchain-community)
  Downloading pydantic_settings-2.9.1-py3-none-any.whl.metadata (3.8 kB)
Collecting httpx-sse<1.0.0,>=0.4.0 (from langchain-community)
  Downloading httpx_sse-0.4.0-py3-none-any.whl.metadata (9.0 kB)
Requirement already satisfied: numpy>=1.26.2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from langchain-community) (2.3.0)
Collecting transformers<5.0.0,>=4.41.0 (from sentence-transformers)
  Using cached transformers-4.52.4-py3-none-any.whl.metadata (38 kB)
Collecting tqdm (from sentence-transformers)
  Using cached tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
Collecting torch>=1.11.0 (from sentence-transformers)
  Downloading torch-2.7.1-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (29 kB)
Collecting scikit-learn (from sentence-transformers)
  Downloading scikit_learn-1.7.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (17 kB)
Collecting scipy (from sentence-transformers)
  Using cached scipy-1.15.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting huggingface-hub>=0.20.0 (from sentence-transformers)
  Using cached huggingface_hub-0.33.0-py3-none-any.whl.metadata (14 kB)
Collecting Pillow (from sentence-transformers)
  Using cached pillow-11.2.1-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (8.9 kB)
Requirement already satisfied: typing_extensions>=4.5.0 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from sentence-transformers) (4.14.0)
Collecting build>=1.0.3 (from chromadb)
  Using cached build-1.2.2.post1-py3-none-any.whl.metadata (6.5 kB)
Collecting fastapi==0.115.9 (from chromadb)
  Downloading fastapi-0.115.9-py3-none-any.whl.metadata (27 kB)
Collecting uvicorn>=0.18.3 (from uvicorn[standard]>=0.18.3->chromadb)
  Using cached uvicorn-0.34.3-py3-none-any.whl.metadata (6.5 kB)
Collecting posthog>=2.4.0 (from chromadb)
  Downloading posthog-4.10.0-py3-none-any.whl.metadata (6.0 kB)
Collecting onnxruntime>=1.14.1 (from chromadb)
  Downloading onnxruntime-1.22.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (4.8 kB)
Collecting opentelemetry-api>=1.2.0 (from chromadb)
  Downloading opentelemetry_api-1.34.1-py3-none-any.whl.metadata (1.5 kB)
Collecting opentelemetry-exporter-otlp-proto-grpc>=1.2.0 (from chromadb)
  Downloading opentelemetry_exporter_otlp_proto_grpc-1.34.1-py3-none-any.whl.metadata (2.4 kB)
Collecting opentelemetry-instrumentation-fastapi>=0.41b0 (from chromadb)
  Downloading opentelemetry_instrumentation_fastapi-0.55b1-py3-none-any.whl.metadata (2.2 kB)
Collecting opentelemetry-sdk>=1.2.0 (from chromadb)
  Downloading opentelemetry_sdk-1.34.1-py3-none-any.whl.metadata (1.6 kB)
Collecting tokenizers>=0.13.2 (from chromadb)
  Using cached tokenizers-0.21.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting pypika>=0.48.9 (from chromadb)
  Downloading PyPika-0.48.9.tar.gz (67 kB)
  Installing build dependencies ... [?25ldone
[?25h  Getting requirements to build wheel ... [?25ldone
[?25h  Preparing metadata (pyproject.toml) ... [?25ldone
[?25hCollecting overrides>=7.3.1 (from chromadb)
  Downloading overrides-7.7.0-py3-none-any.whl.metadata (5.8 kB)
Collecting importlib-resources (from chromadb)
  Downloading importlib_resources-6.5.2-py3-none-any.whl.metadata (3.9 kB)
Collecting grpcio>=1.58.0 (from chromadb)
  Downloading grpcio-1.73.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting bcrypt>=4.0.1 (from chromadb)
  Downloading bcrypt-4.3.0-cp39-abi3-manylinux_2_34_x86_64.whl.metadata (10 kB)
Collecting typer>=0.9.0 (from chromadb)
  Using cached typer-0.16.0-py3-none-any.whl.metadata (15 kB)
Collecting kubernetes>=28.1.0 (from chromadb)
  Downloading kubernetes-33.1.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting mmh3>=4.0.1 (from chromadb)
  Downloading mmh3-5.1.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (16 kB)
Collecting orjson>=3.9.12 (from chromadb)
  Downloading orjson-3.10.18-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (41 kB)
Collecting httpx>=0.27.0 (from chromadb)
  Using cached httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Collecting rich>=10.11.0 (from chromadb)
  Using cached rich-14.0.0-py3-none-any.whl.metadata (18 kB)
Collecting jsonschema>=4.19.0 (from chromadb)
  Using cached jsonschema-4.24.0-py3-none-any.whl.metadata (7.8 kB)
Collecting starlette<0.46.0,>=0.40.0 (from fastapi==0.115.9->chromadb)
  Downloading starlette-0.45.3-py3-none-any.whl.metadata (6.3 kB)
Collecting aiohappyeyeballs>=2.5.0 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Using cached aiohappyeyeballs-2.6.1-py3-none-any.whl.metadata (5.9 kB)
Collecting aiosignal>=1.1.2 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Using cached aiosignal-1.3.2-py2.py3-none-any.whl.metadata (3.8 kB)
Collecting attrs>=17.3.0 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Using cached attrs-25.3.0-py3-none-any.whl.metadata (10 kB)
Collecting frozenlist>=1.1.1 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Downloading frozenlist-1.7.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Using cached multidict-6.4.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.3 kB)
Collecting propcache>=0.2.0 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Downloading propcache-0.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting yarl<2.0,>=1.17.0 (from aiohttp<4.0.0,>=3.8.3->langchain-community)
  Downloading yarl-1.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (73 kB)
Collecting packaging>=19.1 (from build>=1.0.3->chromadb)
  Using cached packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pyproject_hooks (from build>=1.0.3->chromadb)
  Using cached pyproject_hooks-1.2.0-py3-none-any.whl.metadata (1.3 kB)
Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain-community)
  Downloading marshmallow-3.26.1-py3-none-any.whl.metadata (7.3 kB)
Collecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain-community)
  Downloading typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB)
Collecting anyio (from httpx>=0.27.0->chromadb)
  Using cached anyio-4.9.0-py3-none-any.whl.metadata (4.7 kB)
Requirement already satisfied: certifi in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from httpx>=0.27.0->chromadb) (2025.4.26)
Collecting httpcore==1.* (from httpx>=0.27.0->chromadb)
  Using cached httpcore-1.0.9-py3-none-any.whl.metadata (21 kB)
Requirement already satisfied: idna in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from httpx>=0.27.0->chromadb) (3.10)
Collecting h11>=0.16 (from httpcore==1.*->httpx>=0.27.0->chromadb)
  Using cached h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
Collecting filelock (from huggingface-hub>=0.20.0->sentence-transformers)
  Using cached filelock-3.18.0-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec>=2023.5.0 (from huggingface-hub>=0.20.0->sentence-transformers)
  Using cached fsspec-2025.5.1-py3-none-any.whl.metadata (11 kB)
Collecting hf-xet<2.0.0,>=1.1.2 (from huggingface-hub>=0.20.0->sentence-transformers)
  Using cached hf_xet-1.1.3-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (879 bytes)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=4.19.0->chromadb)
  Using cached jsonschema_specifications-2025.4.1-py3-none-any.whl.metadata (2.9 kB)
Collecting referencing>=0.28.4 (from jsonschema>=4.19.0->chromadb)
  Using cached referencing-0.36.2-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=4.19.0->chromadb)
  Using cached rpds_py-0.25.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
Collecting six>=1.9.0 (from kubernetes>=28.1.0->chromadb)
  Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting python-dateutil>=2.5.3 (from kubernetes>=28.1.0->chromadb)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting google-auth>=1.0.1 (from kubernetes>=28.1.0->chromadb)
  Downloading google_auth-2.40.3-py2.py3-none-any.whl.metadata (6.2 kB)
Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes>=28.1.0->chromadb)
  Downloading websocket_client-1.8.0-py3-none-any.whl.metadata (8.0 kB)
Collecting requests-oauthlib (from kubernetes>=28.1.0->chromadb)
  Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB)
Collecting oauthlib>=3.2.2 (from kubernetes>=28.1.0->chromadb)
  Downloading oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB)
Requirement already satisfied: urllib3>=1.24.2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from kubernetes>=28.1.0->chromadb) (2.4.0)
Collecting durationpy>=0.7 (from kubernetes>=28.1.0->chromadb)
  Downloading durationpy-0.10-py3-none-any.whl.metadata (340 bytes)
Collecting jsonpatch<2.0,>=1.33 (from langchain-core<1.0.0,>=0.3.58->langchain)
  Downloading jsonpatch-1.33-py2.py3-none-any.whl.metadata (3.0 kB)
Collecting packaging>=19.1 (from build>=1.0.3->chromadb)
  Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
Collecting requests-toolbelt<2.0.0,>=1.0.0 (from langsmith<0.4,>=0.1.17->langchain)
  Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl.metadata (14 kB)
Collecting zstandard<0.24.0,>=0.23.0 (from langsmith<0.4,>=0.1.17->langchain)
  Using cached zstandard-0.23.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting coloredlogs (from onnxruntime>=1.14.1->chromadb)
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)
Collecting flatbuffers (from onnxruntime>=1.14.1->chromadb)
  Downloading flatbuffers-25.2.10-py2.py3-none-any.whl.metadata (875 bytes)
Collecting protobuf (from onnxruntime>=1.14.1->chromadb)
  Downloading protobuf-6.31.1-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes)
Collecting sympy (from onnxruntime>=1.14.1->chromadb)
  Using cached sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Collecting importlib-metadata<8.8.0,>=6.0 (from opentelemetry-api>=1.2.0->chromadb)
  Using cached importlib_metadata-8.7.0-py3-none-any.whl.metadata (4.8 kB)
Collecting googleapis-common-protos~=1.52 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)
  Using cached googleapis_common_protos-1.70.0-py3-none-any.whl.metadata (9.3 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.34.1 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)
  Downloading opentelemetry_exporter_otlp_proto_common-1.34.1-py3-none-any.whl.metadata (1.9 kB)
Collecting opentelemetry-proto==1.34.1 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb)
  Downloading opentelemetry_proto-1.34.1-py3-none-any.whl.metadata (2.4 kB)
Collecting protobuf (from onnxruntime>=1.14.1->chromadb)
  Downloading protobuf-5.29.5-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)
Collecting opentelemetry-instrumentation-asgi==0.55b1 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Downloading opentelemetry_instrumentation_asgi-0.55b1-py3-none-any.whl.metadata (2.0 kB)
Collecting opentelemetry-instrumentation==0.55b1 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Downloading opentelemetry_instrumentation-0.55b1-py3-none-any.whl.metadata (6.7 kB)
Collecting opentelemetry-semantic-conventions==0.55b1 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Downloading opentelemetry_semantic_conventions-0.55b1-py3-none-any.whl.metadata (2.5 kB)
Collecting opentelemetry-util-http==0.55b1 (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Downloading opentelemetry_util_http-0.55b1-py3-none-any.whl.metadata (2.6 kB)
Collecting wrapt<2.0.0,>=1.0.0 (from opentelemetry-instrumentation==0.55b1->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Using cached wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
Collecting asgiref~=3.0 (from opentelemetry-instrumentation-asgi==0.55b1->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb)
  Downloading asgiref-3.8.1-py3-none-any.whl.metadata (9.3 kB)
Collecting backoff>=1.10.0 (from posthog>=2.4.0->chromadb)
  Downloading backoff-2.2.1-py3-none-any.whl.metadata (14 kB)
Collecting distro>=1.5.0 (from posthog>=2.4.0->chromadb)
  Using cached distro-1.9.0-py3-none-any.whl.metadata (6.8 kB)
Collecting annotated-types>=0.6.0 (from pydantic<3.0.0,>=2.7.4->langchain)
  Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.33.2 (from pydantic<3.0.0,>=2.7.4->langchain)
  Using cached pydantic_core-2.33.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting typing-inspection>=0.4.0 (from pydantic<3.0.0,>=2.7.4->langchain)
  Using cached typing_inspection-0.4.1-py3-none-any.whl.metadata (2.6 kB)
Collecting python-dotenv>=0.21.0 (from pydantic-settings<3.0.0,>=2.4.0->langchain-community)
  Using cached python_dotenv-1.1.0-py3-none-any.whl.metadata (24 kB)
Requirement already satisfied: charset_normalizer<4,>=2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from requests<3,>=2->langchain) (3.4.2)
Collecting markdown-it-py>=2.2.0 (from rich>=10.11.0->chromadb)
  Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=10.11.0->chromadb)
  Using cached pygments-2.19.1-py3-none-any.whl.metadata (2.5 kB)
Collecting greenlet>=1 (from SQLAlchemy<3,>=1.4->langchain)
  Downloading greenlet-3.2.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (4.1 kB)
Collecting setuptools (from torch>=1.11.0->sentence-transformers)
  Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting networkx (from torch>=1.11.0->sentence-transformers)
  Using cached networkx-3.5-py3-none-any.whl.metadata (6.3 kB)
Requirement already satisfied: jinja2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from torch>=1.11.0->sentence-transformers) (3.1.6)
Collecting nvidia-cuda-nvrtc-cu12==12.6.77 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cuda_nvrtc_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.6.77 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cuda_runtime_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.6.80 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cuda_cupti_cu12-12.6.80-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu12==9.5.1.17 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.6.4.1 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cublas_cu12-12.6.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.3.0.4 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cufft_cu12-11.3.0.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.7.77 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_curand_cu12-10.3.7.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.7.1.2 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cusolver_cu12-11.7.1.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.5.4.2 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cusparse_cu12-12.5.4.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparselt-cu12==0.6.3 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cusparselt_cu12-0.6.3-py3-none-manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting nvidia-nccl-cu12==2.26.2 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_nccl_cu12-2.26.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
Collecting nvidia-nvtx-cu12==12.6.77 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_nvtx_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-nvjitlink-cu12==12.6.85 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_nvjitlink_cu12-12.6.85-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufile-cu12==1.11.1.6 (from torch>=1.11.0->sentence-transformers)
  Using cached nvidia_cufile_cu12-1.11.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
Collecting triton==3.3.1 (from torch>=1.11.0->sentence-transformers)
  Using cached triton-3.3.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.5 kB)
Collecting regex!=2019.12.17 (from transformers<5.0.0,>=4.41.0->sentence-transformers)
  Using cached regex-2024.11.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
Collecting safetensors>=0.4.3 (from transformers<5.0.0,>=4.41.0->sentence-transformers)
  Using cached safetensors-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting click>=8.0.0 (from typer>=0.9.0->chromadb)
  Using cached click-8.2.1-py3-none-any.whl.metadata (2.5 kB)
Collecting shellingham>=1.3.0 (from typer>=0.9.0->chromadb)
  Using cached shellingham-1.5.4-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting httptools>=0.6.3 (from uvicorn[standard]>=0.18.3->chromadb)
  Using cached httptools-0.6.4-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.6 kB)
Collecting uvloop>=0.15.1 (from uvicorn[standard]>=0.18.3->chromadb)
  Using cached uvloop-0.21.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)
Collecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb)
  Using cached watchfiles-1.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)
Collecting websockets>=10.4 (from uvicorn[standard]>=0.18.3->chromadb)
  Using cached websockets-15.0.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting joblib>=1.2.0 (from scikit-learn->sentence-transformers)
  Downloading joblib-1.5.1-py3-none-any.whl.metadata (5.6 kB)
Collecting threadpoolctl>=3.1.0 (from scikit-learn->sentence-transformers)
  Downloading threadpoolctl-3.6.0-py3-none-any.whl.metadata (13 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb)
  Downloading cachetools-5.5.2-py3-none-any.whl.metadata (5.4 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb)
  Downloading pyasn1_modules-0.4.2-py3-none-any.whl.metadata (3.5 kB)
Collecting rsa<5,>=3.1.4 (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb)
  Downloading rsa-4.9.1-py3-none-any.whl.metadata (5.6 kB)
Collecting zipp>=3.20 (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.2.0->chromadb)
  Downloading zipp-3.23.0-py3-none-any.whl.metadata (3.6 kB)
Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain-core<1.0.0,>=0.3.58->langchain)
  Downloading jsonpointer-3.0.0-py2.py3-none-any.whl.metadata (2.3 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=10.11.0->chromadb)
  Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Collecting sniffio>=1.1 (from anyio->httpx>=0.27.0->chromadb)
  Using cached sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy->onnxruntime>=1.14.1->chromadb)
  Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Collecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain-community)
  Downloading mypy_extensions-1.1.0-py3-none-any.whl.metadata (1.1 kB)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb)
  Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)
Requirement already satisfied: MarkupSafe>=2.0 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from jinja2->torch>=1.11.0->sentence-transformers) (3.0.2)
Collecting pyasn1<0.7.0,>=0.6.1 (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb)
  Downloading pyasn1-0.6.1-py3-none-any.whl.metadata (8.4 kB)
Downloading langchain-0.3.25-py3-none-any.whl (1.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 46.4 MB/s eta 0:00:00
[?25hDownloading langchain_community-0.3.25-py3-none-any.whl (2.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 56.0 MB/s eta 0:00:00
[?25hDownloading sentence_transformers-4.1.0-py3-none-any.whl (345 kB)
Downloading chromadb-1.0.12-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.3/19.3 MB 71.1 MB/s eta 0:00:00 MB/s eta 0:00:01
[?25hDownloading fastapi-0.115.9-py3-none-any.whl (94 kB)
Downloading aiohttp-3.12.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 51.8 MB/s eta 0:00:00
[?25hDownloading bcrypt-4.3.0-cp39-abi3-manylinux_2_34_x86_64.whl (284 kB)
Using cached build-1.2.2.post1-py3-none-any.whl (22 kB)
Downloading dataclasses_json-0.6.7-py3-none-any.whl (28 kB)
Downloading grpcio-1.73.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 54.1 MB/s eta 0:00:00
[?25hUsing cached httpx-0.28.1-py3-none-any.whl (73 kB)
Downloading httpcore-1.0.9-py3-none-any.whl (78 kB)
Downloading httpx_sse-0.4.0-py3-none-any.whl (7.8 kB)
Using cached huggingface_hub-0.33.0-py3-none-any.whl (514 kB)
Downloading jsonschema-4.24.0-py3-none-any.whl (88 kB)
Downloading kubernetes-33.1.0-py2.py3-none-any.whl (1.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 53.7 MB/s eta 0:00:00
[?25hDownloading langchain_core-0.3.65-py3-none-any.whl (438 kB)
Downloading langchain_text_splitters-0.3.8-py3-none-any.whl (32 kB)
Downloading langsmith-0.3.45-py3-none-any.whl (363 kB)
Downloading mmh3-5.1.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (101 kB)
Downloading onnxruntime-1.22.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.4/16.4 MB 49.3 MB/s eta 0:00:00MB/s eta 0:00:01
[?25hDownloading opentelemetry_api-1.34.1-py3-none-any.whl (65 kB)
Downloading opentelemetry_exporter_otlp_proto_grpc-1.34.1-py3-none-any.whl (18 kB)
Downloading opentelemetry_exporter_otlp_proto_common-1.34.1-py3-none-any.whl (18 kB)
Downloading opentelemetry_proto-1.34.1-py3-none-any.whl (55 kB)
Downloading opentelemetry_instrumentation_fastapi-0.55b1-py3-none-any.whl (12 kB)
Downloading opentelemetry_instrumentation-0.55b1-py3-none-any.whl (31 kB)
Downloading opentelemetry_instrumentation_asgi-0.55b1-py3-none-any.whl (16 kB)
Downloading opentelemetry_semantic_conventions-0.55b1-py3-none-any.whl (196 kB)
Downloading opentelemetry_util_http-0.55b1-py3-none-any.whl (7.3 kB)
Downloading opentelemetry_sdk-1.34.1-py3-none-any.whl (118 kB)
Downloading orjson-3.10.18-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (133 kB)
Downloading overrides-7.7.0-py3-none-any.whl (17 kB)
Downloading posthog-4.10.0-py3-none-any.whl (102 kB)
Downloading pydantic-2.11.7-py3-none-any.whl (444 kB)
Downloading pydantic_core-2.33.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 55.6 MB/s eta 0:00:00
[?25hDownloading pydantic_settings-2.9.1-py3-none-any.whl (44 kB)
Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 56.3 MB/s eta 0:00:00
[?25hDownloading rich-14.0.0-py3-none-any.whl (243 kB)
Downloading sqlalchemy-2.0.41-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 57.2 MB/s eta 0:00:00
[?25hDownloading tenacity-9.1.2-py3-none-any.whl (28 kB)
Downloading tokenizers-0.21.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 51.2 MB/s eta 0:00:00
[?25hDownloading torch-2.7.1-cp312-cp312-manylinux_2_28_x86_64.whl (821.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 821.0/821.0 MB 63.6 MB/s eta 0:00:00m eta 0:00:01[36m0:00:01
[?25hDownloading nvidia_cublas_cu12-12.6.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (393.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 393.1/393.1 MB 56.4 MB/s eta 0:00:00m eta 0:00:010:01:01
[?25hDownloading nvidia_cuda_cupti_cu12-12.6.80-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (8.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.9/8.9 MB 41.1 MB/s eta 0:00:00m eta 0:00:01
[?25hDownloading nvidia_cuda_nvrtc_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl (23.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 53.1 MB/s eta 0:00:0031m57.5 MB/s eta 0:00:01
[?25hDownloading nvidia_cuda_runtime_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (897 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 897.7/897.7 kB 51.8 MB/s eta 0:00:00
[?25hDownloading nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl (571.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 571.0/571.0 MB 60.0 MB/s eta 0:00:00m eta 0:00:010:01:03
[?25hDownloading nvidia_cufft_cu12-11.3.0.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (200.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.2/200.2 MB 63.5 MB/s eta 0:00:00m eta 0:00:010:01:01
[?25hDownloading nvidia_cufile_cu12-1.11.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 70.3 MB/s eta 0:00:00
[?25hDownloading nvidia_curand_cu12-10.3.7.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (56.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.3/56.3 MB 39.1 MB/s eta 0:00:00m eta 0:00:010:01:01
[?25hDownloading nvidia_cusolver_cu12-11.7.1.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (158.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.2/158.2 MB 68.3 MB/s eta 0:00:00m eta 0:00:01[36m0:00:01
[?25hDownloading nvidia_cusparse_cu12-12.5.4.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (216.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 216.6/216.6 MB 57.5 MB/s eta 0:00:00m eta 0:00:01[36m0:00:01
[?25hDownloading nvidia_cusparselt_cu12-0.6.3-py3-none-manylinux2014_x86_64.whl (156.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 156.8/156.8 MB 53.8 MB/s eta 0:00:00m eta 0:00:010:01:01
[?25hDownloading nvidia_nccl_cu12-2.26.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (201.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.3/201.3 MB 58.9 MB/s eta 0:00:00[36m0:00:01[36m0:00:01:02
[?25hDownloading nvidia_nvjitlink_cu12-12.6.85-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (19.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.7/19.7 MB 68.2 MB/s eta 0:00:00 MB/s eta 0:00:01
[?25hDownloading nvidia_nvtx_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89 kB)
Downloading triton-3.3.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (155.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.7/155.7 MB 57.0 MB/s eta 0:00:00m eta 0:00:01[36m0:00:01
[?25hUsing cached tqdm-4.67.1-py3-none-any.whl (78 kB)
Downloading transformers-4.52.4-py3-none-any.whl (10.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 26.3 MB/s eta 0:00:00MB/s eta 0:00:01
[?25hDownloading typer-0.16.0-py3-none-any.whl (46 kB)
Downloading uvicorn-0.34.3-py3-none-any.whl (62 kB)
Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB)
Downloading pillow-11.2.1-cp312-cp312-manylinux_2_28_x86_64.whl (4.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 59.0 MB/s eta 0:00:00
[?25hDownloading scikit_learn-1.7.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.5/12.5 MB 62.4 MB/s eta 0:00:00
[?25hDownloading scipy-1.15.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (37.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.3/37.3 MB 40.5 MB/s eta 0:00:0031m40.4 MB/s eta 0:00:01
[?25hDownloading aiohappyeyeballs-2.6.1-py3-none-any.whl (15 kB)
Downloading aiosignal-1.3.2-py2.py3-none-any.whl (7.6 kB)
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Downloading attrs-25.3.0-py3-none-any.whl (63 kB)
Downloading backoff-2.2.1-py3-none-any.whl (15 kB)
Downloading click-8.2.1-py3-none-any.whl (102 kB)
Downloading distro-1.9.0-py3-none-any.whl (20 kB)
Downloading durationpy-0.10-py3-none-any.whl (3.9 kB)
Downloading frozenlist-1.7.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (241 kB)
Using cached fsspec-2025.5.1-py3-none-any.whl (199 kB)
Downloading google_auth-2.40.3-py2.py3-none-any.whl (216 kB)
Downloading googleapis_common_protos-1.70.0-py3-none-any.whl (294 kB)
Downloading greenlet-3.2.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (605 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 605.5/605.5 kB 66.9 MB/s eta 0:00:00
[?25hDownloading h11-0.16.0-py3-none-any.whl (37 kB)
Using cached hf_xet-1.1.3-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
Downloading httptools-0.6.4-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (510 kB)
Downloading importlib_metadata-8.7.0-py3-none-any.whl (27 kB)
Downloading joblib-1.5.1-py3-none-any.whl (307 kB)
Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Downloading jsonschema_specifications-2025.4.1-py3-none-any.whl (18 kB)
Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Downloading marshmallow-3.26.1-py3-none-any.whl (50 kB)
Downloading multidict-6.4.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (223 kB)
Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
Using cached packaging-24.2-py3-none-any.whl (65 kB)
Downloading propcache-0.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (224 kB)
Downloading protobuf-5.29.5-cp38-abi3-manylinux2014_x86_64.whl (319 kB)
Using cached pygments-2.19.1-py3-none-any.whl (1.2 MB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached python_dotenv-1.1.0-py3-none-any.whl (20 kB)
Downloading referencing-0.36.2-py3-none-any.whl (26 kB)
Downloading regex-2024.11.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (796 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 796.9/796.9 kB 22.9 MB/s eta 0:00:00
[?25hUsing cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
Downloading rpds_py-0.25.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (390 kB)
Downloading safetensors-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (471 kB)
Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
Using cached shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
Downloading starlette-0.45.3-py3-none-any.whl (71 kB)
Using cached anyio-4.9.0-py3-none-any.whl (100 kB)
Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 54.6 MB/s eta 0:00:00
[?25hDownloading threadpoolctl-3.6.0-py3-none-any.whl (18 kB)
Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Downloading typing_inspection-0.4.1-py3-none-any.whl (14 kB)
Downloading uvloop-0.21.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 48.6 MB/s eta 0:00:00
[?25hDownloading watchfiles-1.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (454 kB)
Downloading websocket_client-1.8.0-py3-none-any.whl (58 kB)
Downloading websockets-15.0.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (182 kB)
Downloading yarl-1.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (355 kB)
Using cached zstandard-0.23.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Using cached filelock-3.18.0-py3-none-any.whl (16 kB)
Downloading flatbuffers-25.2.10-py2.py3-none-any.whl (30 kB)
Downloading networkx-3.5-py3-none-any.whl (2.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 64.7 MB/s eta 0:00:00
[?25hUsing cached pyproject_hooks-1.2.0-py3-none-any.whl (10 kB)
Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB)
Downloading asgiref-3.8.1-py3-none-any.whl (23 kB)
Downloading cachetools-5.5.2-py3-none-any.whl (10 kB)
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Downloading jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 86.9 MB/s eta 0:00:00
[?25hDownloading mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB)
Downloading pyasn1_modules-0.4.2-py3-none-any.whl (181 kB)
Downloading rsa-4.9.1-py3-none-any.whl (34 kB)
Using cached sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
Downloading zipp-3.23.0-py3-none-any.whl (10 kB)
Downloading pyasn1-0.6.1-py3-none-any.whl (83 kB)
Building wheels for collected packages: pypika
  Building wheel for pypika (pyproject.toml) ... [?25ldone
[?25h  Created wheel for pypika: filename=pypika-0.48.9-py2.py3-none-any.whl size=53803 sha256=9ee9ad3b0a26b18eaa97d8d97563a52af215fccbe602a699e766c21fb3ece97c
  Stored in directory: /home/user/.cache/pip/wheels/d5/3d/69/8d68d249cd3de2584f226e27fd431d6344f7d70fd856ebd01b
Successfully built pypika
Installing collected packages: pypika, nvidia-cusparselt-cu12, mpmath, flatbuffers, durationpy, zstandard, zipp, wrapt, websockets, websocket-client, uvloop, typing-inspection, tqdm, threadpoolctl, tenacity, sympy, sniffio, six, shellingham, setuptools, scipy, safetensors, rpds-py, regex, PyYAML, python-dotenv, pyproject_hooks, pygments, pydantic-core, pyasn1, protobuf, propcache, Pillow, packaging, overrides, orjson, opentelemetry-util-http, oauthlib, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufile-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, networkx, mypy-extensions, multidict, mmh3, mdurl, jsonpointer, joblib, importlib-resources, humanfriendly, httpx-sse, httptools, hf-xet, h11, grpcio, greenlet, fsspec, frozenlist, filelock, distro, click, cachetools, bcrypt, backoff, attrs, asgiref, annotated-types, aiohappyeyeballs, yarl, uvicorn, typing-inspect, triton, SQLAlchemy, scikit-learn, rsa, requests-toolbelt, requests-oauthlib, referencing, python-dateutil, pydantic, pyasn1-modules, opentelemetry-proto, nvidia-cusparse-cu12, nvidia-cufft-cu12, nvidia-cudnn-cu12, marshmallow, markdown-it-py, jsonpatch, importlib-metadata, huggingface-hub, httpcore, googleapis-common-protos, coloredlogs, build, anyio, aiosignal, watchfiles, tokenizers, starlette, rich, pydantic-settings, posthog, opentelemetry-exporter-otlp-proto-common, opentelemetry-api, onnxruntime, nvidia-cusolver-cu12, jsonschema-specifications, httpx, google-auth, dataclasses-json, aiohttp, typer, transformers, torch, opentelemetry-semantic-conventions, langsmith, kubernetes, jsonschema, fastapi, sentence-transformers, opentelemetry-sdk, opentelemetry-instrumentation, langchain-core, opentelemetry-instrumentation-asgi, opentelemetry-exporter-otlp-proto-grpc, langchain-text-splitters, opentelemetry-instrumentation-fastapi, langchain, langchain-community, chromadb
Successfully installed Pillow-11.2.1 PyYAML-6.0.2 SQLAlchemy-2.0.41 aiohappyeyeballs-2.6.1 aiohttp-3.12.13 aiosignal-1.3.2 annotated-types-0.7.0 anyio-4.9.0 asgiref-3.8.1 attrs-25.3.0 backoff-2.2.1 bcrypt-4.3.0 build-1.2.2.post1 cachetools-5.5.2 chromadb-1.0.12 click-8.2.1 coloredlogs-15.0.1 dataclasses-json-0.6.7 distro-1.9.0 durationpy-0.10 fastapi-0.115.9 filelock-3.18.0 flatbuffers-25.2.10 frozenlist-1.7.0 fsspec-2025.5.1 google-auth-2.40.3 googleapis-common-protos-1.70.0 greenlet-3.2.3 grpcio-1.73.0 h11-0.16.0 hf-xet-1.1.3 httpcore-1.0.9 httptools-0.6.4 httpx-0.28.1 httpx-sse-0.4.0 huggingface-hub-0.33.0 humanfriendly-10.0 importlib-metadata-8.7.0 importlib-resources-6.5.2 joblib-1.5.1 jsonpatch-1.33 jsonpointer-3.0.0 jsonschema-4.24.0 jsonschema-specifications-2025.4.1 kubernetes-33.1.0 langchain-0.3.25 langchain-community-0.3.25 langchain-core-0.3.65 langchain-text-splitters-0.3.8 langsmith-0.3.45 markdown-it-py-3.0.0 marshmallow-3.26.1 mdurl-0.1.2 mmh3-5.1.0 mpmath-1.3.0 multidict-6.4.4 mypy-extensions-1.1.0 networkx-3.5 nvidia-cublas-cu12-12.6.4.1 nvidia-cuda-cupti-cu12-12.6.80 nvidia-cuda-nvrtc-cu12-12.6.77 nvidia-cuda-runtime-cu12-12.6.77 nvidia-cudnn-cu12-9.5.1.17 nvidia-cufft-cu12-11.3.0.4 nvidia-cufile-cu12-1.11.1.6 nvidia-curand-cu12-10.3.7.77 nvidia-cusolver-cu12-11.7.1.2 nvidia-cusparse-cu12-12.5.4.2 nvidia-cusparselt-cu12-0.6.3 nvidia-nccl-cu12-2.26.2 nvidia-nvjitlink-cu12-12.6.85 nvidia-nvtx-cu12-12.6.77 oauthlib-3.2.2 onnxruntime-1.22.0 opentelemetry-api-1.34.1 opentelemetry-exporter-otlp-proto-common-1.34.1 opentelemetry-exporter-otlp-proto-grpc-1.34.1 opentelemetry-instrumentation-0.55b1 opentelemetry-instrumentation-asgi-0.55b1 opentelemetry-instrumentation-fastapi-0.55b1 opentelemetry-proto-1.34.1 opentelemetry-sdk-1.34.1 opentelemetry-semantic-conventions-0.55b1 opentelemetry-util-http-0.55b1 orjson-3.10.18 overrides-7.7.0 packaging-24.2 posthog-4.10.0 propcache-0.3.2 protobuf-5.29.5 pyasn1-0.6.1 pyasn1-modules-0.4.2 pydantic-2.11.7 pydantic-core-2.33.2 pydantic-settings-2.9.1 pygments-2.19.1 pypika-0.48.9 pyproject_hooks-1.2.0 python-dateutil-2.9.0.post0 python-dotenv-1.1.0 referencing-0.36.2 regex-2024.11.6 requests-oauthlib-2.0.0 requests-toolbelt-1.0.0 rich-14.0.0 rpds-py-0.25.1 rsa-4.9.1 safetensors-0.5.3 scikit-learn-1.7.0 scipy-1.15.3 sentence-transformers-4.1.0 setuptools-80.9.0 shellingham-1.5.4 six-1.17.0 sniffio-1.3.1 starlette-0.45.3 sympy-1.14.0 tenacity-9.1.2 threadpoolctl-3.6.0 tokenizers-0.21.1 torch-2.7.1 tqdm-4.67.1 transformers-4.52.4 triton-3.3.1 typer-0.16.0 typing-inspect-0.9.0 typing-inspection-0.4.1 uvicorn-0.34.3 uvloop-0.21.0 watchfiles-1.0.5 websocket-client-1.8.0 websockets-15.0.1 wrapt-1.17.2 yarl-1.20.1 zipp-3.23.0 zstandard-0.23.0

[notice] A new release of pip is available: 24.3.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip
Collecting pypdf
  Downloading pypdf-5.6.0-py3-none-any.whl.metadata (7.2 kB)
Requirement already satisfied: requests in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (2.32.4)
Requirement already satisfied: pydantic in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (2.11.7)
Requirement already satisfied: tqdm in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (4.67.1)
Requirement already satisfied: charset_normalizer<4,>=2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from requests) (3.4.2)
Requirement already satisfied: idna<4,>=2.5 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from requests) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from requests) (2.4.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from requests) (2025.4.26)
Requirement already satisfied: annotated-types>=0.6.0 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from pydantic) (0.7.0)
Requirement already satisfied: pydantic-core==2.33.2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from pydantic) (2.33.2)
Requirement already satisfied: typing-extensions>=4.12.2 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from pydantic) (4.14.0)
Requirement already satisfied: typing-inspection>=0.4.0 in /home/user/.pyenv/versions/3.12.9/lib/python3.12/site-packages (from pydantic) (0.4.1)
Downloading pypdf-5.6.0-py3-none-any.whl (304 kB)
Installing collected packages: pypdf
Successfully installed pypdf-5.6.0

[notice] A new release of pip is available: 24.3.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip

In [3]:

import os
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Chroma
from langchain.document_loaders import PyPDFLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate
from langchain.llms import LlamaCpp
import requests
from tqdm import tqdm
import time

In [ ]:

model_path = "llama-2-7b-chat.Q4_K_M.gguf"
 
if not os.path.exists(model_path):
    print(f"Downloading {model_path}...")
    # You may want to replace the model URL by another of your choice
    model_url = "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_M.gguf"
    response = requests.get(model_url, stream=True)
    total_size = int(response.headers.get('content-length', 0))
    
    with open(model_path, 'wb') as f:
        for data in tqdm(response.iter_content(chunk_size=1024), total=total_size//1024):
            f.write(data)
    print("Download complete!")

Downloading llama-2-7b-chat.Q4_K_M.gguf...

3985356it [00:49, 80597.71it/s]                             

Download complete!

In [ ]:

os.makedirs("docs", exist_ok=True)

# Sample text for demonstration purposes
with open("docs/sample.txt", "w") as f:
    f.write("""
    Retrieval-Augmented Generation (RAG) is a technique that combines retrieval-based and generation-based approaches
    for natural language processing tasks. It involves retrieving relevant information from a knowledge base and then 
    using that information to generate more accurate and informed responses.
    
    RAG models first retrieve documents that are relevant to a given query, then use these documents as additional context
    for language generation. This approach helps to ground the model's responses in factual information and reduces hallucinations.
    
    The llama.cpp library is a C/C++ implementation of Meta's LLaMA model, optimized for CPU usage. It allows running LLaMA models
    on consumer hardware without requiring high-end GPUs.
    
    LocalAI is a framework that enables running AI models locally without relying on cloud services. It provides APIs compatible
    with OpenAI's interfaces, allowing developers to use their own models with the same code they would use for OpenAI services.
    """)

documents = []
for file in os.listdir("docs"):
    if file.endswith(".pdf"):
        loader = PyPDFLoader(os.path.join("docs", file))
        documents.extend(loader.load())
    elif file.endswith(".txt"):
        loader = TextLoader(os.path.join("docs", file))
        documents.extend(loader.load())

# Split documents into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    length_function=len
)

chunks = text_splitter.split_documents(documents)

In [6]:

embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="./chroma_db"
)

/tmp/ipykernel_31780/57795070.py:1: LangChainDeprecationWarning: The class `HuggingFaceEmbeddings` was deprecated in LangChain 0.2.2 and will be removed in 1.0. An updated version of the class exists in the :class:`~langchain-huggingface package and should be used instead. To use it run `pip install -U :class:`~langchain-huggingface` and import as `from :class:`~langchain_huggingface import HuggingFaceEmbeddings``.
  embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

modules.json:   0%|          | 0.00/349 [00:00<?, ?B/s]

config_sentence_transformers.json:   0%|          | 0.00/116 [00:00<?, ?B/s]

README.md:   0%|          | 0.00/10.5k [00:00<?, ?B/s]

sentence_bert_config.json:   0%|          | 0.00/53.0 [00:00<?, ?B/s]

config.json:   0%|          | 0.00/612 [00:00<?, ?B/s]

model.safetensors:   0%|          | 0.00/90.9M [00:00<?, ?B/s]

tokenizer_config.json:   0%|          | 0.00/350 [00:00<?, ?B/s]

vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]

tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]

special_tokens_map.json:   0%|          | 0.00/112 [00:00<?, ?B/s]

config.json:   0%|          | 0.00/190 [00:00<?, ?B/s]

In [ ]:

llm = LlamaCpp(
    model_path='/models/DeepSeek-R1-0528-Qwen3-8B-UD-Q4_K_XL.gguf',
    temperature=0.7,
    max_tokens=2000,
    n_ctx=4096,
    verbose=False
)

llama_context: n_batch is less than GGML_KQ_MASK_PAD - increasing to 64

In [8]:

template = """
Answer the question based on the following context:

{context}

Question: {question}
Answer:
"""
prompt = PromptTemplate(
    template=template,
    input_variables=["context", "question"]
)

In [9]:

rag_pipeline = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 3}),
    return_source_documents=True,
    chain_type_kwargs={"prompt": prompt}
)

In [10]:

def ask_question(question):
    start_time = time.time()
    result = rag_pipeline({"query": question})
    end_time = time.time()
    
    print(f"Question: {question}")
    print(f"Answer: {result['result']}")
    print(f"Time taken: {end_time - start_time:.2f} seconds")
    print("\nSource documents:")
    for i, doc in enumerate(result["source_documents"]):
        print(f"Document {i+1}:")
        print(f"Source: {doc.metadata.get('source', 'Unknown')}")
        print(f"Content: {doc.page_content[:150]}...\n")

In [11]:

ask_question("What is RAG and how does it work?")
ask_question("What is llama.cpp?")
ask_question("How does LocalAI relate to cloud AI services?")

/tmp/ipykernel_31780/1799130441.py:3: LangChainDeprecationWarning: The method `Chain.__call__` was deprecated in langchain 0.1.0 and will be removed in 1.0. Use :meth:`~invoke` instead.
  result = rag_pipeline({"query": question})
Number of requested results 3 is greater than number of elements in index 2, updating n_results = 2
Number of requested results 3 is greater than number of elements in index 2, updating n_results = 2

Question: What is RAG and how does it work?
Answer: RAG is a technique that combines retrieval-based and generation-based approaches for natural language processing tasks. It involves retrieving relevant information from a knowledge base and then using that information to generate more accurate and informed responses. The model first retrieves documents that are relevant to a given query, then uses these documents as additional context for language generation. This approach helps to ground the model's responses in factual information and reduces hallucinations.
Time taken: 36.98 seconds

Source documents:
Document 1:
Source: docs/sample.txt
Content: Retrieval-Augmented Generation (RAG) is a technique that combines retrieval-based and generation-based approaches
    for natural language processing ...

Document 2:
Source: docs/sample.txt
Content: The llama.cpp library is a C/C++ implementation of Meta's LLaMA model, optimized for CPU usage. It allows running LLaMA models
    on consumer hardwar...

Number of requested results 3 is greater than number of elements in index 2, updating n_results = 2

Question: What is llama.cpp?
Answer: llama.cpp is a C/C++ implementation of Meta's LLaMA model, optimized for CPU usage. It allows running LLaMA models on consumer hardware without requiring high-end GPUs.
Time taken: 23.34 seconds

Source documents:
Document 1:
Source: docs/sample.txt
Content: The llama.cpp library is a C/C++ implementation of Meta's LLaMA model, optimized for CPU usage. It allows running LLaMA models
    on consumer hardwar...

Document 2:
Source: docs/sample.txt
Content: Retrieval-Augmented Generation (RAG) is a technique that combines retrieval-based and generation-based approaches
    for natural language processing ...

Question: How does LocalAI relate to cloud AI services?
Answer: LocalAI is a framework that enables running AI models locally without relying on cloud services. It provides APIs compatible with OpenAI's interfaces, allowing developers to use their own models with the same code they would use for OpenAI services. In other words, LocalAI allows developers to run their own AI models locally instead of relying on cloud-based services like OpenAI.
Time taken: 8.79 seconds

Source documents:
Document 1:
Source: docs/sample.txt
Content: The llama.cpp library is a C/C++ implementation of Meta's LLaMA model, optimized for CPU usage. It allows running LLaMA models
    on consumer hardwar...

Document 2:
Source: docs/sample.txt
Content: Retrieval-Augmented Generation (RAG) is a technique that combines retrieval-based and generation-based approaches
    for natural language processing ...

In [ ]: