mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
10a6c5abba
## Summary Add `scripts/milvus-debug.sh` to automate GDB coredump analysis using the `-debug` image that accompanies each Milvus release starting from v2.6.14. ## Background Starting with v2.6.14, each Milvus release publishes two image variants: | Tag | Content | Size | Use | |-----|---------|------|-----| | `milvusdb/milvus:<tag>` | **stripped** (production) | ~1/3 | Production deployment | | `milvusdb/milvus:<tag>-debug` | unstripped (full symbols) | full | GDB coredump analysis | The stripped production image is **byte-identical** to the debug image in its code sections (`strip --strip-debug` removes only `.debug_*` sections and symbol tables). So GDB can use the debug image's symbols to resolve addresses in a coredump produced by the stripped image. ## What the script does 1. Pulls the debug image 2. Mounts the coredump file read-only into a container 3. Installs `gdb` on demand (not present in the runtime image by default) 4. Launches GDB with `solib-search-path=/milvus/lib` so symbols from all shared libraries (`libknowhere.so`, `libtantivy-binding.so`, etc.) resolve correctly 5. Auto-prints the initial backtrace ## Typical usage after a production crash ```bash # 1. Copy the coredump out of the crashed pod kubectl cp <ns>/<pod>:/tmp/cores/core.<pid> ./core.<pid> # 2. Run this script (GDB starts automatically with backtrace) ./scripts/milvus-debug.sh ./core.<pid> \ milvusdb/milvus:v2.6.14 milvusdb/milvus:v2.6.14-debug ``` ## Test plan - [x] `bash -n scripts/milvus-debug.sh` (syntax check passes) - [x] Mirrors the documented procedure in the strip-image-coredump-debug-guide Feishu doc - [ ] Manual validation on a real coredump (waiting for v2.6.14 production crash; can be exercised with any coredump for now) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Compile and install milvus cluster
Environment
OS: Ubuntu 20.04
go：1.21
cmake: >=3.18
gcc: >= 11
Install dependencies
Install compile dependencies
$ sudo apt install -y g++ gcc make libssl-dev zlib1g-dev libboost-regex-dev \
libboost-program-options-dev libboost-system-dev libboost-filesystem-dev \
libboost-serialization-dev python3-dev libboost-python-dev libcurl4-openssl-dev gfortran libtbb-dev
$ export GO111MODULE=on
$ go get github.com/golang/protobuf/protoc-gen-go@v1.3.2
Install OpenBlas library
install using apt
sudo apt install -y libopenblas-dev
or build from source code
$ wget https://github.com/xianyi/OpenBLAS/archive/v0.3.9.tar.gz && \
$ tar zxvf v0.3.9.tar.gz && cd OpenBLAS-0.3.9 && \
$ make TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 USE_THREAD=0 USE_OPENMP=0 FC=gfortran CC=gcc COMMON_OPT="-O3 -g -fPIC" FCOMMON_OPT="-O3 -g -fPIC -frecursive" NMAX="NUM_THREADS=128" LIBPREFIX="libopenblas" INTERFACE64=0 NO_STATIC=1 && \
$ make PREFIX=/usr install
Compile
Generate the go files from proto file
$ make check-proto-product
Check code specifications
$ make verifiers
Compile milvus
$ make milvus
Install docker-compose
refer: https://docs.docker.com/compose/install/
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
$ docker-compose --version
$ docker compose --version
Start service
Start third-party service:
$ cd [milvus project path]/deployments/docker/cluster
$ docker-compose up -d
$ docker compose up -d
Start milvus cluster:
$ cd [milvus project path]
$ ./scripts/start_cluster.sh
Run unittest
Run all unittest including go and cpp cases:
$ make unittest
You also can run go unittest only:
$ make test-go
Run cpp unittest only:
$ make test-cpp
Run code coverage
Run code coverage including go and cpp:
$ make codecov
You also can run go code coverage only:
$ make codecov-go
Run cpp code coverage only:
$ make codecov-cpp