## What this PR does
Removes two unused C++ third-party dependencies from `internal/core` to
reduce dependency count, comprehension cost, and link/build surface.
### rapidjson
Linked into `milvus_core` but has **zero `#include` usage** in the C++
sources (only a stale comment in `common/Json.h`). knowhere v3.0.3 and
milvus-storage do not depend on it. Removes the conan requirement,
`find_package(RapidJSON)`, and the link target.
### OpenDAL backend
Experimental `ChunkManager` backend gated behind `USE_OPENDAL`
(**default OFF**) — released binaries never built it. Removes
`OpenDALChunkManager`, the vendored OpenDAL Rust dependency, and all
`USE_OPENDAL` build plumbing (Makefile / core_build.sh /
3rdparty_build.sh / CMake). The Go-side `"opendal"` storageType value
remains a backward-compatible alias to the standard remote chunk
manager.
### Kept
`simde` is **not** removed — knowhere v3.0.3 requires it
(`find_package(simde REQUIRED)`, used in
`src/index/sparse/codec/varintdecode.c`).
issue: #50551🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Revert shell env inheritance of `CGO_LDFLAGS` / `CGO_CFLAGS` in
`Makefile` `build-go` and `milvus-gpu` targets. This disabled Go's
default `-O2 -g` CGo flags when both the shell env and the Makefile
variable are empty, compiling DataDog/zstd (indirect dep) and other CGo
code at `-O0 -g` and causing a ~77% regression in QueryNode segment load
wall time.
Users can still pass custom flags via `make CGO_LDFLAGS=... build-go`.
ASan builds remain unaffected.
issue: #49286
## Test plan
- [ ] Build succeeds with default flags
- [ ] Build succeeds with ASan (`USE_ASAN=ON make milvus`)
- [ ] Build succeeds with user-provided CGO flags (`make build-go
CGO_LDFLAGS=-L/some/path`)
- [ ] Segment load wall returns to pre-regression baseline
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
## Summary
Remove the entire SuffixSnapshot time-travel layer from RootCoord
metadata storage, replacing it with plain KV operations.
Replace SnapShotKV with TxnKV in Catalog (~30 call sites)
Delete suffix_snapshot.go (~750 lines) and tests (~917 lines)
Remove SnapShotKV interface from pkg/kv/kv.go
Remove snapshot GC config params (ttl, reserveTime)
Keep IsTombstone/ComposeSnapshotKey constants for migration tool
compatibility
Why: Investigation confirmed snapshot keys are write-only artifacts that
nobody reads:
No client/proxy sends non-zero timestamp for time-travel metadata
queries
Tombstone sweeper calls RemoveCollection(ts=0), bypassing snapshot
tombstone writes entirely
Snapshot keys accumulate as orphans — the old GC couldn't fully clean
them because ts=0 deletion path never wrote tombstone markers
Net reduction: -2180 lines
related: #48697
## Test plan
- [x] Unit tests for GC batch logic (expiry, cursor progress,
latest-per-group protection, tombstone cleanup)
- [x] Unit tests for Save/Load with time-travel verification
- [x] `go vet` passes on all changed packages
- [ ] CI passes
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gci misclassifies crypto/boring as a third-party package, but it is
actually a Go standard library package (available in BoringCrypto
builds). This conflicts with gofumpt which correctly identifies it as
standard, causing the two tools to produce different import groupings
and making lint-fix non-idempotent.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #48574
## Summary
- Upgrade Go from 1.24.12 to 1.25.8 across all go.mod files and
Dockerfiles
- Fixes CVE-2025-68121 (CRITICAL), CVE-2026-27142 (HIGH), CVE-2026-25679
(HIGH) in Go stdlib
- All three CVEs affect the Go standard library and are resolved by
upgrading to Go 1.25.8
## Changes
- Updated `go` directive in go.mod files (root, pkg/, client/,
tests/go_client/, examples/)
- Updated Go download URLs in Dockerfiles (build/docker/builder/)
- Updated `toolchain` directives where present
issue: https://github.com/milvus-io/milvus/issues/TBD
## Test plan
- [x] `make milvus` builds successfully with Go 1.25.8
- [ ] CI passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Add `--match 'v*'` to `git describe --exact-match --tags` in
`MILVUS_VERSION` calculation
- Prevents `pkg/v2.6.13` (Go submodule tag) from being picked instead of
`v2.6.13` on release builds
- Without this fix, `GetVersion` returns `pkg/v2.6.13` instead of
`2.6.13` because `sed 's/^v//'` doesn't strip the `pkg/` prefix
### Root Cause
The milvus repo uses Go multi-module: `pkg/` has its own `go.mod` with
tags like `pkg/v2.6.13`. When both `v2.6.13` and `pkg/v2.6.13` exist on
the same commit, `git describe --exact-match --tags` may return the
`pkg/` prefixed tag.
issue: #47823
## Test plan
- [ ] Verify on a tagged commit with both `v*` and `pkg/v*` tags: `git
describe --exact-match --tags --match 'v*'` returns only the root tag
- [ ] Dev builds unaffected (no exact tag match, falls through to
branch-date-commit format)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Layer 2 (OpenSSL FIPS) changes:
- Add programmatic FIPS activation via OSSL_LIB_CTX_load_config in
boring_enabled.go (gated by //go:build boringcrypto)
- Add openssl-fips.cnf with fips + default providers and
default_properties = fips=yes
- Use absolute .include path for fipsmodule.cnf — OpenSSL resolves
relative .include from the process working directory, not the config
file's directory, causing silent FIPS provider load failure
- Add RAND_bytes probe after config load to verify the FIPS provider is
truly functional (EVP_default_properties_is_fips_enabled only checks the
property string, not whether the provider loaded)
- Dockerfiles: add openssl fipsinstall + OPENSSL_MODULES env var
- Log OpenSSL FIPS status from C++ via
EVP_default_properties_is_fips_enabled
Layer 1 (Go BoringCrypto) changes:
- Add GOEXPERIMENT=boringcrypto build flag (conditional on
MILVUS_FIPS_ENABLED=ON)
- Add boringEnabled() build-tagged functions for startup logging
s2n-tls upgrade:
- Override s2n 1.4.1 (from aws-c-io) to 1.6.0 in conanfile.py. s2n 1.4.1
only detects FIPS via the legacy OPENSSL_FIPS define (not set by OpenSSL
3.x). s2n 1.6.0 adds EVP_default_properties_is_fips_enabled() detection
so s2n enters FIPS mode and uses RAND_bytes() through the FIPS provider.
See also: #48202, #48301
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Fix `GetVersion` returning `unknown-YYYYMMDD-` instead of
`master-YYYYMMDD-<commit>` when building inside containers (e.g. via
`builder.sh`)
- Replace `git config --global --add safe.directory '*'` with a
per-command `-c safe.directory='*'` approach that doesn't pollute
`~/.gitconfig`
- Remove stale `safe.directory` setup from `print-build-info` and
`print-gpu-build-info` targets
## Root Cause
PR #47822 introduced `MILVUS_VERSION` using `:=` (immediate evaluation).
The `safe.directory` config was set in `print-build-info` target (line
253), but `MILVUS_VERSION` on line 250 is evaluated first during
Makefile parsing. So all git commands (`GIT_BRANCH`, `GIT_COMMIT`) fail
silently inside the builder container where `.git` is owned by a
different user.
## Fix
Define `GIT := git -c safe.directory='*'` and use `$(GIT)` for all git
shell calls. This:
1. Ensures `safe.directory` is effective for every git command at parse
time
2. Never modifies `~/.gitconfig` (no `--global --add` accumulation)
3. Scopes the security bypass to this Makefile only, not all repos on
the machine
issue: #47822
## Test plan
- [ ] Build milvus inside builder container via `builder.sh`, verify
`GetVersion` returns `master-YYYYMMDD-<commit>`
- [ ] Verify `~/.gitconfig` is not modified after `make build-go`
- [ ] Run `make print-build-info` and confirm correct output
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Make `GetVersion` API return version strings consistent with Docker
image tags, replacing the hardcoded semver
- Assemble `MilvusVersion` in Makefile via `git describe --exact-match
--tags` (release) or `branch-date-commit` (dev), inject via ldflags
- Both `detail=False` (`GetVersion` RPC) and `detail=True` (`Connect`
RPC `server_info.build_tags`) return the new format
| Scenario | Before | After |
|----------|--------|-------|
| Release (tag v2.6.11) | `2.6.11` | `2.6.11` |
| Dev (master branch) | `2.6.11` | `master-20260224-2d14975d18` |
| Feature branch | `2.6.11` | `feature-xxx-20260224-2d14975d18` |
Banner still displays semver via `common.Version.String()` — unchanged.
issue: #47823
## Test plan
- [x] Makefile dry-run verified: `MilvusVersion` correctly injected in
all 4 ldflags targets
- [x] Tag scenario verified: `v2.6.11` → `2.6.11` (v prefix stripped)
- [x] Dev scenario verified:
`feature/getversion-docker-tag-format-20260224-2d14975d18`
- [x] Go syntax verified: `gofmt` and `go vet` pass
- [ ] Full `make build-go` (requires C++ env)
- [ ] E2E: `client.get_server_version()` and
`client.get_server_version(detail=True)`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
issue: #46481
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: DiskANN requires OS support for asynchronous I/O
(AIO); the Makefile now encodes this by defaulting disk_index=OFF on
Darwin (macOS) and disk_index=ON on other OSes where AIO is available.
- Simplified logic: the build-time default was inverted for non-Darwin
platforms so DiskANN is enabled by default; redundant conditional
handling that previously forced OFF everywhere has been removed in favor
of an OS-based default while preserving the single manual override
variable.
- No data loss or regression: this is a compile-time change only — it
toggles inclusion of DiskANN code paths at build time and does not
modify runtime persistence or existing index files. macOS builds still
skip AIO-dependent DiskANN code paths, and Linux/other builds merely
compile support by default; no migration or runtime data-path changes
are introduced.
- Backward compatibility / fix for issue #46481: addresses the reported
need to enable DiskANN by default (issue #46481) while keeping explicit
disk_index overrides intact for CI and developer workflows, so existing
build scripts and deployments that pass disk_index continue to behave
unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/44452
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
**Scalar Bench Decoupled from Milvus Build System**
- **Core assumption**: Scalar-bench is now managed as an independent
build artifact outside the milvus repository, eliminating the need for
conditional compilation integration within milvus's Makefile and
CMakeLists.txt.
- **Build infrastructure simplified**: Removed `scalar-bench` and
`scalar-bench-ui` targets from Makefile and deleted the entire
`ENABLE_SCALAR_BENCH` conditional block in
`internal/core/unittest/CMakeLists.txt` (which handled FetchContent,
cache variables, and subdirectory integration)—this eliminates optional,
redundant build-time coupling that is no longer necessary.
- **No regression introduced**: The removal only affects optional
build-time integration paths. Core C++ builds continue functioning as
before, and unit tests remain unaffected since `ENABLE_SCALAR_BENCH` was
always optional (not a required dependency); the newly added
`plan-parser-so` dependency on core build targets appears to be a
separate, required component.
- **Decoupling benefit**: Scalar-benchmark can now evolve and release on
its own schedule independent of milvus release cycles, while maintaining
clean separation of concerns between the two projects.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
### **Description**
- Add `-buildvcs=false` flag to Go test commands in coverage script
- Increase default session TTL from 10s to 15s
- Update SessionTTL parameter default value from 30 to 15
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: czs007 <zhenshan.cao@zilliz.com>
generated a library that wraps the go expr parser, and embedded that
into libmilvus-core.so
issue: https://github.com/milvus-io/milvus/issues/45702
see `internal/core/src/plan/milvus_plan_parser.h` for the exposed
interface
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced C++ API for plan parsing with schema registration and
expression parsing capabilities.
* Plan parser now available as shared libraries instead of a standalone
binary tool.
* **Refactor**
* Reorganized build system to produce shared library artifacts instead
of executable binaries.
* Build outputs relocated to standardized library and include
directories.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
Related to #44620
Related to unstable ut "internal/querycoordv2 TestServer/TestNodeUp"
Introduce SessionWatcher interface to fix race condition and goroutine
leak that caused unstable unit test TestServer/TestNodeUp.
Changes:
- Add SessionWatcher interface with EventChannel() and Stop() methods
- Refactor WatchServices() to return SessionWatcher instead of raw
channel
- Fix cleanup order in QueryCoordV2: stop watcher before session
- Update DataCoord, ConnectionManager to use SessionWatcher
- Add MockSessionWatcher for testing
Fixes race condition between session context cancellation and internal
loop exit. Eliminates goroutine leak by providing explicit lifecycle
management.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #44761#38339
This commit consolidates the proxy package's mockery generation to use a
centralized `.mockery.yaml` configuration file, aligning with the
pattern used by other packages like querycoordv2.
Changes
- **Makefile**: Replace multiple individual mockery commands with a
single config-based invocation for `generate-mockery-proxy` target
- **internal/proxy/.mockery.yaml**: Add mockery configuration defining
all mock interfaces for proxy and proxy/shardclient packages
- **Mock files**: Regenerate mocks using the new configuration:
- `mock_cache.go`: Clean up by removing unused interface methods
(credential, shard cache, policy methods)
- `shardclient/mock_lb_balancer.go`: Update type comments (nodeInfo →
NodeInfo)
- `shardclient/mock_lb_policy.go`: Update formatting
- `shardclient/mock_shardclient_manager.go`: Fix parameter naming
consistency (nodeInfo1 → nodeInfo)
- **task_search_test.go**: Remove obsolete mock expectations for
deprecated cache methods
Benefits
- Centralized mockery configuration for easier maintenance
- Consistent with other packages (querycoordv2, etc.)
- Cleaner mock interfaces by removing unused methods
- Better type consistency in generated mocks
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #43897
- add ddl messages proto and add some message utilities.
- support shard/exclusive resource-key-lock.
- add all ddl callbacks future into broadcast registry.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Add support for running mixcoord unit tests independently by:
- Adding test-mixcoord target to Makefile for easy test execution
- Adding test_mixcoord function to run_go_unittest.sh script
- Including mixcoord in test_all function for comprehensive testing
- Adding mixcoord case to test tag handling in script
This enables developers to run mixcoord tests specifically using: make
test-mixcoord
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See #42091
In this PR,
- Added a new `-t` option to specify the build type (Debug/Release).
- Updated the Makefile to pass the build type to the 3rdparty build
script.
- Included usage instructions for the script to improve user guidance.
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
issue: #40730
also see: https://github.com/milvus-io/cgosymbolizer/pull/2
After these PR, at linux:
- the milvus will always enable jemalloc by default.
- jemalloc will always compiled with --enable-prof options.
- all image will always enable the jemalloc prof by default.
- a pprof http service for jemalloc at `/debug/jemalloc/` will be
registered into restful.
- `jeprof` can remote profile the memory of milvus.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #33285
- move most cgo opeartions related to search/query into segcore package
for reusing for streamingnode.
- add go unittest for segcore operations.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #34553
when rootcoord trigger graceful stop progress, it will block until all
rpc finished. for create collection request, rootcoord need to block
until datacoord finish to watch all channels, but datacoord need to call
`rootcoord.Alloc` during watch channel, and rootcoord doesn't respond to
new request anymore. which cause create collection stucks, and graceful
stop progress stucks.
This PR remove the func call `rootcoord.Alloc` to solve the logic dead
lock during graceful stop progress.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
