## Summary
Keep raw search distances through segment search, iterator output, index
query, and chunk merging so ranking/reduce order is not affected by
`round_decimal`.
Apply `round_decimal` only when producing the final search response,
after reduce/rerank/order-by has already determined result order.
Returned scores keep the requested precision without changing TopK
selection.
issue: #50347
---------
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- Upgrade Knowhere dependency to 49507ef3 to pick up the fix that
removes the num_build_thread upper-bound validation.
issue: #42937
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
Reject search iterator v2 for hybrid search because the current response
metadata model only supports iterator v2 results for a single query
info. This avoids returning hybrid search hits with empty iterator
metadata that cannot be used for the next page.
issue: #50407
## Test Plan
- [x] `git diff --check`
- [x] `make milvus`
- [x] `MILVUSCONF=/tmp/milvus_issue-50407/configs
PKG_CONFIG_PATH=$(pwd)/internal/core/output/lib/pkgconfig
LD_LIBRARY_PATH=$(pwd)/internal/core/output/lib go test -gcflags="all=-N
-l" -tags dynamic,test ./internal/proxy -run
"^TestSearchTask_ArrayOfVectorHybridSearch$" -count=1 -ldflags="-r
$(pwd)/internal/core/output/lib"`
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
## Summary
- Upgrade Knowhere dependency to v3.0.4.
- Upgrade milvus-common dependency to the version required by Knowhere
v3.0.4.
issue: #50610
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
## Summary
When a QueryNode does not report `MaximumIndexVersion`, use its
`CurrentIndexVersion` as the conservative maximum for cluster-wide
vector index version resolution. This prevents `targetVecIndexVersion`
from advancing beyond old QueryNodes during rolling upgrade.
issue: #48470
## Tests
- `source ./scripts/setenv.sh && make SKIP_3RDPARTY=1
build-cpp-with-unittest`
- `source ./scripts/setenv.sh && make build-go`
- `source ./scripts/setenv.sh && LOCAL_STORAGE_SIZE=100 go test -v
-gcflags="all=-N -l" -tags dynamic,test ./internal/datacoord -run
"Test_IndexEngineVersionManager_(GetMaximumIndexEngineVersion|ResolveVecIndexVersion)"
-count=1 -ldflags="-r ${RPATH}"`
- `LOCAL_STORAGE_SIZE=100 ./scripts/run_go_unittest.sh -t datacoord`
---------
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
## Summary
Add E2E coverage for vanilla FAISS index support across Milvus clients:
- Add Python client index tests for vanilla FAISS factory strings,
metrics, vector types, search params, scalar filters,
release/load/search, range search, and expected unsupported behavior
- Add Go client coverage through the generic index API
- Add REST client create/list/describe metadata coverage
Dependency note: this PR is temporarily based on Milvus #49912 and
currently depends on the Knowhere `faiss-passthrough` branch. The
dependency will be replaced after the final Knowhere reference is
available.
issue: #50124
## Tests
- python -m py_compile
tests/python_client/testcases/indexes/idx_faiss.py
tests/python_client/testcases/indexes/test_faiss.py
tests/restful_client_v2/testcases/test_index_operation.py
- pytest -c /dev/null
tests/python_client/testcases/indexes/test_faiss.py -q -o
cache_dir=/tmp/pytest-cache-vanilla-faiss --tb=short
- pytest tests/restful_client_v2/testcases/test_index_operation.py -q -k
faiss --tb=short -o cache_dir=/tmp/pytest-cache-vanilla-faiss-rest
- cd tests/go_client && go test ./testcases -run
TestCreateIndexVanillaFaissGeneric -count=1 -v
---------
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
## Summary
Disable Knowhere search-pool scheduling for vector iterators created by
Milvus. Milvus consumes these iterators from its own execution threads,
and the extra Knowhere thread control adds per-Next scheduling overhead
for iterator-heavy group-by search paths.
issue: #42132
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #42937
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: callers must explicitly close output streams (call
Close()) instead of relying on RemoteOutputStream's destructor to
perform closure.
- Logic removed/simplified: RemoteOutputStream's destructor no longer
closes or asserts on the underlying arrow::io::OutputStream; an explicit
public Close() method was added and closure responsibility moved to that
code path.
- Why this is safe (no data loss/regression): callers now invoke Close()
before reading or destroying streams (e.g.,
DiskFileManagerTest::ReadAndWriteWithStream calls os->Close() before
opening the input stream). Write paths remain unchanged
(RemoteOutputStream::Write -> output_stream_->Write), and Close invokes
output_stream_->Close() with status assertion, ensuring
flush/confirmation via the same API and preserving data integrity;
removing destructor-side asserts prevents unexpected failures during
object destruction without changing write/close semantics.
- Chore: updated third-party pins — internal/core/thirdparty/knowhere
CMakeLists.txt: KNOWHERE_VERSION -> a59816e;
internal/core/thirdparty/milvus-common CMakeLists.txt:
MILVUS-COMMON-VERSION -> b6629f7.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #46481
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: DiskANN requires OS support for asynchronous I/O
(AIO); the Makefile now encodes this by defaulting disk_index=OFF on
Darwin (macOS) and disk_index=ON on other OSes where AIO is available.
- Simplified logic: the build-time default was inverted for non-Darwin
platforms so DiskANN is enabled by default; redundant conditional
handling that previously forced OFF everywhere has been removed in favor
of an OS-based default while preserving the single manual override
variable.
- No data loss or regression: this is a compile-time change only — it
toggles inclusion of DiskANN code paths at build time and does not
modify runtime persistence or existing index files. macOS builds still
skip AIO-dependent DiskANN code paths, and Linux/other builds merely
compile support by default; no migration or runtime data-path changes
are introduced.
- Backward compatibility / fix for issue #46481: addresses the reported
need to enable DiskANN by default (issue #46481) while keeping explicit
disk_index overrides intact for CI and developer workflows, so existing
build scripts and deployments that pass disk_index continue to behave
unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #46349
When using brute-force search, the iterator results from multiple chunks
are merged; at that point, we need to pay attention to how the metric
affects result ranking.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #45597
The streaming node currently cannot use GPU resources and does not need
to perform initialization.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #44746
Support modifying vector index behavior during loading by change/add
aram knowhere.xxx.load.xxx.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #38666
Add int8 support for autoindex to ensure it can be independently
configured. At the same time, remove the restriction on int8 type for
vectorDiskIndex (note that vectorDiskIndex only determines the building
and loading method of the index, not the index type).
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #42028
W20250522 09:52:55.785657 12779 ChunkedSegmentSealedImpl.cpp:1752]
[SERVER][generate_interim_index][CGO_LOAD][]fail to generate binlog
index, because bad optional access
After the cachelayer is added, num_rows_ can not be obtained before
interim index generated , and an external parameter pass is required
Signed-off-by: foxspy <xianliang.li@zilliz.com>
issue: #39090
The num_build_thread parameter will limit the number of build omps. This
parameter will override the effect of buildIndexThreadPoolRatio.
Removing this parameter will have no actual effect. This parameter is
actually only used in the growing index, where it will be explicitly
set.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #34298
Viper uses yaml.v2 as the parser. This PR will adopt the parsing logic
from Viper to handle YAML files, ensuring maximum consistency in
parsing.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #34298
fix key: null defined in the yaml file.
viper will parse it as "", and yaml v3 will parse it as "null".
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
issue: #34298
because all vector index config checker has been moved into
vector_index_checker, then the useless checkers can be removed.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
add prometheus dependency for monitor module. Or else Some compilers may
report a compilation failure.
issue: #35077
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
add scalar filtering and vector search latency metrics to distinguish
the cost of scalar filtering.
To add metrics in query chain, add a monitor module and move the metric
files from original storage module.
issue: #34780
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
patch search cache param from index configs when index meta could not
get the search cache size key
#issue: #30113
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
related to : #29417
cardinal indexes upload index files in `Serialize` interface, and throw
exception when the `Serialize` failed.
Signed-off-by: xianliang <xianliang.li@zilliz.com>
