mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
master
2374
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 zhuwenxingandGitHub
|
743da442b1 |
test: add XGBoost FunctionChain coverage (#51545)
## What changed - Add Python MilvusClient coverage for XGBoost `FunctionChainStage.L0_RERANK`. - Validate realistic `2 x 3000` indexed sealed segments plus `1 x 3000` growing segment data topology. - Cover FLAT, HNSW, and DISKANN indexes; filtered, range, group-by, iterator, and hybrid-search behavior. - Verify score and field correctness across numeric/null/binary features, delete, and upsert mutations. - Cover invalid stage/model contracts, concurrency, scale, cache identity, restart/failover, rolling update, and chaos scenarios. - Classify the suite into L0/L1/L2/L3 and replace the iterator xfail with its code `1100` error contract. Related issues: #51306, #51310, #51332 ## Validation - `ruff format --check tests/python_client/milvus_client/test_milvus_client_function_chain.py` - `ruff check tests/python_client/milvus_client/test_milvus_client_function_chain.py` - `python -m py_compile tests/python_client/milvus_client/test_milvus_client_function_chain.py` - `pytest --collect-only`: 37 cases collected - L0: 1 passed - L1: 8 passed - L2: 4 passed - Iterator regression: 1 passed against master image `0671607` An initial combined serial run observed a DISKANN load timeout with a loaded segment reporting `index_id=0`; the unchanged isolated L2 rerun passed. The strict assertion remains in place. --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
|
82bb4054dd |
test: Add REST query order by coverage (#51460)
## What this PR does
- Adds REST v2 query coverage for orderByFields across ascending,
descending, multi-field, filtering, pagination, nullable fields, and
invalid parameters.
- Uses class-scoped REST clients and shared collection setup consistent
with the existing REST v2 test pattern.
- Verifies that omitting limit applies the REST default of 100 rows.
- Covers duplicate-key pagination behavior and exact validation error
code 1100.
## Test
Against schema-evolution-v3-latest through port-forward:
cd tests/restful_client_v2
../python_client/.venv/bin/python -m pytest
testcases/test_query_order_by.py -v --tb=short -p no:rerunfailures
--endpoint http://127.0.0.1:19530 --token root:Milvus
Result: 16 passed, 16 warnings in 40.87s.
---------
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
||
|
zhuwenxingandGitHub
|
977a27fb47 |
test: add struct array null semantics and lifecycle coverage (#51524)
issue: #51381 related: #51414 related: #51416 ### What this PR does - Adds Python client E2E coverage for nullable Struct Array NULL-versus-empty semantics. - Covers compaction, snapshot restoration, partial updates, drop and re-add isolation, imports, regex indexes, aggregation, and dynamic fields. - Adds exact ANN oracles for Struct Array element and embedding-list searches without assuming self-hit behavior. - Covers hybrid search collapse strategies and element identity preservation. - Marks known regressions with strict xfail markers linked to their corresponding issues while keeping unaffected parameters active. ### Why Struct Array rows can represent NULL, empty arrays, and non-empty arrays. These states affect predicates, nested indexes, element offsets, and vector search candidates differently. The added coverage protects these semantics across write, reload, compaction, import, and snapshot lifecycle paths. ### Validation - `ruff check` passed for all changed Python files. - `ruff format --check` passed for all changed Python files. - Python compilation passed for all changed Python files. - Pytest collection passed. - The nine known regression nodes executed as expected: `9 xfailed`. - The three unaffected parameterized cases passed. - Targeted E2E validation used Milvus master build `09c44f2dbb`. --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
Spade AandGitHub
|
149903e0fd |
fix: fix three-values nullbility and race in growing segmentArrayOffets (#51415)
issue: https://github.com/milvus-io/milvus/issues/51381 --------- Signed-off-by: SpadeA <tangchenjie1210@gmail.com> Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> |
||
|
|
5def5ced4a |
enhance: enforce function-field binding principle for function DDL (#51360)
## What Enforce a function–field binding principle for function DDL (add / drop / alter function on an existing collection), so a function and its output field stay coupled and already-stored vectors can never be silently invalidated by DDL. BM25 and MinHash follow the strict binding: add/drop only via the unified `add_function_field` / `drop_function_field`, output field always coupled. **TextEmbedding is a temporary carve-out** — its legacy `AddCollectionFunction` / `DropCollectionFunction` paths are retained, so attach-over-existing-field and detach are still possible for it, because `add_function_field` embedding backfill is not yet implemented. It will be folded into the unified path in a follow-up. So the "always coupled" invariant currently holds for BM25/MinHash, not TextEmbedding. ## Changes - **AlterCollectionSchema invariant**: reject standalone add-function (a function must be added together with its new output field), reject detaching a function without dropping its output field, and reject function cascade (a new function's input being another function's output). - **Legacy RPCs**: `AddCollectionFunction` / `DropCollectionFunction` are rejected for BM25/MinHash (must use `add_function_field` / `drop_function_field`) and retained only for TextEmbedding (see carve-out). `AlterCollectionFunction` is kept and gains the whitelist below. In the alter and legacy-add paths, function field IDs are always re-derived from field names (never trusted from the request), so a request cannot inject an unrelated field ID that a later `drop_function_field` would delete. - **alter_function whitelist**: only connection/runtime params may change (TextEmbedding: `url`, `credential`, `timeout_ms`, `max_client_batch_size`, `region`, `location`, `projectid`, `user`); BM25/MinHash have no alterable params. Function identity (type, name, input/output fields) and output-shaping params (`dim`, `model_name`, `endpoint` — the TEI model identity, `normalize`, `truncate*`, prompts, ...) are immutable. Params are normalized (keys lowercased, duplicate keys rejected) so a crafted duplicate key cannot bypass the diff. - **drop_function_field**: allow any output-producing function (dropping needs no backfill), removing the previous BM25/MinHash-only restriction. ## Not in scope / follow-up - Extending `add_function_field` to TextEmbedding (post-creation add would require backfilling every existing row through the external embedding model) — deferred; folding the TextEmbedding legacy paths into the unified binding follows this. - MinHash input-field analyzer immutability lives on a different DDL path (`AlterCollectionField`); tracked as a separate follow-up. ## Breaking changes - `DropCollectionFunction` on a **MinHash** function (detach — remove the function but keep its output field) is no longer supported; use `drop_function_field` instead (drops the function together with its output field). If a released version supported MinHash detach, migrate any such call to `drop_function_field`. issue: #51348 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
224f4ab7e9 |
fix: use correct MinIO root path in text LOB layout test (#51535)
## What this PR does - Change the default `MILVUS_MINIO_ROOT_PATH` from `files` to `file` in the Text LOB inline/object layout test. - Keep the environment-variable override available for deployments using a custom root path. issue: #51257 ## Verification - `python3 -m py_compile tests/python_client/milvus_client/test_milvus_client_text_lob.py` - Targeted pytest node collection completed successfully. - Verified the committed diff contains only the requested one-line default change. Signed-off-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Eric Hou <eric.hou@zilliz.com> |
||
|
zhuwenxingandGitHub
|
43264fc57a |
test: use container kill for CDC failover scenarios (#51479)
## What changed - Replace Pod deletion with Chaos Mesh `container-kill` in CDC force-promote and source-down failover tests. - Kill every regular container in matching Pods while preserving the Pod objects. - Verify that Pod UIDs remain unchanged and each container's restart count increases. - Clean up the generated `PodChaos` resources after fault injection. ## Why Deleting Pods changes their identity and tests Pod recreation rather than container-level recovery. These scenarios need to verify CDC failover behavior when containers restart inside the existing Pods. ## Impact CDC failover tests now exercise container failures while explicitly validating that Kubernetes does not recreate the affected Pods. ## Validation - `ruff check` passed for all modified files. - `ruff format --check` passed for all modified files. - Python compilation passed for all modified files. - Full CDC E2E was not run locally because it requires a Kubernetes environment with Chaos Mesh. --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
zhuwenxingandGitHub
|
0fe1a5ba6f |
test: improve force merge target size coverage (#51350)
issue: #51248 ## What changed - correct the omitted-`target_size` case to assert ordinary manual compaction semantics - add a physical Force Merge target-size case using persistent segment IDs and MinIO insert-log sizes - add deterministic Go boundary coverage for grouping selection at the configured threshold and threshold + 1 - add target-size memory clamp coverage for cluster and standalone co-location modes - extend Optimize format, signed-int64 boundary, loaded-segment refresh, and live async lifecycle coverage - add the async Optimize wrapper and update the PyMilvus test dependency to 3.1.0rc64 ## Why The previous cases did not prove that an explicit Force Merge target affected physical output, assumed the wrong grouping threshold, and depended on manual log inspection for algorithm selection. Loaded refresh and the public async Optimize workflow also lacked live end-to-end coverage. The async case exposed milvus-io/pymilvus#3680 on PyMilvus 3.1.0rc62. Version 3.1.0rc64 contains the tuple-unpacking fix and passes the unmodified test. ## Validation - all 10 planned case IDs pass; 0 failed and 0 blocked - `milvus-dev-cli` Go UT job `go-ut-local-zhuwenxi-zhuwenxing-i-4082040-30413568` succeeded; both test functions and all five subtests passed - physical target-size L3 case passed against a real cluster and MinIO in 200.68s - loaded refresh, target format/boundary, and manual compaction cases passed against the same server version - async Optimize L3 passed unmodified with PyMilvus 3.1.0rc64 in 56.91s - 56 Python nodes collect successfully; Python compilation, Ruff, and `git diff --check` pass --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
|
c322c84c22 |
test: cover add-field default values on growing segments (#51303)
## What this PR does - Remove the `xfail` marker from the existing drop-and-readd analyzer-field regression for #50484. - Add L1 E2E coverage for the live QueryNode growing-segment reopen path when an `enable_match` VARCHAR field with `default_value` is added to loaded growing data. - Verify the default value is immediately searchable on pre-existing unflushed rows, eventually searchable on a subsequent defaulted insert after the growing text index catches up, and remains searchable after flush/reload. Related fix: #51201 issue: #50484 ## Scope note This PR validates the live reopen path: ```text loaded collection -> insert rows that stay in a QueryNode growing segment -> add an `enable_match` VARCHAR field with `default_value` -> query triggers LazyCheckSchema/Reopen -> default values for pre-existing growing rows are indexed and searchable immediately ``` It does not cover the QueryNode recovery / LoadGrowing path. That path requires forcing a QueryNode restart, replacement, or channel rewatch while the data is still growing: the new QueryNode can create the segment with the latest schema and then load old binlogs written before AddField. Since the segment schema is already current in that path, LazyCheckSchema/Reopen is not the mechanism being tested here. Recovery coverage should be handled by a focused stability test. ## Test results - `test_milvus_client_add_match_field_with_default_value_on_growing_data`: 3/3 passed - `test_drop_then_add_same_name_analyzer_field`: passed - Ruff lint and format checks: passed - `git diff --check`: passed Signed-off-by: lyyyuna <yiyang.li@zilliz.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
zhuwenxingandGitHub
|
cdd9136fd7 |
test: enable FileResource regression cases for closed issues (#51239)
## What changed - Remove the stale `xfail` marker from `test_file_with_bom_and_crlf`, which covers the UTF-8 BOM regression tracked by [#49684](https://github.com/milvus-io/milvus/issues/49684) and fixed by [#49691](https://github.com/milvus-io/milvus/pull/49691). - Remove the stale `skip` marker from `test_drop_collection_then_remove_remote_resources_no_panic`, which covers the FileResource lifecycle panic tracked by [#49279](https://github.com/milvus-io/milvus/issues/49279) and fixed by [#49412](https://github.com/milvus-io/milvus/pull/49412). Both issues are closed. This change makes the existing cases active CI regressions; it does not change their fixtures, test bodies, assertions, or server behavior. ## User impact CI will now detect regressions where stop-word files with a UTF-8 BOM are parsed incorrectly, or where dropping a collection and removing its remote analyzer resources can panic the server. ## Validation The existing cases were previously run against the latest `upstream/master` commit `03762320e8537f855c18c28133834bf292b45361` using image `harbor.milvus.io/manta/milvus:master-20260710-0376232`. The `-p no:skipping` option only bypassed pytest's skip/xfail marker handling; no case code, fixture, or assertion was changed: ```bash python -W ignore -m pytest -v -p no:skipping \ milvus_client/test_milvus_client_file_resource.py::TestMilvusClientFileResourceContent::test_file_with_bom_and_crlf \ milvus_client/test_milvus_client_file_resource.py::TestMilvusClientFileResourceLifecycleAdvanced::test_drop_collection_then_remove_remote_resources_no_panic \ --host <temporary-milvus-host> --port 19530 --token 'root:Milvus' \ --minio_host <temporary-minio-host> --minio_bucket fr-closed-issues-master \ --tb=short ``` Result: `2 passed in 2.16s`. The #49279 case was then repeated ten times with the same marker bypass: ```bash python -W ignore -m pytest -q -p no:skipping \ milvus_client/test_milvus_client_file_resource.py::TestMilvusClientFileResourceLifecycleAdvanced::test_drop_collection_then_remove_remote_resources_no_panic \ --count=10 \ --host <temporary-milvus-host> --port 19530 --token 'root:Milvus' \ --minio_host <temporary-minio-host> --minio_bucket fr-closed-issues-master \ --tb=short ``` Result: `10 passed in 14.59s`. Milvus remained healthy, pod restart count was zero, and logs contained no panic, `tokenizer.h:31`, fatal error, or segmentation fault. The temporary validation instance has been cleaned up. The two node IDs also collect successfully after removing the markers: ```bash python -W ignore -m pytest --collect-only -q \ milvus_client/test_milvus_client_file_resource.py::TestMilvusClientFileResourceContent::test_file_with_bom_and_crlf \ milvus_client/test_milvus_client_file_resource.py::TestMilvusClientFileResourceLifecycleAdvanced::test_drop_collection_then_remove_remote_resources_no_panic ``` Result: `2 tests collected in 0.09s`. Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
zhuwenxingandGitHub
|
408204099e |
test: add struct array partial update coverage (#51166)
## What - Enable StructArray coverage in `PartialUpdateChecker` and verify StructArray-only partial update rows. - Add MilvusClient E2E coverage for StructArray partial update over sealed and growing data, including HNSW and DISKANN embedding-list search validation. - Bump Python client test dependency to `pymilvus==3.1.0rc62`. ## Verification - `python3 -m py_compile tests/python_client/chaos/checker.py tests/python_client/milvus_client/test_milvus_client_partial_update.py` - `ruff check tests/python_client/chaos/checker.py` - `ruff check --select E9,F63,F7,F82 tests/python_client/milvus_client/test_milvus_client_partial_update.py` - `git diff --check` - Targeted live run against 2.6-latest: `test_milvus_client_partial_update_struct_array_sealed_growing` passed for HNSW and DISKANN. --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
junjiejiangjjjandGitHub
|
694c2e6dba |
feat: support xgboost function chain expr (#51195)
issue: #51192 design doc: docs/design-docs/design_docs/20260708-xgboost-function-chain.md Add native xgboost FunctionChain expression support for L0 rerank with FileResource-backed UBJ models. This change includes: - xgboost FunctionChain expression registration, parameter validation, and execution - FileResource-based UBJ model discovery and local path resolution - lazy model loading with singleflight, lease/refcount lifecycle protection, and stale eviction on FileResource sync - cgo bridge for Arrow C Data based batch prediction - native C++ UBJ model parser and predictor for supported tree models - runtime-disabled stub for builds without cgo and with_xgboost - validation for unsupported params, output modes, feature count mismatch, invalid models, unsupported objectives, unsupported boosters, multiclass models, multi-target leaf vectors, and unsupported input column types - C++ unit tests, Go tests, native parity tests, and Python client L0 E2E tests - xgboost FunctionChain design document L2 rerank support is intentionally deferred because Proxy does not yet support FileResource sync and local resolution. Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com> |
||
|
wei liuandGitHub
|
536ece6737 |
fix: Check external fields against loaded manifest (#50773)
issue: https://github.com/milvus-io/milvus/issues/50416 ## What This PR moves external field checks into the C++ Search/Retrieve path after `LazyCheckSchema` has refreshed the segment schema and load info. Instead of probing `HasFieldData` or `FieldAccessible`, the guard checks whether every referenced external field exists in the currently loaded external manifest: - `CheckExternalFieldsInLoadedSnapshot` is renamed to `CheckExternalFieldsInLoadedManifest` to match the real readiness boundary. - Plan `access_entries_` now includes expression/execution fields and requested output fields, so Search and Retrieve use one field-reference list for external manifest checks. - Search filter-only skips the vector field and output target fields because that path does not access them. - Non-manifest segments keep the existing behavior. The loaded manifest column-group cache is also preserved across schema-only reopen when the manifest path is unchanged, so the query path can check manifest membership without parsing remote manifests or falsely reporting missing columns after schema refresh. ## Why For external collections, a newly added field can appear in the Go collection schema before the loaded C++ segment has refreshed to a manifest that contains the corresponding external column. If Search/Retrieve touches that field against the stale loaded manifest, segcore can surface an internal field-state failure instead of a clear external-field-not-ready error. The real readiness condition here is whether the loaded manifest contains the external column, not whether the field is loaded into memory. ## Validation - `ninja -C cmake_build milvus_core all_tests` - `DYLD_LIBRARY_PATH=cmake_build/src cmake_build/unittest/all_tests --gtest_filter='PlanProto.SearchPlanCollectsFieldAccessInfo:PlanProto.RetrievePlanCollectsFieldAccessInfo:SegmentLoadInfoTest.HasManifestColumnUsesManifestColumnNames:SegmentLoadInfoTest.CachedManifestColumnsCanBeInherited'` - `/opt/homebrew/opt/llvm@15/bin/clang-format --dry-run -Werror internal/core/src/query/PlanImpl.h internal/core/src/query/PlanProto.cpp internal/core/src/query/PlanProtoTest.cpp internal/core/src/segcore/ChunkedSegmentSealedImpl.cpp internal/core/src/segcore/ChunkedSegmentSealedImpl.h internal/core/src/segcore/SegmentInterface.h internal/core/src/segcore/SegmentLoadInfo.cpp internal/core/src/segcore/SegmentLoadInfo.h internal/core/src/segcore/SegmentLoadInfoTest.cpp internal/core/src/segcore/segment_c.cpp` - `CLANG_FORMAT=/opt/homebrew/opt/llvm@15/bin/clang-format make cppcheck` - `git diff --check milvus/master...HEAD` --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com> |
||
|
yanliang567andGitHub
|
b542464f30 |
test: Add JSON filtering e2e coverage (#51171)
## What does this PR do? Adds Python e2e coverage for JSON filtering UNKNOWN semantics in `tests/python_client/milvus_client/expressions/test_milvus_client_json_filtering.py`. This PR covers: - raw filtering paths on growing and sealed segments - JSON arithmetic/filtering UNKNOWN behavior for missing, null, and type-mismatch values - JSON array predicate behavior for missing and non-array values - ARRAY subscript UNKNOWN behavior for null, empty, and out-of-range values - contradiction rewrite cases, including outer `not (...)`, to ensure UNKNOWN is not rewritten into TRUE - JSON path index filtering with `json_cast_type=DOUBLE` - JSON path index filtering with `json_cast_type=VARCHAR` - JSON flat index coverage for currently passing UNKNOWN cases - query/search filter consistency for representative JSON filters, including indexed collections - L2 coverage for boolean JSON values, fractional double values, additional comparison/logical operators, JSON path array subscripts, mixed arrays, and `ARRAY_DOUBLE` JSON path index The known JSON flat index array-vs-scalar comparison mismatch is documented in #51193 and covered by a skipped regression case so the coverage can land first without taking server-side fixes in this PR. issue: #50976 issue: #51193 Related #50699 #50698 ## Test Plan - [x] `git diff --check` - result: passed - [x] `uvx ruff check --config tests/ruff.toml tests/python_client/milvus_client/expressions/test_milvus_client_json_filtering.py` - result: passed - [x] `uvx ruff format --check --config tests/ruff.toml tests/python_client/milvus_client/expressions/test_milvus_client_json_filtering.py` - result: passed - [x] `/Users/yanliang.qiao/fork/milvus/tests/.venv/bin/python -W ignore -m pytest milvus_client/expressions/test_milvus_client_json_filtering.py --collect-only -q` - result: `29 tests collected` - [x] `milvus-dev-cli deploy create harbor.milvus.io/manta/milvus:master-20260709-9a0d7f2 --name json-filtering-l2-9a0d7f2 --namespace chaos-testing --no-auth --wait-timeout 1800 --wait-interval 10` - result: Healthy - endpoint: `lb-6d57ndgl-sw4giqjhodl3bn15.clb.bj-tencentclb.net:19530` - [x] `/Users/yanliang.qiao/fork/milvus/tests/.venv/bin/python -W ignore -m pytest milvus_client/expressions/test_milvus_client_json_filtering.py --uri http://lb-6d57ndgl-sw4giqjhodl3bn15.clb.bj-tencentclb.net:19530 --token "" -s -v --tb=short` - result: `28 passed, 1 skipped in 215.82s` ## Notes The skipped case is `test_json_flat_index_array_scalar_comparison_unknown_semantics_query`, linked to #51193. The skip can be removed after the server-side JSON flat index UNKNOWN semantics issue is fixed. Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com> |
||
|
|
aea5d701bd |
enhance: bind index meta to add function field DDL (#51105)
issue: #51104 companion SDK PR (merge first): milvus-io/pymilvus#3670 ## Summary `add_function_field` previously accepted a vector output field without any index meta; bump-schema-version compaction then stalled forever (no segment index task can be created for a field with no index defined, and `GetQueryVChanPositions` never hands off unindexed compacted-to segments — the fail-closed chain is silent). This PR makes the index meta a mandatory, atomically-bound part of the DDL: - **Mandatory at every layer**: SDK requires `index_params`; proxy and rootcoord reject vector function-output fields without an explicit `index_type` strictly before the broadcast (no silent AUTOINDEX). - **create_index-aligned materialization**: proxy normalizes the params with the same logic as `create_index` (name rules via `validateIndexName`, checker existence, `CheckTrain` incl. dimension filling, function-type defaults such as `bm25_k1/bm25_b/bm25_avgdl`) and writes them back into the request; rootcoord prepare allocates index id/name, rejects name conflicts and unknown index types, and serializes a complete `indexpb.FieldIndex` into the new `AlterCollectionMessageUpdates.bound_field_indexes` WAL field. - **Atomic apply**: the alter-collection ack callback commits the schema and then applies the bound index by dispatching a synthetic `CreateIndexMessage` to datacoord's existing `createIndexV2AckCallback` (same pattern as `cascadeDropFieldIndexesInline`). The callback is replayed until success across crashes and is fully idempotent (all ids come from the message body), so `DDL success ⇒ index meta exists` holds with no partial terminal state. - **Reuse promotion**: `ValidateIndexParams`/`CheckDuplidateKey` moved from datacoord to `internal/util/indexparamcheck`; shared `ExpandIndexParams` / `FillFunctionOutputIndexParams` / `PrepareFunctionOutputIndexParams` helpers now back both the create_index path and this DDL. - **Dead fan-out removed**: the ordinary `CreateIndex` broadcast is reverted to control-channel-only; the vchannel fan-out had no consumer and wrote one dead WAL entry per vchannel per index creation. - **No milvus-proto change**: the request already carried `FieldInfo.index_name`/`extra_params` (previously unused); this feature activates them. - e2e suites updated to the new semantics: `test_add_function_field_feature.py` rewritten (explicit `index_params` everywhere, post-DDL `create_index` blocks removed since the bound index conflicts with a second distinct index per existing semantics, new negative cases for the mandatory checks); the external-table negative case updated as well. ## Verification - New/updated unit tests: rootcoord DDL callbacks (mandatory/AUTOINDEX/unknown-index-type rejections, bound-index materialization + ack-callback application via a recording fake), proxy task validation (invalid index name, normalization write-back), shared helper tests. - End-to-end on a live cluster (StorageV3 + bump compaction enabled): request without params rejected; index meta visible with create_index-aligned params immediately after the DDL returns; bump compaction rewrote 2000-row sealed segments and the bound index built on the results (the exact chain that previously stalled); BM25 search over pre-DDL rows succeeds once the compacted segments are loaded. ## Limitations (follow-ups) - Live propagation of the new index meta to already-loaded QueryNode delegators is intentionally out of scope; a loaded collection observes the bound index through the segment load path (e.g. reload / compacted-segment load). Tracked as a separate follow-up PR. - The python e2e suites require the companion pymilvus change: milvus-io/pymilvus#3670 is merged and `tests/python_client/requirements.txt` is bumped to `pymilvus==3.1.0rc61` in this PR (all 22 rewritten cases + the external-table case verified locally against the released rc61). - During a rolling-upgrade window an old coordinator replaying the new message ignores `bound_field_indexes` (proto3 unknown field), i.e. pre-existing behavior; avoid the new argument in mixed-version clusters. - `AddCollectionFunction` on an existing field and plain add-vector-field keep their current behavior (the WAL carrier is generic for future extension). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
yanliang567andGitHub
|
e0857f1bd9 |
fix: Validate REST quick-create enum fields (#51088)
## What
Validate REST v2 quick-create enum-like fields before creating the
collection.
This PR fixes `vectorFieldType` handling for quick collection creation:
- rejects invalid `vectorFieldType` values instead of silently using
`FloatVector`
- supports `FloatVector`, `BinaryVector`, `Float16Vector`,
`BFloat16Vector`, and `SparseFloatVector`
- defaults quick-create index metric by vector type: dense `COSINE`,
binary `HAMMING`, sparse `IP`
- validates explicit binary/sparse `metricType` before collection
creation
- rejects `SparseFloatVector` quick-create with `dimension`
It also fixes top-level quick-create `consistencyLevel` handling:
- parses and validates top-level `consistencyLevel`
- preserves existing `params.consistencyLevel` behavior
- rejects conflicting top-level and `params.consistencyLevel` values
Fixes #51085
Fixes #51084
## Why
Previously, `/v2/vectordb/collections/create` ignored top-level
`vectorFieldType` and always generated a `FloatVector` schema in
quick-create mode. A typo such as `InvalidVectorType` could return
success and mask client-side bugs.
Binary and sparse quick-create requests without `metricType` could also
create the collection first and then fail index creation because the old
fallback metric was `COSINE`, which is incompatible with those vector
types.
For #51084, top-level `consistencyLevel` was not present in
`CollectionReq`, so JSON decoding silently dropped it. Invalid values
such as `"consistencyLevel": "Invalid"` were ignored and the collection
could be created with the default consistency level.
## Tests
- `./bin/gofumpt -w internal/distributed/proxy/httpserver/request_v2.go
internal/distributed/proxy/httpserver/handler_v2.go
internal/distributed/proxy/httpserver/handler_v2_test.go`
- `./bin/gci write internal/distributed/proxy/httpserver/request_v2.go
internal/distributed/proxy/httpserver/handler_v2.go
internal/distributed/proxy/httpserver/handler_v2_test.go
--skip-generated -s standard -s default -s
"prefix(github.com/milvus-io)" --custom-order`
- `git diff --check`
- `python3 -m py_compile
tests/restful_client_v2/testcases/test_collection_operations.py`
- `rg -n
"string\\(paramtable\\.(BinaryVectorDefaultMetricType|SparseFloatVectorDefaultMetricType)\\)|interface\\("
internal/distributed/proxy/httpserver/handler_v2.go
internal/distributed/proxy/httpserver/handler_v2_test.go`
Attempted locally but blocked by missing native dependencies:
```
go test ./internal/distributed/proxy/httpserver -run 'TestCreateCollection(QuickVectorFieldType|TopLevelConsistencyLevel)$' -count=1
# missing rocksdb.pc and milvus_core.pc
```
CI investigation before this push:
```
# build-ut-cov/code-check failed on c6 predecessor because of unconvert:
# internal/distributed/proxy/httpserver/handler_v2.go:2054:16 unnecessary conversion
# internal/distributed/proxy/httpserver/handler_v2.go:2056:16 unnecessary conversion
# internal/distributed/proxy/httpserver/handler_v2_test.go:2185:28 unnecessary conversion
# internal/distributed/proxy/httpserver/handler_v2_test.go:2217:28 unnecessary conversion
# Fixed by removing string(...) around paramtable default metric constants.
# e2e-default failed in job 40810 on:
# testcases/test_geometry_operations.py::TestGeometryCollection::test_spatial_query_and_search[False-True-sealed-ST_CROSSES]
# error: Connection refused to /v2/vectordb/collections/flush on port 19530
# the same job showed querynode/streamingnode restarts and proxy ContainerCreating during cleanup.
# REST collection tests in that job had already passed, so this looked unrelated to this PR.
# e2e-default failed again in dispatcher 13914 / job-1 32439 on:
# testcases/test_collection_operations.py::TestCreateCollectionNegative::test_create_collection_quick_setup_with_invalid_consistency_level
# root cause: the test used collection_create(), whose wrapper auto-injects params.consistencyLevel=Strong.
# That made the request hit the new conflict branch instead of the intended top-level invalid consistencyLevel branch.
# Fixed in
|
||
|
Buqian ZhengandGitHub
|
933ce30344 |
fix: propagate unknown for missing nested values (#50979)
## Summary issue: #50976 This PR fixes NULL / UNKNOWN propagation gaps in filter expressions for missing nested values: - Treat missing JSON arithmetic operands and JSON `array_length` extraction failures as UNKNOWN instead of definite booleans. - Preserve UNKNOWN during parser/RBO contradiction folds for nested JSON paths, array subscripts, and struct-array subscripts. - Treat missing array / struct-array elements as UNKNOWN during execution. - Propagate JSON path/cast index typed-known masks so missing paths and cast failures are not matched by `!=`, `not in`, or outer `not(...)`. - Propagate JSON flat index comparable-value masks for comparison predicates. The commits are split by issue area: - `4f15d6fe79` `fix: treat missing json arithmetic operands as unknown` - `69d6e95917` `fix: preserve unknown for missing nested predicates in rbo` - `51b58edf05` `fix: treat missing array elements as unknown` - `d9dd4c9334` `fix: honor json path index unknown values` - `a9d813b322` `fix: honor json flat index unknown values` ## Verification - `LOCALSTORAGE_PATH=/tmp/milvus-test-localstorage go test -buildvcs=false -count=1 ./internal/parser/planparserv2/rewriter` - `ninja -C /tmp/milvus-50976-build all_tests -j32` - focused C++ regressions, including `JsonFlatIndexTest.*:JsonFlatIndexExprTest.*:JsonIndexTest.*:JsonPathIndexTest.*` (49 tests) - focused array regressions including `Expr.TestArraySubscriptMissingElementIsUnknown` and `Expr.TestArray*` - `git diff --check origin/master...HEAD` ## Note While adding extra JsonFlatIndex null-expression coverage, a separate pre-existing `PhyNullExpr::DetermineExecPath()` / `std::call_once` deadlock was found. That hanging test is not included here; this PR keeps JsonFlatIndex coverage scoped to comparison UNKNOWN behavior. --------- Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com> |
||
|
|
b1bbaafc88 |
fix: skip index raw data for array output (#51109)
issue: https://github.com/milvus-io/milvus/issues/51033 ref: https://github.com/milvus-io/milvus/issues/42148 --------- Signed-off-by: SpadeA <tangchenjie1210@gmail.com> Co-authored-by: zhuwenxing <wxzhuyeah@gmail.com> |
||
|
nicoandGitHub
|
c701920a32 |
test: fix struct array nullable field error expectation (#51073)
This PR updates the struct array invalid-case test expectation to match the current error message for nullable sub-fields inside a non-nullable struct. The affected parametrized cases are: - test_struct_array_with_nullable_field[clip_embedding1] - test_struct_array_with_nullable_field[scalar_field] This is a test-only change. --------- Signed-off-by: nico <cheng.yuan@zilliz.com> |
||
|
junjiejiangjjjandGitHub
|
116877a0aa |
enhance: Support L0 chain (#51012)
issue: https://github.com/milvus-io/milvus/issues/51011 Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com> |
||
|
|
4fad6e9dbb |
test: Add TEXT LOB Python client coverage (#50563)
## Summary - Add Python client TEXT LOB coverage for read/write, lifecycle, text_match, BM25, compaction, and storage layout checks - Add bulk import coverage for TEXT LOB payloads across JSON, CSV, and Parquet via RemoteBulkWriter - Include a small analyzer-token cache and reduced payload setup to keep the suite runtime lower issue: #50562 ## Test Plan - [x] python3 -m py_compile tests/python_client/milvus_client/test_milvus_client_text_lob.py tests/python_client/bulk_insert/test_bulk_insert_api.py - [x] python3 -m pytest -o addopts='' --collect-only -q tests/python_client/milvus_client/test_milvus_client_text_lob.py tests/python_client/bulk_insert/test_bulk_insert_api.py::TestBulkInsert::test_text_lob_bulk_import_json_csv_parquet - [x] Full TEXT LOB Python client file run reported at 95s with 6 workers --------- Signed-off-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Eric Hou <eric.hou@zilliz.com> |
||
|
|
590a14fdf7 |
fix: reject bare NULL literal in expressions instead of misparsing it as a field (#50889)
issue: #50882 ## Problem `NULL` is not a grammar token in the plan parser, so the lexer treats a bare `NULL`/`null` as an ordinary identifier. When it appears where a value is expected — e.g. inside an `in [...]` value list: ``` id in [6560, NULL, 6722, -7856, -6757] ``` `VisitIdentifier` runs a field lookup on `NULL`: - **without a dynamic field** it fails with the misleading `field NULL not exist`, which misleads users into thinking they must add a column named `NULL` to their schema; - **with a dynamic field** it is silently mistaken for a JSON key and accepted. This affects both `delete()` and `query()` (they share the same `ParseExpr` path). ## Fix Treat bare `null`/`NULL` (case-insensitive) as a reserved word in `VisitIdentifier` and reject it up-front with an actionable message, regardless of whether a dynamic field exists: ``` NULL literal is not supported in expressions; use '<field> is null' or '<field> is not null' instead ``` This implements option 1 (robust rejection) from the issue. `<field> is null` / `is not null` are unaffected (they parse via dedicated tokens, not `VisitIdentifier`). **The guard is schema-aware for backward compatibility** (review feedback): `null` only becomes a create-time keyword in this PR, so a legacy collection may own a field literally named `null`, and the bare identifier is the only syntax that can reference a top-level scalar field. The rejection is therefore gated on a strict `GetFieldFromName` lookup — a real declared field named `null` resolves exactly as before this PR (including `null is null`, which now parses as a valid predicate on that field), while the dynamic-field fallback (the source of the original misparse) still rejects. A JSON **sub-key** literally named `null` additionally remains reachable via quoting, e.g. `field["null"]` / `$meta["null"]`, whose base identifier is the field name, not `null`. ## Test Added `TestExpr_NullLiteral` covering rejection of `NULL` inside `in [...]`, as a comparison operand, and as a left operand, plus confirmation that `is null` / `is not null` still work. Added `TestExpr_NullLiteral_LegacyNullField` locking the legacy path: a scalar field named `null` stays queryable via the bare identifier, a JSON field named `null` stays reachable as `null["x"]`, and any casing that doesn't name a real field keeps the reserved-word rejection. Full `planparserv2` package tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Note: field-name validation tightened Alongside the expression-level rejection, `validateFieldName` (proxy) and `validateAddedStructFieldName` (rootcoord) now go through `common.IsFieldNameKeyword`, which rejects **any casing** of `null` (`null`/`NULL`/`nUlL`/…) as a field name — consistent with how a bare `NULL` literal is rejected in expressions. Previously `FieldNameKeywords` had no `null` entry, so a field literally named `null` could be created. This only affects **new** field creation (validation runs at create time); existing collections/fields are unaffected — and thanks to the schema-aware guard above, a legacy field literally named `null` also stays queryable. --------- Signed-off-by: xiaofanluan <xf@hjjaq.com> Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com> Co-authored-by: xiaofanluan <xf@hjjaq.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
秀吉andGitHub
|
3f2ce12895 |
enhance: support text_match_fuzzy (edit-distance) scalar filter (#50933)
issue: #50920 design doc: [docs/design-docs/design_docs/20260702-text_match_fuzzy.md](docs/design-docs/design_docs/20260702-text_match_fuzzy.md) ## What Adds a scalar filter operator `text_match_fuzzy(field, "query", max_edit_distance=K)` for typo-tolerant (Levenshtein edit-distance) matching on analyzed VARCHAR fields that have a text-match index. Filter-only: it returns a boolean bitset, no scoring (scoring is tracked separately in #50921). `K` is an integer in `[0, 2]` (tantivy's fuzzy automaton hard cap); out-of-range is rejected. ## How Mirrors the existing `text_match` / `phrase_match` path through every layer, reusing the term-dictionary FST and tantivy's `FuzzyTermQuery` (no new index, no new storage): - Tantivy binding: `fuzzy_match_query` tokenizes the query with the field analyzer and ORs one `FuzzyTermQuery` per token; exposed via the `tantivy_fuzzy_match_query` C entry (cbindgen header regenerated). - C++ index: `TextMatchIndex::FuzzyMatchQuery` plus the wrapper, mirroring `MatchQuery`. - proto: `OpType.TextMatchFuzzy = 17` (next free value; 15/16 are already `InnerMatch`/`RegexMatch`). The edit distance rides in the existing `UnaryRangeExpr.extra_values`, like slop / min_should_match, so no new field is added. - grammar + Go parser: a new `text_match_fuzzy(...)` rule and `VisitTextMatchFuzzy`, which validates `K` to `[0, 2]` and stores it in `extra_values`. - C++ executor: routes the op to the text-index path (dispatch, exec-path, and offset-input guards) and calls `FuzzyMatchQuery`, with a `[0, 2]` re-check on the C++ side (the same way `PhraseMatch` validates slop) for plans not built through the parser. ## Tests - Rust unit test: distance 0/1/2, fuzzy-vs-exact, multi-token OR, and the distance `K+1` exclusion boundary. - C++ gtest (`TextMatch.FuzzyIndex`, `GrowingNaive`, `SealedNaive`): index-level typo matching and the `K+1` boundary, plus the executor + growing-segment path — a typo matches the same rows as the exact term, `not text_match_fuzzy(...)`, and the executor guard rejects out-of-range / missing `max_edit_distance`. - Go parser tests: valid 0/1/2 produce the right op and extra value; a templated query preserves the distance; out-of-range / missing / non-integer / overflow / non-string field / match-not-enabled are all rejected. Existing `text_match` and `phrase_match` behavior is unchanged (full planparserv2 suite green). ## Notes - `max_edit_distance` becomes a reserved keyword in filter expressions, consistent with `minimum_should_match` and `threshold`. - Out of scope, left as follow-ups: scoring (#50921), prefix-fuzzy, ES-style `AUTO` fuzziness, and combining with `minimum_should_match`. --------- Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com> |
||
|
|
3353b53de7 |
test: add raw-string LIKE escape-model coverage (#50843)
## Summary This PR now contains **only supplementary test coverage** for the `LIKE` escape model — no production code. The production fix it originally carried (`scanLikePattern` in `internal/parser/planparserv2/pattern_match.go`, aligning the Go optimizer with the C++ canonical matcher in `RegexQuery.cpp`) **already landed in master via #50845**, which was stacked on this PR. After rebasing onto master that production commit is dropped as already-upstream, so the diff here reduces to tests only. `git diff master` touches exactly three files, all tests: | file | layer | |---|---| | `internal/parser/planparserv2/plan_parser_v2_test.go` | Go optimizer (parser-level) | | `internal/core/src/common/RegexQueryUtilTest.cpp` | C++ executor / canonical matcher | | `tests/python_client/.../test_milvus_client_scalar_filtering.py` | Python e2e | ## What the tests add Now that raw string literals `r"..."` exist (#50845), the escape tests read with a **single** backslash instead of the doubled/quadrupled backslashes the normal string-literal layer forced — exercising the raw-string path end to end. **Go — `TestExpr_RawString_LikeEscapeModel`** drives the optimizer end-to-end through `r"..."` LIKE patterns and asserts the lowered op + literal operand: - escaped wildcard → literal byte (`r"a\%bc"` → `Equal "a%bc"`); - a literal `%` coexisting with an **unescaped** `%` boundary (`r"abc\%def%"` → `PrefixMatch "abc%def"`, `r"%abc\%def%"` → `InnerMatch "abc%def"`); - `\\` collapsing to one literal `\`; - a dangling trailing backslash — un-expressible as a raw string (parse error), and `OpType_Match` fallback via a normal string. **C++ — `RegexQueryUtilTest.cpp`** keeps the canonical-matcher escape tests (literal `%`/`_` matched verbatim, multi-char decoys, long-span positives). Raw strings are a Go-parser-only feature, so the C++ executor tests use plan-level inputs. **Python e2e** — the escaped-wildcard `LIKE` expressions and `test_like_escaped_wildcard_is_literal` now use `r"..."` (one backslash); the eval oracle gains a raw-`LIKE` branch that re-quotes the verbatim raw content via `repr()` so it stays in lock-step with the server. A raw-string dangling-backslash case (rejected at lex time) is added; the matcher-level dangling case stays a normal string since a raw string cannot end in an odd number of backslashes. ## Verification ``` go test -tags dynamic,test -gcflags="all=-N -l" -count=1 ./internal/parser/planparserv2/ ``` planparserv2 package passes; `gofmt` clean; Python `ruff check` / `ruff format` clean; the oracle's raw-`LIKE` branch verified to reproduce the documented expected result sets. issue: #43864 --------- Signed-off-by: xiaofanluan <xf@hjjaq.com> Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com> Co-authored-by: xiaofanluan <xf@hjjaq.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
yanliang567andGitHub
|
182134e29e |
test: Update aggregation and groupby E2E coverage (#50963)
## Summary - strengthen search aggregation reject coverage for unsupported group-by, iterator, and dynamic-field combinations - add controlled search order_by + group_by_field + offset regression coverage - update query aggregation GROUP BY limit and output projection coverage to current semantics issue: #50960 ## Test Plan - [x] PYTHONPATH=. /Users/yanliang.qiao/Documents/Codex/2026-06-30/new-chat/work/milvus-e2e-venv/bin/python -m pytest -q --tb=short --disable-warnings --timeout=300 --host 127.0.0.1 --port 19530 --user root --password Milvus <targeted agg/groupby cases>: 11 passed, 1 xfailed in 51.81s - [x] git diff --check -- tests/python_client/milvus_client/test_milvus_client_search_aggregation.py tests/python_client/milvus_client/test_milvus_client_search_order.py tests/python_client/testcases/test_query_aggregation.py Milvus test target: qa-milvus/yanliang-mas2-milvus-standalone-6686d94498-lhp7k, Running 1/1, restart count 0 Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com> |
||
|
Bingyi SunandGitHub
|
70f144a91c |
enhance: add delete/shard by namespace (#50153)
Shard data by namespace. issue: #50154 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com> |
||
|
junjiejiangjjjandGitHub
|
c17ba38a3b |
enhance: Add Hugging Face inference provider support (#50818)
issue: https://github.com/milvus-io/milvus/issues/50816 Add Hugging Face Inference Providers client support for feature extraction and sentence similarity APIs, and wire it into text embedding and rerank model providers. The new provider supports: - text embedding via feature-extraction - rerank scoring via sentence-similarity - Hugging Face router provider selection with hf_provider - MILVUS_HUGGINGFACE_API_KEY credential fallback - provider config entries for text embedding and rerank Also add focused tests for the Hugging Face client, rerank provider, and paramtable provider docs. Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com> |
||
|
Li YiyangandGitHub
|
d8d823e782 |
test: add schema evolution e2e coverage (#50826)
## What - Add drop field e2e coverage for MilvusClient. - Add function field e2e coverage. - Refine schema evolution e2e coverage. - Bump pymilvus test dependency to 3.1.0rc49. ## Test - Not run in this turn; PR contains Python e2e test coverage changes. --------- Signed-off-by: lyyyuna <yiyang.li@zilliz.com> |
||
|
pymilvus-botandGitHub
|
e2304fedf9 |
test: Increase PyMilvus version to 3.1.0rc52 from master for master branch (#50875)
Automated bump from pymilvus master branch to milvus master branch. Updates tests/python_client/requirements.txt. Signed-off-by: pymilvus-bot <pymilvus@zilliz.com> |
||
|
Buqian ZhengandGitHub
|
e05e50e144 |
fix: preserve unknown semantics for json path predicates (#50702)
issue: #50699 Fixes #50699 ## What this PR does This PR makes JSON path predicate missing/null/type-mismatch behavior preserve UNKNOWN consistently: - Treats parent JSON NULL, missing nested paths, incompatible path types, and invalid array paths as UNKNOWN instead of operator-specific boolean constants. - Preserves JSON stats/index validity bitmaps when returning cached bitmap results. - Updates parser rewrite behavior and tests that previously assumed scalar JSON missing-path `!=` was a definite true result. - Aligns raw, stats/index, brute-force, offset/iterative, and `NOT` behavior around SQL-style three-valued logic. ## Verification - `cmake -S internal/core -B cmake_build` - `ninja -v unittest/all_tests` - `go test -buildvcs=false -count=1 ./internal/parser/planparserv2/rewriter` - Focused C++ JSON nullable/contains suite passed 705/705. - Additional binary-range/non-nullable JSON range/term suite passed 540/540. - `git diff --check` passed. - Staged diff check passed. --------- Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com> |
||
|
zhuwenxingandGitHub
|
98a8cffcf2 |
test: add struct array feature coverage (#50795)
## What type of PR is this? /test ## What this PR does / why we need it Adds Python client coverage for struct array feature scenarios on master: - Covers struct-array vector subfields across JSON, JSONL, CSV, and Parquet import files. - Covers LocalBulkWriter generated JSON, JSONL, CSV, and Parquet files for FLOAT_VECTOR, FLOAT16_VECTOR, BFLOAT16_VECTOR, INT8_VECTOR, and BINARY_VECTOR struct subfields. - Aligns BulkWriter test inputs with ordinary vector field input forms, including FP16/BF16 ndarray inputs and INT8 ndarray inputs. - Adds hybrid search, range, group-by, element_scope regression coverage, and bitmap index coverage for struct array element fields. - Updates Python client test dependency pin to pymilvus 3.1.0rc50 so the merged BulkWriter JSONL and struct vector fixes are exercised. ## Which issue(s) this PR fixes None. ## Special notes for your reviewer The BulkWriter coverage intentionally uses native insert-format inputs for non-float32 vectors to match ordinary vector field behavior. ## Tests ```bash python -W ignore -m pytest -v -n 6 --tb=short \ milvus_client/test_milvus_client_struct_array.py::TestMilvusClientStructArrayImport::test_import_struct_array_vector_subfield_with_local_bulk_writer \ --host 10.100.36.207 --port 19530 --token root:Milvus \ --minio_host 10.100.36.206 --minio_bucket struct-array-master-ff3dbeb ``` Result: `20 passed in 120.02s` Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
|
0bbe4d02a8 |
test: Add sparse inverted invalid parameter checks (#50775)
issue: #50108 This PR updates sparse inverted index negative coverage to verify the server rejects invalid build parameters during create_index. Changes: - Remove stale xfail markers for invalid sparse inverted index codec and non-positive block size cases. - Assert the concrete validation messages for unsupported inverted_index_codec and block_max_block_size values 0/-1. Verification: - python3 -m py_compile tests/python_client/milvus_client/test_milvus_client_sparse_inverted_index.py - python3 -m pytest -o "addopts=-p no:locust -v" tests/python_client/milvus_client/test_milvus_client_sparse_inverted_index.py::TestSparseInvertedIndexV3Negative -q --host 10.104.18.28 --port 19530 Signed-off-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Eric Hou <eric.hou@zilliz.com> |
||
|
|
e67f265a0c |
test: update python client test expectations (#50642)
Update Python client test expectations for current master behavior. - Update describe_collection expected schema metadata. - Update binary vector dimension mismatch error messages for insert/upsert. - Update drop_collection_function negative cases to assert expected errors. Tests: - Not run per request. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: nico <cheng.yuan@zilliz.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
yanliang567andGitHub
|
c61863f419 |
test: Add bulk insert UTF-8 coverage (#50746)
## Summary Add UTF-8 coverage to the existing all-field parquet bulk insert test. - Add opt-in UTF-8 string generation for parquet bulk insert data. - Enable UTF-8 generation in the existing all-field parquet case. - Validate imported UTF-8 values for VARCHAR, analyzer-enabled VARCHAR, ARRAY<VARCHAR>, and JSON fields. issue: #50745 ## Test Plan - [x] `python -m py_compile tests/python_client/common/bulk_insert_data.py tests/python_client/testcases/test_bulk_insert.py` - [x] `git diff --check` - [x] `python -W ignore -m pytest 'testcases/test_bulk_insert.py::TestBulkInsert::test_bulk_insert_all_field_with_parquet[False-False-False-False-False-2000-128-False]' --host 10.104.26.151 --port 19530 --minio_host 10.104.26.126 --minio_bucket yanliang-26x -q` Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com> |
||
|
|
6c84618986 |
test: remove stale xfail on TextEmbedding drop-function e2e tests (#50647)
## Summary Bug #50424 — *"dropping a TextEmbedding function incorrectly removes the output vector field"* — was fixed and closed by #50471 ("fix: Split function drop semantics by request flag"). The three `drop_collection_function` e2e cases in `test_text_embedding_function_e2e.py` were still marked `@pytest.mark.xfail(strict=True)` referencing #50424. Now that the behavior is correct, the tests **pass**, and with `strict=True` pytest reports them as `XPASS(strict)` → **FAILED**, breaking `ci-v2/e2e-default` on `master` and on every open PR. ``` [XPASS(strict)] issue: .../issues/50424, dropping TextEmbedding function incorrectly removes output vector field = 3 failed, 5857 passed, 339 skipped, 2 xfailed, 1 xpassed, 9 rerun = ``` This is deterministic (not flaky): the `DataNotMatchException: Insert missed an field 'dense'` log lines are the tests' own expected negative-path assertions (after detaching the function, inserting without the vector must fail). ## Changes - Remove the three `@pytest.mark.xfail(...)` decorators from: - `TestTextEmbeddingFunctionCURD::test_drop_collection_function_verify_crud` - `TestTextEmbeddingFunctionCURD::test_drop_collection_function_one_of_multiple` - `TestTextEmbeddingFunctionCURD::test_drop_collection_function_then_add_again` - Remove the now-unused `DROP_TEXT_EMBEDDING_FUNCTION_XFAIL_REASON` constant. The tests now run as normal positive cases validating the fixed drop-function semantics from #50471. issue: #50646 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: xiaofanluan <xf@hjjaq.com> Co-authored-by: xiaofanluan <xf@hjjaq.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
zhuwenxingandGitHub
|
a95a54e875 |
test: add REST import 2PC coverage (#50462)
## What type of PR is this? /kind test ## What this PR does / why we need it: Adds RESTful import 2PC coverage for Milvus import lifecycle and CDC-focused scenarios. This PR adds: - REST import job helpers for `/describe`, `/commit`, `/abort`, and state polling - pytest options for secondary CDC endpoint/object storage/topology parameters - a new REST test suite for import 2PC lifecycle, visibility, commit/abort, delete semantics, TTL, compaction, formats, data types, indexes, partitions, DML interleaving, CDC, and fault-tolerance gates - xfail regression coverage for known issues: - #50458 REST import commit/abort authorization gap - #50459 NaN/Inf FloatVector import/query behavior - #50460 REST `options.auto_commit=null` validation gap ## Which issue(s) this PR fixes: N/A ## Special notes for your reviewer: The CDC cases require passing secondary cluster REST/MinIO options and topology arguments. Without those options, CDC-specific cases skip with an explicit environment precondition message. The manual compaction-after-commit case is marked non-strict xfail because compaction may remain `Executing` on current import 2PC test builds. ## Test result: ```text python -m py_compile tests/restful_client_v2/conftest.py tests/restful_client_v2/api/milvus.py tests/restful_client_v2/testcases/test_import_2pc_operation.py ruff check tests/restful_client_v2/api/milvus.py tests/restful_client_v2/conftest.py tests/restful_client_v2/testcases/test_import_2pc_operation.py python -m pytest --collect-only -q testcases/test_import_2pc_operation.py 141 tests collected in 2.14s ``` --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
zhuwenxingandGitHub
|
b3fe69d844 |
test: add struct array nullable dynamic field coverage (#50109)
### What this PR does Adds Python client and REST coverage for struct array nullable fields, including dynamic field scenarios and nullable vector config variants. ### Tests Not run; PR creation only. --------- Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
yanliang567andGitHub
|
d2bf17e734 |
test: Add external table add field E2E tests (#50420)
Related to #50416 ### What changed - Add MilvusClient E2E coverage for external table `add_collection_field` on scalar and vector fields. - Cover refresh/load/search/query behavior after adding fields, including about 10% NULL values in newly added scalar/vector fields. - Add non-Parquet source coverage for add scalar field on Lance, Iceberg, and Vortex. - Add full supported DataType matrix coverage for external add field, including scalar, array, geometry, and vector fields. - Add negative coverage for missing/duplicate `external_field`, type/dim mismatch, unsupported public data types, add/drop function, and drop field APIs. - Update Python client test dependency pin to `pymilvus==3.1.0rc35`. ### Verification - `python -m py_compile tests/python_client/milvus_client/test_milvus_client_external_table.py tests/python_client/common/external_table_common.py tests/python_client/base/client_v2_base.py tests/python_client/check/func_check.py` - `python -m ruff check tests/python_client/milvus_client/test_milvus_client_external_table.py tests/python_client/common/external_table_common.py tests/python_client/base/client_v2_base.py tests/python_client/check/func_check.py` - `python -m pytest -n 4 milvus_client/test_milvus_client_external_table.py --host 10.104.18.101 --port 19530 --minio_host 10.104.18.27 --minio_bucket yanliang-mas2 --tb=short -q -s` Result: ```text 182 passed, 1 xfailed in 318.94s (0:05:18) ``` The xfailed case is linked to #50416 and validates that old-field search still works before the server returns the current internal QueryNode assert for newly added fields before refresh. --------- Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com> |
||
|
junjiejiangjjjandGitHub
|
d19de1110c |
enhance: add boost score support (#50372)
issue: https://github.com/milvus-io/milvus/issues/46565 Implement boost score evaluation for the Go search reduce pipeline, including QueryNode task integration, segcore C API bindings, and score expression combination support. Add boost score runner logic in core, expose segment-level boost scoring to Go, and cover the new behavior with C++, Go, and Python client tests. Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com> |
||
|
zhuwenxingandGitHub
|
3e34efd40c |
test: include chaos checker error samples in assertions (#50509)
## What this PR does When chaos checker assertions fail, the final pytest assertion message currently only includes normalized error text, which can drop useful values such as expected/actual row counts. This PR keeps a compact sample of the original error alongside the normalized error type so the useful failure detail appears at the end of the pytest output. ## Verification - python3 -m py_compile tests/python_client/chaos/checker.py Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com> |
||
|
|
e2787d3981 |
enhance: standardize error handling on merr + Sys/Input classification (#50221)
issue: #47420 ## What this PR does Project-wide migration of raw `fmt.Errorf` / `errors.New` in function bodies onto the `merr` framework, plus the Sys-vs-Input error classification and the machinery it drives (retriability, fine-grained metrics, segcore unification), plus the convention docs and a linter that keeps it from regressing. Scope: storage, proxy, coordinators (root/data/query), query node, data node, `pkg/util` & `internal/util`, expression parser, message queue, streaming, and misc packages. Bare raw-error usages went from ~3000 to a ~340 allowlist (package-level sentinels / build-tag / test sites). --- ## How to review this PR It is large but the vast majority is mechanical. Changes fall into three tiers; spend review budget on Part 2 and Part 3. ### Part 1 — Mechanical standardization (low risk, verify by rule) Each converted call follows one of a small fixed set of rules. To review, check that each site obeys the matching rule rather than reading every line: | Pattern | Rule | |---|---| | `fmt.Errorf("...")` originating a new error | → `merr.WrapErrXxxMsg("...")` with a code matching the failure's meaning | | Adding context to an existing typed error | → `merr.Wrap(err, "...")` / `merr.Wrapf(...)` — **preserves** the inner code (never `WrapErr*Err`, which overwrites it) | | Errors inside the streaming subsystem | → `status.New*` factories (StreamingError), **not** merr — this is the component-internal dialect (see `docs/dev/error_handling_guide.md`) | | Low-level / control-flow signal caught by `errors.Is` | → kept as a package-level `errors.New` sentinel (lowercase, same-package) | Conventions are documented in `docs/dev/error_handling_guide.md` (how-to) and `docs/dev/error_sentinel_convention.md` (rules + audit). A `gocritic`/`ruleguard` rule (`rawmerrerror`, in `rules.go`) enforces "no raw `return errors.New/fmt.Errorf`" under `make verifiers`. ### Part 2 — Behavior changes (review these closely) These are the sites where the wire contract or runtime behavior changes, not just the source text. Listed by category; representative locations given, full set in the diff. **A. gRPC wire-code shifts: `UnexpectedError(1)/Code 65535` → typed code.** Where a handler previously returned a raw error (collapsed to `Code=65535` on the wire), it now returns a typed merr, so the client sees a real code. The most common shift is to `IllegalArgument(5)/Code 1100` (ParameterInvalid). Touch points include datanode task handlers (CreateTask/Query/Drop), proxy Upsert, querynode GetMetrics, datacoord CreateIndex, httpserver query-response builder, and typeutil schema validation. One code refinement: an index-param validation moved `1100` → `1101` (ParameterMissing). **Client/SDK assertions and any code that switched on `Code=65535` for these paths must be re-checked** (the go_client e2e assertions were already aligned in this PR). **B. Prometheus `status` label contract change (externally visible).** The proxy metric's coarse `fail` / `rejected` values are split into `fail_input` / `fail_system` and `rejected_user` / `rejected_system` (in `requestutil.ParseMetricLabel`; auth/privilege rejections count as `rejected_user`), so dashboards can attribute a failure to caller vs operator. **Dashboards/alerts querying `status="fail"` must migrate to `status=~"fail_.*"`, and `status="rejected"` to `status=~"rejected_.*"`.** The in-repo Grafana dashboard is already migrated; external dashboards built on the old values silently go empty after upgrade. This is the one change that requires an ops-side migration. **C. Retriability semantics.** - C1: `merr.Status(err)` now forces `Retriable=false` when the error is an `InputError` — a malformed request can never succeed on blind retry, so clients never get the self-contradictory "your input is wrong but you may retry". - C2: `retry.Do` short-circuits an `InputError` (non-retriable) — **but only when the caller did not pass a `RetryErr` predicate**. The check is an `if c.isRetryErr != nil { ... } else if InputError { ... }` *mutually exclusive* branch (`pkg/util/retry/retry.go`): an explicit `RetryErr` takes precedence and bypasses the InputError abort. `retry.Handle` deliberately does **not** apply the InputError abort (its callers signal abort via `shouldRetry=false`). Four flusher startup callsites that must retry through transient "not ready" errors were given explicit `RetryErr` escape hatches. **D. segcore (C++→Go) error classification.** A single shared Go-side table (`pkg/util/merr/segcore.go`) maps each segcore code to a merr sentinel + InputError/signal category, replacing scattered hand-written `if errorCode == ...` switches in the cgo wrappers. **Wire `Code` values change for every segcore pass-through error, not just the remapped ones.** Named sentinels remap (C++ `2003` → merr `2001`, `2033` → `2002`, Folly/Knowhere codes likewise); **all remaining pass-through codes (`2004`–`2043`, previously surfaced to clients as raw C++ enum values) now serialize as `2000`** (`ErrSegcore`), with the original C++ code preserved in the `Reason` text (`segcoreCode=...`); unknown/future codes collapse to `2000` as well (pinned by the `wire_code_projection` test). Transient segcore classes (object storage / file IO / OOM / mmap / FieldNotLoaded — 11 codes) now report `Retriable=true`. **Any client switching on raw segcore codes in the `2004`–`2043` range must be re-checked**; the in-Reason code remains available for diagnostics. Signal codes (PretendFinished / FollyCancel) are recognized centrally. `errors.Is`-based control flow on these (e.g. scheduler skip/retry) is preserved. **E. InputError classification (25 sentinels + dynamic marks).** 25 sentinels in `errors.go` carry `WithErrorType(InputError)` (the Collection / ResourceGroup / Database families, `ErrIndexDuplicate`, `ErrParameterInvalid`, `ErrPrivilegeNotAuthenticated`, `ErrImportFailed`, `ErrQueryPlan`, ...), plus dynamic marks for the 8 segcore input codes (ExprInvalid, DimNotMatch, MetricTypeInvalid, FieldIDInvalid, ...) and `WrapErrAsInputError`. The widest blast radius is `ErrParameterInvalid` (1100): ~2335 `WrapErrParameterInvalid*` callsites now classify as input / non-retriable. Because of C1/C2 this changes retriability for any path that returns these. **The audit to confirm no transient path was mis-marked is the single most important review item** (see Part 3). One reverse correction: storage field-stats parsing moved from `ErrParameterInvalid` (input) to `ErrDataIntegrity` — a corrupted stored stat is data corruption, not user input. ### Part 3 — Known risks & traps (called out proactively) 1. **`merr.Wrap` vs `WrapErr*Err` (code-masking).** `WrapErr*Err` builds a `wrappedMilvusError{sentinel: ErrServiceInternal}` whose `code()` returns the *outer* sentinel — it overwrites the inner typed code and hides the `errors.Is` chain. This is intentional (use it to *deliberately* downgrade), but it was a recurring conversion defect; the rule "add context with `merr.Wrap`, downgrade with `WrapErr*Err`" is enforced by convention and reviewed across the diff. 2. **InputError × `retry.Do` blast radius.** Marking a sentinel `InputError` makes any `retry.Do(...)` without a `RetryErr` predicate stop retrying it. Reviewers should sanity-check that no transient use of the 19 newly-marked sentinels (especially `ErrParameterInvalid`) sits inside a retry loop that needed to keep spinning. The known flusher cases were handled (see C2). 3. **The ~340 raw-error allowlist.** What remains as bare `errors.New` is, by design: package-level sentinels (caught by `errors.Is`), `//go:build test` sites, and out-of-band trees (`cmd/`, `tests/`, codegen, walimpls). The linter only bans the *direct-return* form; assignment-then-return escapes and the full no-exceptions ban are deferred to an AST-based linter (Tier 2, documented). 4. **segcore C++ second step deferred.** This PR unifies classification on the Go side; splitting the dual-semantic C++ codes at the source is a follow-up. --- ## Validation - `make verifiers`: Go side clean (gofmt + static-check across modules, including the new `rawmerrerror` rule with a 0-hit baseline repo-wide). - `make test-go`: passing; the one real regression introduced (a datanode `invalid_task_type` assertion shifting `1` → `5` from a ParameterInvalid conversion) was fixed in-tree. - go_client e2e CreateIndex assertions aligned to the new merr messages. --------- Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
yanliang567andGitHub
|
4f2489fb3b |
test: Restore CSV nullable vector bulk writer coverage (#50482)
## Summary Restore the CSV branch in the nullable vector RemoteBulkWriter regression test. This removes the previous skip and the explicit `nullkey="null"` workaround so the test now covers the default CSV import path for nullable `FLOAT_VECTOR` values. Fixes #49678 ## Test Plan - [x] `/Users/yanliang.qiao/fork/.venv-py312/bin/python -m pytest testcases/test_bulk_insert.py::TestBulkInsertNullableVector::test_bulk_writer_nullable_float_vector --collect-only -q` - [x] `/Users/yanliang.qiao/fork/.venv-py312/bin/python -m pytest -q -s 'testcases/test_bulk_insert.py::TestBulkInsertNullableVector::test_bulk_writer_nullable_float_vector[60-32-4]' --host 10.104.30.186 --port 19530 --minio_host 10.104.18.198 --minio_bucket yanliang-mas2` Note: I also ran the full JSON/CSV/Parquet parametrized test against the same temporary cluster. JSON and CSV passed; the Parquet parameter later hit a `get_bulk_insert_state` RPC `DEADLINE_EXCEEDED` while polling import state, unrelated to this CSV regression change. Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com> |
||
|
yihao.daiandGitHub
|
a69e9368b6 |
test: Fix CDC force-promote cleanup before final diff (#50326)
## Summary issue: #50320 Clean downstream-only collections created during CDC force-promote verification before restoring the topology back to `upstream -> downstream`. The affected tests create `*_after` collections while downstream is force-promoted primary. If teardown restores `A -> B` first, downstream becomes a replica again and `drop_collection` is rejected, leaving `test_fp_*_after` collections that fail the final upstream/downstream diff. ## Changes - Add targeted cleanup for downstream-only force-promote verification collections. - Remove those collections from the generic cleanup list after the targeted cleanup attempt. - Add a focused unit test for the cleanup helper behavior. ## Test Plan - [x] `PYTHONPATH=tests/python_client conda run -n milvus2 python -m pytest -c /tmp/empty-pytest.ini --confcutdir=tests/python_client/cdc/testcases tests/python_client/cdc/testcases/test_force_promote_cleanup.py -q` - [x] `conda run -n milvus2 python -m py_compile tests/python_client/cdc/testcases/test_force_promote.py tests/python_client/cdc/testcases/test_force_promote_cleanup.py` - [x] `git diff --check` - [ ] `make static-check` could not complete locally because the CGo core artifacts in this workspace do not match the latest master headers. It first failed without `milvus_core.pc`; with `PKG_CONFIG_PATH=/Users/yihao.dai/workspace/milvus/internal/core/output/lib/pkgconfig`, it failed at `internal/util/initcore` with missing `C.CArrowReaderConfig` / `C.InitArrowReaderConfig`. Signed-off-by: Yihao Dai <yihao.dai@zilliz.com> |
||
|
yanliang567andGitHub
|
a87bc5844c |
test: Add hybrid search partition isolation coverage (#50401)
## What changed - Add Python client, RESTful v2, and Go SDK regression coverage for hybrid search on collections with `partitionkey.isolation=true`. - Verify normal search still rejects unsupported partition-key isolation filters: - `tenant in ["tenant_a", "tenant_b"]` - missing tenant filter - Keep the current hybrid search server bug as conditional xfail/skip, so baseline search regressions are not hidden. - Add RESTful v2 coverage for MinHash + dense hybrid search with explicit `partitionNames`. ## Issues Related to #50398 Related to #50396 ## Verification - `ruff check ../tests/python_client/milvus_client/test_milvus_client_partition_key_isolation.py ../tests/restful_client_v2/testcases/test_vector_operations.py` - `python -m py_compile tests/python_client/milvus_client/test_milvus_client_partition_key_isolation.py tests/restful_client_v2/testcases/test_vector_operations.py` - `git diff --check` - Python client 50398 case against `10.104.18.101`: `1 xfailed` - RESTful v2 50398 case against `10.104.18.101`: `1 xfailed` - Go SDK 50398 case against `10.104.18.101`: reproduces current bug and conditionally skips Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com> |
||
|
|
fe827c9dc4 |
test: align python client expectations with current API responses (#50414)
## Summary - update `test_milvus_client_collection_describe` to expect `properties.max_field_id` in `describe_collection` - align `search_aggregation` JSON/dynamic field rejection cases with the current server error messages - keep the affected Python client regressions covered without asserting outdated responses ## Test plan - [x] `python3 -m py_compile tests/python_client/milvus_client/test_milvus_client_collection.py tests/python_client/milvus_client/test_milvus_client_search_aggregation.py` - [ ] full pytest execution was not run locally in this submission 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: nico <cheng.yuan@zilliz.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wei liuandGitHub
|
1853a14897 |
enhance: make external vectors nullable by default (#50362)
issue: https://github.com/milvus-io/milvus/issues/45881 ## Summary - Make external user vector fields nullable by default during schema normalization. - Keep external system/virtual fields and generated function output fields excluded from forced nullable normalization. - Update schema, REST, and MilvusClient external table coverage so omitted vector nullable settings are expected to describe as nullable. ## Why External collection schemas already make user scalar fields nullable by default because external files can contain null values that Milvus cannot reject at collection creation time. Vector fields were temporarily excluded while the nullable vector offset mapping path still depended on eager load-time mapping. After lazy offset mapping for nullable vectors, that special-case exclusion is no longer needed. External vector fields can now follow the same default nullable normalization rule as scalar fields. Related lazy offset mapping PR: https://github.com/milvus-io/milvus/pull/49168 ## Validation - `git diff --check` - `python3 -m py_compile tests/restful_client_v2/testcases/test_external_collection_operations.py tests/python_client/milvus_client/test_milvus_client_external_table.py` - `GOTOOLCHAIN=auto go test -tags dynamic,test github.com/milvus-io/milvus/pkg/v3/util/typeutil -run TestNormalizeAndValidateExternalCollectionSchema -count=1 -v` - `GOTOOLCHAIN=auto go test -tags dynamic,test github.com/milvus-io/milvus/pkg/v3/util/typeutil -count=1` ## Local environment note - `/reset-milvus` reset etcd/minio before Go test runs, but this worktree does not have `bin/milvus`, so standalone Milvus could not be restarted locally. The validated tests above are pure schema/typeutil tests. Signed-off-by: Wei Liu <wei.liu@zilliz.com> |
||
|
|
fb652661b4 |
test: add woodpecker chaos test yamls for all chaos categories (#50391)
Fixes #50365 ## What changed - Add Woodpecker Chaos Mesh YAMLs for pod_failure, pod_kill, network_partition, network_latency, mem_stress, and io_latency. - Add Woodpecker entries to the corresponding chaos testcases.yaml files. ## Verification - git diff --check upstream/master...HEAD - Woodpecker chaos flow verified working before PR submission. Signed-off-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
yanliang567andGitHub
|
8a9710a6fd |
test: Add search order pagination regression coverage (#50397)
issue: #49879 ## What changed - Add an independent Python client regression test for search with `order_by_fields` plus `offset`, using controlled ANN candidates from #49879. - Mark the regression as strict xfail because current behavior still applies offset before scalar ordering. - Mark `search_aggregation` top_hits sorting by dynamic fields as strict xfail because dynamic fields are explicitly rejected for search aggregation today. ## Verification ```bash /Users/yanliang.qiao/fork/.venv-py312/bin/python -m pytest -s --tb=short --host 10.104.18.101 --port 19530 \ milvus_client/test_milvus_client_search_order.py::TestMilvusClientSearchOrderIndependent::test_milvus_client_search_order_by_with_offset \ milvus_client/test_milvus_client_search_aggregation.py::TestSearchAggregationIndependent::test_search_aggregation_top_hits_sort_by_dynamic_field ``` Result: `2 xfailed in 13.46s`. Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com> |
||
|
yanliang567andGitHub
|
2f37c0b481 |
fix: Restrict Kafka chaos selectors to broker pods (#50380)
issue: #50379 Fixes #50379 This PR narrows Kafka chaos selectors to broker pods by adding `app.kubernetes.io/component: kafka`. The previous selector matched both broker pods and the Kafka exporter pod because both use `app.kubernetes.io/name: kafka`. Changes: - Add `app.kubernetes.io/component: kafka` to Kafka container-kill chaos. - Add the same broker selector to Kafka pod-kill chaos. - Update Kafka pod-failure chaos from the legacy `component: kafka` selector to `app.kubernetes.io/component: kafka`. Verification: - `git diff --check` - Parsed the three Kafka chaos YAML files and verified each selector contains `app.kubernetes.io/component=kafka` with no stale `component` selector. Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com> |
||
|
junjiejiangjjjandGitHub
|
43010a4f18 |
enhance: implement Go-based search reduce pipeline with Late Materialization (#48984)
#48986 Replace C++ ReduceSearchResultsAndFillData with a Go reduce pipeline: - Export per-segment search results as Arrow RecordBatch via C Data Interface - HeapMergeReduce: k-way heap merge with PK dedup and GroupBy support - Late Materialization: single CGO call (FillOutputFieldsOrdered) for output fields - ExportSearchResultAsArrow supports extra field IDs for future L0 rerank Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com> |