## What
Enforce a function–field binding principle for function DDL (add / drop
/ alter function on an existing collection), so a function and its
output field stay coupled and already-stored vectors can never be
silently invalidated by DDL.
BM25 and MinHash follow the strict binding: add/drop only via the
unified `add_function_field` / `drop_function_field`, output field
always coupled. **TextEmbedding is a temporary carve-out** — its legacy
`AddCollectionFunction` / `DropCollectionFunction` paths are retained,
so attach-over-existing-field and detach are still possible for it,
because `add_function_field` embedding backfill is not yet implemented.
It will be folded into the unified path in a follow-up. So the "always
coupled" invariant currently holds for BM25/MinHash, not TextEmbedding.
## Changes
- **AlterCollectionSchema invariant**: reject standalone add-function (a
function must be added together with its new output field), reject
detaching a function without dropping its output field, and reject
function cascade (a new function's input being another function's
output).
- **Legacy RPCs**: `AddCollectionFunction` / `DropCollectionFunction`
are rejected for BM25/MinHash (must use `add_function_field` /
`drop_function_field`) and retained only for TextEmbedding (see
carve-out). `AlterCollectionFunction` is kept and gains the whitelist
below. In the alter and legacy-add paths, function field IDs are always
re-derived from field names (never trusted from the request), so a
request cannot inject an unrelated field ID that a later
`drop_function_field` would delete.
- **alter_function whitelist**: only connection/runtime params may
change (TextEmbedding: `url`, `credential`, `timeout_ms`,
`max_client_batch_size`, `region`, `location`, `projectid`, `user`);
BM25/MinHash have no alterable params. Function identity (type, name,
input/output fields) and output-shaping params (`dim`, `model_name`,
`endpoint` — the TEI model identity, `normalize`, `truncate*`, prompts,
...) are immutable. Params are normalized (keys lowercased, duplicate
keys rejected) so a crafted duplicate key cannot bypass the diff.
- **drop_function_field**: allow any output-producing function (dropping
needs no backfill), removing the previous BM25/MinHash-only restriction.
## Not in scope / follow-up
- Extending `add_function_field` to TextEmbedding (post-creation add
would require backfilling every existing row through the external
embedding model) — deferred; folding the TextEmbedding legacy paths into
the unified binding follows this.
- MinHash input-field analyzer immutability lives on a different DDL
path (`AlterCollectionField`); tracked as a separate follow-up.
## Breaking changes
- `DropCollectionFunction` on a **MinHash** function (detach — remove
the function but keep its output field) is no longer supported; use
`drop_function_field` instead (drops the function together with its
output field). If a released version supported MinHash detach, migrate
any such call to `drop_function_field`.
issue: #51348🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: #51192
design doc:
docs/design-docs/design_docs/20260708-xgboost-function-chain.md
Add native xgboost FunctionChain expression support for L0 rerank with
FileResource-backed UBJ models.
This change includes:
- xgboost FunctionChain expression registration, parameter validation,
and execution
- FileResource-based UBJ model discovery and local path resolution
- lazy model loading with singleflight, lease/refcount lifecycle
protection, and stale eviction on FileResource sync
- cgo bridge for Arrow C Data based batch prediction
- native C++ UBJ model parser and predictor for supported tree models
- runtime-disabled stub for builds without cgo and with_xgboost
- validation for unsupported params, output modes, feature count
mismatch, invalid models, unsupported objectives, unsupported boosters,
multiclass models, multi-target leaf vectors, and unsupported input
column types
- C++ unit tests, Go tests, native parity tests, and Python client L0
E2E tests
- xgboost FunctionChain design document
L2 rerank support is intentionally deferred because Proxy does not yet
support FileResource sync and local resolution.
Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
## Summary
Bug #50424 — *"dropping a TextEmbedding function incorrectly removes the
output vector field"* — was fixed and closed by #50471 ("fix: Split
function drop semantics by request flag").
The three `drop_collection_function` e2e cases in
`test_text_embedding_function_e2e.py` were still marked
`@pytest.mark.xfail(strict=True)` referencing #50424. Now that the
behavior is correct, the tests **pass**, and with `strict=True` pytest
reports them as `XPASS(strict)` → **FAILED**, breaking
`ci-v2/e2e-default` on `master` and on every open PR.
```
[XPASS(strict)] issue: .../issues/50424, dropping TextEmbedding function incorrectly removes output vector field
= 3 failed, 5857 passed, 339 skipped, 2 xfailed, 1 xpassed, 9 rerun =
```
This is deterministic (not flaky): the `DataNotMatchException: Insert
missed an field 'dense'` log lines are the tests' own expected
negative-path assertions (after detaching the function, inserting
without the vector must fail).
## Changes
- Remove the three `@pytest.mark.xfail(...)` decorators from:
-
`TestTextEmbeddingFunctionCURD::test_drop_collection_function_verify_crud`
-
`TestTextEmbeddingFunctionCURD::test_drop_collection_function_one_of_multiple`
-
`TestTextEmbeddingFunctionCURD::test_drop_collection_function_then_add_again`
- Remove the now-unused `DROP_TEXT_EMBEDDING_FUNCTION_XFAIL_REASON`
constant.
The tests now run as normal positive cases validating the fixed
drop-function semantics from #50471.
issue: #50646🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
### What this PR does
Adds Python client and REST coverage for struct array nullable fields,
including dynamic field scenarios and nullable vector config variants.
### Tests
Not run; PR creation only.
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Related to #50416
### What changed
- Add MilvusClient E2E coverage for external table
`add_collection_field` on scalar and vector fields.
- Cover refresh/load/search/query behavior after adding fields,
including about 10% NULL values in newly added scalar/vector fields.
- Add non-Parquet source coverage for add scalar field on Lance,
Iceberg, and Vortex.
- Add full supported DataType matrix coverage for external add field,
including scalar, array, geometry, and vector fields.
- Add negative coverage for missing/duplicate `external_field`, type/dim
mismatch, unsupported public data types, add/drop function, and drop
field APIs.
- Update Python client test dependency pin to `pymilvus==3.1.0rc35`.
### Verification
- `python -m py_compile
tests/python_client/milvus_client/test_milvus_client_external_table.py
tests/python_client/common/external_table_common.py
tests/python_client/base/client_v2_base.py
tests/python_client/check/func_check.py`
- `python -m ruff check
tests/python_client/milvus_client/test_milvus_client_external_table.py
tests/python_client/common/external_table_common.py
tests/python_client/base/client_v2_base.py
tests/python_client/check/func_check.py`
- `python -m pytest -n 4
milvus_client/test_milvus_client_external_table.py --host 10.104.18.101
--port 19530 --minio_host 10.104.18.27 --minio_bucket yanliang-mas2
--tb=short -q -s`
Result:
```text
182 passed, 1 xfailed in 318.94s (0:05:18)
```
The xfailed case is linked to #50416 and validates that old-field search
still works before the server returns the current internal QueryNode
assert for newly added fields before refresh.
---------
Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com>
issue: #47420
## What this PR does
Project-wide migration of raw `fmt.Errorf` / `errors.New` in function
bodies onto
the `merr` framework, plus the Sys-vs-Input error classification and the
machinery it drives (retriability, fine-grained metrics, segcore
unification),
plus the convention docs and a linter that keeps it from regressing.
Scope: storage, proxy, coordinators (root/data/query), query node, data
node,
`pkg/util` & `internal/util`, expression parser, message queue,
streaming, and
misc packages. Bare raw-error usages went from ~3000 to a ~340 allowlist
(package-level sentinels / build-tag / test sites).
---
## How to review this PR
It is large but the vast majority is mechanical. Changes fall into three
tiers;
spend review budget on Part 2 and Part 3.
### Part 1 — Mechanical standardization (low risk, verify by rule)
Each converted call follows one of a small fixed set of rules. To
review, check
that each site obeys the matching rule rather than reading every line:
| Pattern | Rule |
|---|---|
| `fmt.Errorf("...")` originating a new error | →
`merr.WrapErrXxxMsg("...")` with a code matching the failure's meaning |
| Adding context to an existing typed error | → `merr.Wrap(err, "...")`
/ `merr.Wrapf(...)` — **preserves** the inner code (never `WrapErr*Err`,
which overwrites it) |
| Errors inside the streaming subsystem | → `status.New*` factories
(StreamingError), **not** merr — this is the component-internal dialect
(see `docs/dev/error_handling_guide.md`) |
| Low-level / control-flow signal caught by `errors.Is` | → kept as a
package-level `errors.New` sentinel (lowercase, same-package) |
Conventions are documented in `docs/dev/error_handling_guide.md`
(how-to) and
`docs/dev/error_sentinel_convention.md` (rules + audit). A
`gocritic`/`ruleguard`
rule (`rawmerrerror`, in `rules.go`) enforces "no raw `return
errors.New/fmt.Errorf`"
under `make verifiers`.
### Part 2 — Behavior changes (review these closely)
These are the sites where the wire contract or runtime behavior changes,
not just
the source text. Listed by category; representative locations given,
full set in
the diff.
**A. gRPC wire-code shifts: `UnexpectedError(1)/Code 65535` → typed
code.**
Where a handler previously returned a raw error (collapsed to
`Code=65535` on the
wire), it now returns a typed merr, so the client sees a real code. The
most
common shift is to `IllegalArgument(5)/Code 1100` (ParameterInvalid).
Touch
points include datanode task handlers (CreateTask/Query/Drop), proxy
Upsert,
querynode GetMetrics, datacoord CreateIndex, httpserver query-response
builder,
and typeutil schema validation. One code refinement: an index-param
validation
moved `1100` → `1101` (ParameterMissing). **Client/SDK assertions and
any code
that switched on `Code=65535` for these paths must be re-checked** (the
go_client
e2e assertions were already aligned in this PR).
**B. Prometheus `status` label contract change (externally visible).**
The proxy metric's coarse `fail` / `rejected` values are split into
`fail_input` / `fail_system` and `rejected_user` / `rejected_system` (in
`requestutil.ParseMetricLabel`; auth/privilege rejections count as
`rejected_user`), so dashboards can attribute a failure to caller vs
operator.
**Dashboards/alerts querying `status="fail"` must migrate to
`status=~"fail_.*"`, and `status="rejected"` to
`status=~"rejected_.*"`.** The
in-repo Grafana dashboard is already migrated; external dashboards built
on the
old values silently go empty after upgrade. This is the one change that
requires an ops-side migration.
**C. Retriability semantics.**
- C1: `merr.Status(err)` now forces `Retriable=false` when the error is
an
`InputError` — a malformed request can never succeed on blind retry, so
clients
never get the self-contradictory "your input is wrong but you may
retry".
- C2: `retry.Do` short-circuits an `InputError` (non-retriable) — **but
only when
the caller did not pass a `RetryErr` predicate**. The check is an
`if c.isRetryErr != nil { ... } else if InputError { ... }` *mutually
exclusive*
branch (`pkg/util/retry/retry.go`): an explicit `RetryErr` takes
precedence and
bypasses the InputError abort. `retry.Handle` deliberately does **not**
apply
the InputError abort (its callers signal abort via `shouldRetry=false`).
Four
flusher startup callsites that must retry through transient "not ready"
errors
were given explicit `RetryErr` escape hatches.
**D. segcore (C++→Go) error classification.**
A single shared Go-side table (`pkg/util/merr/segcore.go`) maps each
segcore code
to a merr sentinel + InputError/signal category, replacing scattered
hand-written
`if errorCode == ...` switches in the cgo wrappers. **Wire `Code` values
change
for every segcore pass-through error, not just the remapped ones.**
Named
sentinels remap (C++ `2003` → merr `2001`, `2033` → `2002`,
Folly/Knowhere codes
likewise); **all remaining pass-through codes (`2004`–`2043`, previously
surfaced to clients as raw C++ enum values) now serialize as `2000`**
(`ErrSegcore`), with the original C++ code preserved in the `Reason`
text
(`segcoreCode=...`); unknown/future codes collapse to `2000` as well
(pinned by
the `wire_code_projection` test). Transient segcore classes (object
storage /
file IO / OOM / mmap / FieldNotLoaded — 11 codes) now report
`Retriable=true`.
**Any client switching on raw segcore codes in the `2004`–`2043` range
must be
re-checked**; the in-Reason code remains available for diagnostics.
Signal
codes (PretendFinished / FollyCancel) are recognized centrally.
`errors.Is`-based
control flow on these (e.g. scheduler skip/retry) is preserved.
**E. InputError classification (25 sentinels + dynamic marks).**
25 sentinels in `errors.go` carry `WithErrorType(InputError)` (the
Collection /
ResourceGroup / Database families, `ErrIndexDuplicate`,
`ErrParameterInvalid`,
`ErrPrivilegeNotAuthenticated`, `ErrImportFailed`, `ErrQueryPlan`, ...),
plus dynamic
marks for the 8 segcore input codes (ExprInvalid, DimNotMatch,
MetricTypeInvalid, FieldIDInvalid, ...) and
`WrapErrAsInputError`. The widest blast radius is `ErrParameterInvalid`
(1100):
~2335 `WrapErrParameterInvalid*` callsites now classify as input /
non-retriable. Because of C1/C2 this changes retriability for
any path that returns these. **The audit to confirm no transient path
was
mis-marked is the single most important review item** (see Part 3). One
reverse
correction: storage field-stats parsing moved from `ErrParameterInvalid`
(input)
to `ErrDataIntegrity` — a corrupted stored stat is data corruption, not
user
input.
### Part 3 — Known risks & traps (called out proactively)
1. **`merr.Wrap` vs `WrapErr*Err` (code-masking).** `WrapErr*Err` builds
a
`wrappedMilvusError{sentinel: ErrServiceInternal}` whose `code()`
returns the
*outer* sentinel — it overwrites the inner typed code and hides the
`errors.Is`
chain. This is intentional (use it to *deliberately* downgrade), but it
was a
recurring conversion defect; the rule "add context with `merr.Wrap`,
downgrade
with `WrapErr*Err`" is enforced by convention and reviewed across the
diff.
2. **InputError × `retry.Do` blast radius.** Marking a sentinel
`InputError` makes
any `retry.Do(...)` without a `RetryErr` predicate stop retrying it.
Reviewers
should sanity-check that no transient use of the 19 newly-marked
sentinels
(especially `ErrParameterInvalid`) sits inside a retry loop that needed
to keep
spinning. The known flusher cases were handled (see C2).
3. **The ~340 raw-error allowlist.** What remains as bare `errors.New`
is, by
design: package-level sentinels (caught by `errors.Is`), `//go:build
test`
sites, and out-of-band trees (`cmd/`, `tests/`, codegen, walimpls). The
linter
only bans the *direct-return* form; assignment-then-return escapes and
the full
no-exceptions ban are deferred to an AST-based linter (Tier 2,
documented).
4. **segcore C++ second step deferred.** This PR unifies classification
on the Go
side; splitting the dual-semantic C++ codes at the source is a
follow-up.
---
## Validation
- `make verifiers`: Go side clean (gofmt + static-check across modules,
including
the new `rawmerrerror` rule with a 0-hit baseline repo-wide).
- `make test-go`: passing; the one real regression introduced (a
datanode
`invalid_task_type` assertion shifting `1` → `5` from a ParameterInvalid
conversion) was fixed in-tree.
- go_client e2e CreateIndex assertions aligned to the new merr messages.
---------
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
Restore the CSV branch in the nullable vector RemoteBulkWriter
regression test.
This removes the previous skip and the explicit `nullkey="null"`
workaround so the test now covers the default CSV import path for
nullable `FLOAT_VECTOR` values.
Fixes#49678
## Test Plan
- [x] `/Users/yanliang.qiao/fork/.venv-py312/bin/python -m pytest
testcases/test_bulk_insert.py::TestBulkInsertNullableVector::test_bulk_writer_nullable_float_vector
--collect-only -q`
- [x] `/Users/yanliang.qiao/fork/.venv-py312/bin/python -m pytest -q -s
'testcases/test_bulk_insert.py::TestBulkInsertNullableVector::test_bulk_writer_nullable_float_vector[60-32-4]'
--host 10.104.30.186 --port 19530 --minio_host 10.104.18.198
--minio_bucket yanliang-mas2`
Note: I also ran the full JSON/CSV/Parquet parametrized test against the
same temporary cluster. JSON and CSV passed; the Parquet parameter later
hit a `get_bulk_insert_state` RPC `DEADLINE_EXCEEDED` while polling
import state, unrelated to this CSV regression change.
Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com>
## Summary
Add E2E coverage for vanilla FAISS index support across Milvus clients:
- Add Python client index tests for vanilla FAISS factory strings,
metrics, vector types, search params, scalar filters,
release/load/search, range search, and expected unsupported behavior
- Add Go client coverage through the generic index API
- Add REST client create/list/describe metadata coverage
Dependency note: this PR is temporarily based on Milvus #49912 and
currently depends on the Knowhere `faiss-passthrough` branch. The
dependency will be replaced after the final Knowhere reference is
available.
issue: #50124
## Tests
- python -m py_compile
tests/python_client/testcases/indexes/idx_faiss.py
tests/python_client/testcases/indexes/test_faiss.py
tests/restful_client_v2/testcases/test_index_operation.py
- pytest -c /dev/null
tests/python_client/testcases/indexes/test_faiss.py -q -o
cache_dir=/tmp/pytest-cache-vanilla-faiss --tb=short
- pytest tests/restful_client_v2/testcases/test_index_operation.py -q -k
faiss --tb=short -o cache_dir=/tmp/pytest-cache-vanilla-faiss-rest
- cd tests/go_client && go test ./testcases -run
TestCreateIndexVanillaFaissGeneric -count=1 -v
---------
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
## Summary
- Align snapshot wrapper and test call signatures with the latest
PyMilvus snapshot API.
- Update Python client test dependency to `pymilvus==3.1.0rc5`.
- Make snapshot test cleanup safe under `pytest -n 6` by isolating
cleanup lists and tracking RBAC users/roles per test.
- Skip the `CreateSnapshot`/`DropCollection` race case and link it to
#49761.
- Remove stale xfail marks for snapshot RBAC privilege group cases now
covered by the fixed server behavior.
## Test plan
- `.venv/bin/python -m py_compile base/client_base.py
base/client_v2_base.py milvus_client/test_milvus_client_snapshot.py`
- `git diff --check`
- `.venv/bin/python -m pytest -q
milvus_client/test_milvus_client_snapshot.py --dist loadgroup -n 6
--host 10.100.36.198 --port 19530 --tb=short`
- `119 passed, 1 skipped in 293.48s`
- Cleanup audit after the full run:
- `collections []`
- `users ['root']`
- `roles ['admin', 'public']`
- `databases ['default']`
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
## Summary
- **TestPartitionsNumExceedsMax**: Use management API (`POST
:9091/management/config/alter`) to temporarily lower
`rootCoord.maxPartitionNum` to 10 during the test, reducing 1023 serial
`CreatePartition` RPCs to 9. This eliminates the ~50% timeout failure
rate on branch triggers. Falls back to original behavior with 600s
timeout if management API is unavailable.
- **test_invalid_sparse_inverted_index_algo**: Update expected error
code from 999 to 1100 (`ErrParameterInvalid`) and add three new sparse
index algorithms (`BLOCK_MAX_MAXSCORE`, `BLOCK_MAX_WAND`, `SINDI`) added
in #49041 but never reflected in the test assertion. This test is L2
(nightly-only) and has been 100% failing since Apr 20.
- **common_type.py**: Sync `inverted_index_algo` list to match
server-side `SparseInvertedIndexAlgos`.
issue: https://github.com/milvus-io/milvus/issues/49135
## Test plan
- [ ] `go test -tags dynamic,test -gcflags="all=-N -l"
./tests/go_client/testcases/... -run TestPartitionsNumExceedsMax` passes
consistently
- [ ] CI go-sdk pipeline passes without flaky timeout
- [ ] Python test_invalid_sparse_inverted_index_algo assertion matches
server response
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
this PR allows to create Sort/Bitmap/Hybrid index for json path index,
and also removed the `JsonInvertedIndex` class, to use
`InvertedIndexTantivy` directly for Inverted index type.
C++ changes:
- Add ConvertJsonToTypedFieldData<T>() to extract typed values from JSON
field data with separate tracking of non_exist_offsets for EXISTS
semantics
- Add JsonScalarIndexWrapper<T, BaseIndex> template for Sort/Bitmap with
dual file-manager pattern (original JSON schema for reading, cast-type
schema for base index dispatching)
- Add JsonHybridScalarIndex<T> with validity-aware cardinality counting
- Add IndexBase::Exists() virtual method; override in Sort/Bitmap/Hybrid
wrappers using non_exist_offsets (serialized via
WriteEntries/LoadEntries)
- Simplify ExistsExpr to use index->Exists() uniformly
- Extend IndexFactory::CreateJsonIndex() to route STL_SORT/BITMAP/HYBRID
Go changes:
- Update STL_SORT/Bitmap/Hybrid checkers to accept JSON with cast_type
and json_path validation
- Change AUTOINDEX default for JSON from INVERTED to HYBRID
- Bump ScalarIndexEngineVersion to 4
- Add version gate in DataCoord CreateIndex and snapshot RestoreIndexes
- Update test fixtures to cover new (index_type, cast_type) combinations
<img width="2380" height="708" alt="image"
src="https://github.com/user-attachments/assets/8b3923a0-2cb3-4af7-b73a-b76d1d1ec2d0"
/>
issue: https://github.com/milvus-io/milvus/issues/48954
design-doc:
https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260410-json_path_index_multi_type.md
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
## Summary
- Mechanical cleanup of accumulated ruff lint debt in two
`tests/python_client/` files so future PRs that touch them aren't
blocked by file-scope lint failures unrelated to their actual change.
- Both files now pass `ruff check` and `ruff format --check` against
`tests/ruff.toml`.
- No behavior change. `pytest --collect-only` still collects the
same 265 tests in `test_index.py`.
issue: #49543
First follow-up cleanup for the framework introduced in #49130, which
intentionally deferred mass-reformatting the existing 316 `.py` files
under `tests/` ("will be cleaned up incrementally in follow-up PRs per
sub-directory").
## What changed
`tests/python_client/check/func_check.py` (25 errors)
- `ruff --fix --unsafe-fixes` handles `UP032` (`.format()` → f-string),
`F541` (empty-placeholder f-strings), `F841` (one unused
`error_code` local), `I001` (import order), `F401` (unused imports).
- Four `UP031` percent-format strings hand-converted to f-strings.
- One orphan subscript expression left behind by the unsafe-fix pass
removed.
`tests/python_client/testcases/test_index.py` (60+ errors)
- Same auto-fix sweep, plus `E712` (`== True` → truthy check).
- Star imports made explicit:
- `from common.constants import *` → dropped (only
`default_entities` was referenced, and only inside a commented-out
line).
- `from utils.util_pymilvus import *` → narrowed to `MyThread`,
`default_dim`, `default_float_vec_field_name`,
`default_binary_vec_field_name`.
- The `time` module that the star import used to re-export is now
imported directly.
## Test plan
- [x] `ruff check --config tests/ruff.toml` passes on both files.
- [x] `ruff format --check --config tests/ruff.toml` passes on both
files.
- [x] `pytest --collect-only
tests/python_client/testcases/test_index.py`
collects 265 tests (unchanged).
- [ ] CI's `Python Lint (tests/)` job goes green (was the original
block).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
issue: #49202
This PR consolidates Python client tests by moving the remaining
milvus_client_v2 search tests into the main milvus_client suite and
removing the milvus_client_v2 folder.
It also reduces repeated collection/index setup in the data integrity
expression test by looping through expression fields within a single
collection setup.
Verification:
- python3 -m py_compile on touched Python test files
- PYTHONPATH=tests/python_client python3 -m pytest -c /dev/null
--collect-only -q for migrated tests and the refactored data integrity
test
---------
Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes#48917
This fixes the multilingual NGRAM test by removing the incorrect
even-distribution count assertion. The test data is inserted in two
batches of 3000 rows, and each batch resets keyword assignment with j %
16, so keywords are not distributed evenly across all 6000 rows.
The test now verifies the actual intent: NGRAM-indexed JSON LIKE results
are non-empty and exactly match the non-indexed JSON LIKE scan results
for each multilingual keyword.
Verification:
```bash
source /Users/yanliang.qiao/fork/milvus/tests/python_client/.venv/bin/activate
cd tests/python_client
python3 -m pytest testcases/indexes/test_ngram.py::TestNgramBuildParams::test_ngram_search_with_multilingual_utf8_strings -q --host 10.104.15.131
```
Result: 1 passed in 19.32s
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
## Summary
- Add E2E test for high-cardinality GROUP BY aggregation (issue #47569)
- Tests GROUP BY with 2500 unique group keys (above the original 1792
HashTable slot limit)
- Covers both growing segment (before flush) and sealed segment (after
flush)
- Verifies group count, uniqueness, and aggregation correctness against
pandas ground truth
- Lightweight: 3000 rows, ~7 seconds runtime
issue: #47569
## Test plan
- [x] `pytest
testcases/test_query_aggregation.py::TestQueryAggregationIndependentV2::test_high_cardinality_group_by`
— passed in 6.59s
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Add two new tests for the `query_mode=large_topk` collection property:
- **`test_create_index_after_insert_with_large_topk`** (L1): Verifies
`create_index` succeeds on a populated collection with
`query_mode=large_topk`, and that large topk search (limit > 16384)
works correctly after index build. This isolates
data-at-index-build-time behavior from the alter/drop property flow.
- **`test_large_topk_boundary_2m_rows`** (L3): Verifies topk boundary
enforcement with 2M rows:
- `limit = 999,999` → succeeds
- `limit = 1,000,000` (max) → succeeds, returns 1M results
- `limit = 1,000,001` → fails with expected error
Also adds `large_topk_max = 1_000_000` constant documenting the maximum
supported topk.
### Test Plan
- [x] `test_create_index_after_insert_with_large_topk` (L1) — verified
passing on a standalone Milvus instance
- [x] `test_large_topk_boundary_2m_rows` (L3) — boundary searches
verified manually: limit=1,000,000 passed in 33.8s, limit=1,000,001
returned expected error
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
## Summary
When a user provides only `range_filter` without `radius` in search
params, the request silently degrades to a regular top-K ANN search —
the `range_filter` constraint is completely ignored and results violate
the specified bound.
This fix adds early validation in the proxy layer to reject such
requests with a clear `ErrParameterInvalid` error.
**Root cause**: The C++ core (`CheckAndUpdateKnowhereRangeSearchParam`
in `Utils.cpp`) uses `radius` as the gate to enter range search mode.
Without `radius`, it returns `false` and the caller falls back to top-K
search, silently discarding `range_filter`. The Go proxy layer had no
validation to catch this before it reached the core.
**Affected metric types**: COSINE, IP, L2 — all three ignore
`range_filter` when `radius` is absent.
issue: #48915
## Changes
- `internal/proxy/search_util.go`: reject `range_filter` without
`radius` with `ErrParameterInvalid`
- `internal/proxy/task_search_test.go`: add unit test
`range_filter_without_radius` covering the new validation
## Test Plan
- [x] New unit test
`TestSearchTask_parseSearchInfo/parseSearchInfo_error/range_filter_without_radius`
verifies the error is returned
- [x] gofumpt and gci checks pass on changed files
- [x] Manually reproduced the bug on a live Milvus instance (COSINE, IP,
L2 all affected)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- Migrate all RBAC test cases into a single file
(`test_milvus_client_rbac.py`) using MilvusClient API, with 5 test
classes and 136 tests covering user/role/privilege CRUD, invalid params,
advanced scenarios, privilege groups, and grant v2 API
- Remove 6 duplicated ORM-style RBAC classes (~4700 lines) from
`test_utility.py` and 1 from `test_connection.py`
- Add robust shared teardown (`_teardown_rbac`) with cross-db privilege
revocation, v2 API fallback, and proper user-role unbinding
## Test plan
- [x] All 136 RBAC tests pass against live Milvus instance
(master-20260313-541d3a8)
- [x] Environment fully cleaned up after test run (no residual
users/roles/dbs/collections)
- [x] Teardown handles edge cases: cross-db privileges, custom privilege
groups, v1/v2 API fallback
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: nico <cheng.yuan@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## What does this PR do?
Supplements #48736 with missing interface coverage for
`query_mode=large_topk` collection property.
### New test cases (4)
**`TestLargeTopkShared`** (shared collection, read-only):
- `test_query_large_limit` (L1): `query()` with `limit=16385` succeeds
when property is set
- `test_query_without_property_fails` (L2): `query()` with `limit=16385`
is rejected without property
**`TestLargeTopkIndependent`** (per-test collection):
- `test_hybrid_search_large_topk` (L1): `hybrid_search()` with
`limit=16385` succeeds when property is set
- `test_hybrid_search_without_property_fails` (L2): `hybrid_search()`
with `limit=16385` is rejected without property
### Why not iterator interfaces?
`search_iterator` and `query_iterator` are **not affected** by
`query_mode=large_topk`:
- `batch_size > 16384` is rejected client-side by the SDK (ParamError,
independent of the property)
- Iterator `limit` (total result count) uses internal pagination with
`batch_size <= 16384` per request, so per-request topk never exceeds
16384
### Error message note
`query()` and `hybrid_search()` return a different error message than
`search()` for the same restriction:
- `search()` → `"topk [N] is invalid, it should be in range [1, 16384]"`
- `query()` / `hybrid_search()` → `"invalid max query result window,
(offset+limit) should be in range [1, 16384], but got N"`
All 4 tests verified against a live Milvus instance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
Rebase of closed PR #48348. Fixes test compatibility issues across
multiple test files for none_default, TTL, range_search, and alias
tests.
### Key Changes
- Fix TTL test to verify search/query across all consistency levels
(Eventually, Bounded, Session, Strong)
- Use parameterized field names instead of hardcoded strings
- Repair error codes and trivially-true assertions in range search/alias
tests
- Use `request.cls` to set shared_data as class attribute in fixture
- Use NullValue sentinel for SQL NULL semantics in expression filter
tests
- Update alias error messages to match current Milvus server responses
### Changes
**Modified Files:**
- `test_milvus_client_ttl.py`: Multi-consistency-level search/query
verification
- `test_milvus_client_range_search.py`: Fix error codes and assertions
- `test_milvus_client_search_none_default.py`: NullValue sentinel,
fixture fix
- `test_milvus_client_alias.py`: Updated error messages
issue: https://github.com/milvus-io/milvus/issues/48348🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LoadBalance uses Contains() which matches all node types (RW, RO, SQ).
With streaming nodes in replicas, a non-existent query node ID could
match a streaming node, bypassing the replica check and producing a
misleading "segment not found" error instead of "node not found".
Fix server to use ContainRWNode() for both src and dst node validation,
and fix E2E tests to derive invalid node IDs from all nodes (not just
query nodes) to avoid collisions with streaming node IDs.
issue: #48674
Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Migrate 25 insert test cases from v1 ORM style
(`testcases/test_insert.py`) to v2 MilvusClient style
(`milvus_client/test_milvus_client_insert.py`)
- Delete migrated v1 test cases, retain
DataFrame/column-based/async-specific tests that have no v2 equivalent
- New v2 test `test_insert_with_pk_varchar_auto_id_true`: validates
varchar PK with auto_id=True, includes query verification for
auto-generated IDs
## Related Issues
#48048
## Test plan
- [x] Verified `test_insert_with_pk_varchar_auto_id_true` passes against
local Milvus (port 19531)
- [x] Cross-checked all 58 v1 cases: 25 migrated, 33 skipped
(DataFrame/column-based/v1 async), 0 remaining
There are a lot of insert test cases remain skipped, you can view
[test_insert.py Migration
Checklist](https://zilliverse.feishu.cn/docx/YAsedYP1go9qlexDyi0cgKwgnIj)
for reason.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## What does this PR do?
Fix a flaky test in `test_milvus_client_hybrid_search_v2.py`.
## Why?
`test_hybrid_search_as_search` asserts that hybrid search (single field
+ WeightedRanker) returns the same ordered result list as a regular
search. When two results have tied scores, tie-breaking order is
non-deterministic across architectures and runs due to floating-point
differences — this causes spurious failures on ARM e2e CI.
Comparing ID sets instead of ordered lists preserves the intent of the
test (same result candidates returned) while being robust to ordering of
ties.
---------
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
## Summary
- Migrate 15 partition key test cases from
`testcases/test_partition_key.py` (v1 ORM style) to
`milvus_client/test_milvus_client_partition_key.py` (v2 MilvusClient
style)
- Migrate 4 partition key isolation test cases from
`testcases/test_partition_key_isolation.py` to
`milvus_client/test_milvus_client_partition_key_isolation.py`
- Optimize isolation test parameters to prevent CI timeout: data_size
10000→1000, dim 768→128, HNSW M=30→16, efConstruction=360→64
- Remove migrated v1 test files
## Related Issues
#48048
## Test plan
- [x] All 19 migrated test cases (46 parametrized combinations) pass on
cluster (19531)
- [x] Partition key isolation tests (4 cases) all pass (~68s total, down
from >360s CI timeout)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Migrate 50+ partition test cases from `testcases/test_partition.py`
(v1 ORM style) to `milvus_client/test_milvus_client_partition.py` (v2
MilvusClient style)
- Add `gen_default_rows_data_for_upsert` helper in `common_func.py` for
row-based upsert data generation
- Enhance existing v2 tests with partition name validation and `None`
partition name coverage
- Remove migrated test cases from v1 file, keeping only v2-incompatible
cases (description params, dataframe insert)
## Related Issues
#48048
## Test plan
- [x] v1 remaining tests pass on standalone (19530) and cluster (19531)
- [x] v2 full test suite passes on standalone (148 passed, 3 failed -
replica tests need cluster)
- [x] v2 full test suite passes on cluster (151 passed, 0 failed)
- [x] v2 tests pass with 8-worker concurrency (`-n 8`) on both
environments
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Fixes#47316
- **GroupingSet::outputRowCount()**: Add null check for `lookup_` to
prevent SIGSEGV when filter matches zero rows and no input data arrives
- **ExecPlanNodeVisitor::setupRetrieveResult()**: When query result is
nullptr, build empty field_data arrays with correct schema (0 rows, N
columns) so downstream receives proper column structure instead of
missing columns
- Enable previously skipped E2E test `test_empty_result_aggregation`
## Test plan
- [x] E2E test `test_empty_result_aggregation` passes (GROUP BY +
aggregation with filter matching 0 rows returns empty result `[]`)
- [ ] Existing aggregation E2E tests still pass
- [ ] CI green
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Add 4 test cases to `TestQueryAggregationSharedV2` covering GROUP BY on
nullable fields, verifying the fix for #47350 (PR #47445).
**Related issue**: #47350
### Changes
**Schema additions** to the shared collection:
- `c10_nullable_varchar`: VARCHAR (nullable, 7 unique values + ~15%
NULL)
- `c11_nullable_int16`: INT16 (nullable, 5 unique values + ~15% NULL)
**New tests:**
| Test | What it verifies |
|------|-----------------|
| `test_group_by_nullable_varchar_field` | GROUP BY nullable VARCHAR
returns actual values (not all NULL) + aggregation correctness |
| `test_group_by_nullable_int16_field` | GROUP BY nullable INT16 returns
actual values + aggregation correctness |
| `test_multi_column_group_by_with_nullable_field` | Multi-column GROUP
BY with one nullable column |
| `test_search_group_by_nullable_field` | Search with GROUP BY on
nullable field returns actual group values |
### Test Plan
- [x] All 4 new tests passed against Milvus 2.6.6
- [x] All 28 existing tests still pass (0 regressions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
## Summary
- Enable search by ids tests that were commented out pending fix of
#47065
- Fix syntax error in test: `for i in len()` → `for i in range(len())`
- Update test expectations to match the fixed behavior:
- Search on null vectors now returns empty results instead of error
- Fix assertion to use correct `num_entities_with_not_null_vector` count
## Test plan
- [x] `test_milvus_client_add_nullable_vector_field_search` - verified
passing
- [x] `test_milvus_client_collection_null_vector_field_search` -
verified passing
issue: #47065🤖 Generated with [Claude Code](https://claude.ai/code)
---------
Signed-off-by: Yan Liang <yanliang@zilliz.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
## Summary
- Add mock TEI server utility (`common/mock_tei_server.py`) for testing
text embedding functions
- Add test cases for PR #46984 fix (alter function when other function
is invalid)
## Test Cases
1. `test_alter_function_when_other_function_is_invalid` - Verify that
altering a valid function succeeds even when another function in the
collection is invalid
2. `test_alter_invalid_function_to_valid_endpoint` - Verify that users
can fix an invalid function by altering it to a valid endpoint
## Related Issues
- Issue: https://github.com/milvus-io/milvus/issues/46949
- Fix PR: https://github.com/milvus-io/milvus/pull/46984
## Test Plan
- [x] Verified test fails with unfixed Milvus
(`master-20260112-7bcd3b10`)
- [x] Verified test passes with fixed Milvus (`master-20260112-b39ecb0`)
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
Migrate advanced upsert valid test cases from ORM-based implementation.
- concurrent upserts on the same primary key
- multiple upsert operations
- dynamic field support
- default and None value handling
- sparse vector upsert cases
Signed-off-by: zilliz <jiaming.li@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
Migrate core upsert valid test cases from ORM-based implementation.
- upsert with non-existing / existing primary keys
- upsert with auto_id enabled
- upsert with string primary key
- upsert with binary vector data
- upsert with data identical to inserted data
- upsert in specific or mismatched partitions
---------
Signed-off-by: zilliz <jiaming.li@zilliz.com>
