82 Commits
Author SHA1 Message Date
yanliang567andGitHub d2bf17e734 test: Add external table add field E2E tests (#50420)
Related to #50416

### What changed

- Add MilvusClient E2E coverage for external table
`add_collection_field` on scalar and vector fields.
- Cover refresh/load/search/query behavior after adding fields,
including about 10% NULL values in newly added scalar/vector fields.
- Add non-Parquet source coverage for add scalar field on Lance,
Iceberg, and Vortex.
- Add full supported DataType matrix coverage for external add field,
including scalar, array, geometry, and vector fields.
- Add negative coverage for missing/duplicate `external_field`, type/dim
mismatch, unsupported public data types, add/drop function, and drop
field APIs.
- Update Python client test dependency pin to `pymilvus==3.1.0rc35`.

### Verification

- `python -m py_compile
tests/python_client/milvus_client/test_milvus_client_external_table.py
tests/python_client/common/external_table_common.py
tests/python_client/base/client_v2_base.py
tests/python_client/check/func_check.py`
- `python -m ruff check
tests/python_client/milvus_client/test_milvus_client_external_table.py
tests/python_client/common/external_table_common.py
tests/python_client/base/client_v2_base.py
tests/python_client/check/func_check.py`
- `python -m pytest -n 4
milvus_client/test_milvus_client_external_table.py --host 10.104.18.101
--port 19530 --minio_host 10.104.18.27 --minio_bucket yanliang-mas2
--tb=short -q -s`

Result:

```text
182 passed, 1 xfailed in 318.94s (0:05:18)
```

The xfailed case is linked to #50416 and validates that old-field search
still works before the server returns the current internal QueryNode
assert for newly added fields before refresh.

---------

Signed-off-by: Yanliang Qiao <yanliang.qiao@zilliz.com>
2026-06-17 14:02:23 +08:00
e2787d3981 enhance: standardize error handling on merr + Sys/Input classification (#50221)
issue: #47420

## What this PR does

Project-wide migration of raw `fmt.Errorf` / `errors.New` in function
bodies onto
the `merr` framework, plus the Sys-vs-Input error classification and the
machinery it drives (retriability, fine-grained metrics, segcore
unification),
plus the convention docs and a linter that keeps it from regressing.

Scope: storage, proxy, coordinators (root/data/query), query node, data
node,
`pkg/util` & `internal/util`, expression parser, message queue,
streaming, and
misc packages. Bare raw-error usages went from ~3000 to a ~340 allowlist
(package-level sentinels / build-tag / test sites).

---

## How to review this PR

It is large but the vast majority is mechanical. Changes fall into three
tiers;
spend review budget on Part 2 and Part 3.

### Part 1 — Mechanical standardization (low risk, verify by rule)

Each converted call follows one of a small fixed set of rules. To
review, check
that each site obeys the matching rule rather than reading every line:

| Pattern | Rule |
|---|---|
| `fmt.Errorf("...")` originating a new error | →
`merr.WrapErrXxxMsg("...")` with a code matching the failure's meaning |
| Adding context to an existing typed error | → `merr.Wrap(err, "...")`
/ `merr.Wrapf(...)` — **preserves** the inner code (never `WrapErr*Err`,
which overwrites it) |
| Errors inside the streaming subsystem | → `status.New*` factories
(StreamingError), **not** merr — this is the component-internal dialect
(see `docs/dev/error_handling_guide.md`) |
| Low-level / control-flow signal caught by `errors.Is` | → kept as a
package-level `errors.New` sentinel (lowercase, same-package) |

Conventions are documented in `docs/dev/error_handling_guide.md`
(how-to) and
`docs/dev/error_sentinel_convention.md` (rules + audit). A
`gocritic`/`ruleguard`
rule (`rawmerrerror`, in `rules.go`) enforces "no raw `return
errors.New/fmt.Errorf`"
under `make verifiers`.

### Part 2 — Behavior changes (review these closely)

These are the sites where the wire contract or runtime behavior changes,
not just
the source text. Listed by category; representative locations given,
full set in
the diff.

**A. gRPC wire-code shifts: `UnexpectedError(1)/Code 65535` → typed
code.**
Where a handler previously returned a raw error (collapsed to
`Code=65535` on the
wire), it now returns a typed merr, so the client sees a real code. The
most
common shift is to `IllegalArgument(5)/Code 1100` (ParameterInvalid).
Touch
points include datanode task handlers (CreateTask/Query/Drop), proxy
Upsert,
querynode GetMetrics, datacoord CreateIndex, httpserver query-response
builder,
and typeutil schema validation. One code refinement: an index-param
validation
moved `1100` → `1101` (ParameterMissing). **Client/SDK assertions and
any code
that switched on `Code=65535` for these paths must be re-checked** (the
go_client
e2e assertions were already aligned in this PR).

**B. Prometheus `status` label contract change (externally visible).**
The proxy metric's coarse `fail` / `rejected` values are split into
`fail_input` / `fail_system` and `rejected_user` / `rejected_system` (in
`requestutil.ParseMetricLabel`; auth/privilege rejections count as
`rejected_user`), so dashboards can attribute a failure to caller vs
operator.
**Dashboards/alerts querying `status="fail"` must migrate to
`status=~"fail_.*"`, and `status="rejected"` to
`status=~"rejected_.*"`.** The
in-repo Grafana dashboard is already migrated; external dashboards built
on the
old values silently go empty after upgrade. This is the one change that
requires an ops-side migration.

**C. Retriability semantics.**
- C1: `merr.Status(err)` now forces `Retriable=false` when the error is
an
`InputError` — a malformed request can never succeed on blind retry, so
clients
never get the self-contradictory "your input is wrong but you may
retry".
- C2: `retry.Do` short-circuits an `InputError` (non-retriable) — **but
only when
  the caller did not pass a `RetryErr` predicate**. The check is an
`if c.isRetryErr != nil { ... } else if InputError { ... }` *mutually
exclusive*
branch (`pkg/util/retry/retry.go`): an explicit `RetryErr` takes
precedence and
bypasses the InputError abort. `retry.Handle` deliberately does **not**
apply
the InputError abort (its callers signal abort via `shouldRetry=false`).
Four
flusher startup callsites that must retry through transient "not ready"
errors
  were given explicit `RetryErr` escape hatches.

**D. segcore (C++→Go) error classification.**
A single shared Go-side table (`pkg/util/merr/segcore.go`) maps each
segcore code
to a merr sentinel + InputError/signal category, replacing scattered
hand-written
`if errorCode == ...` switches in the cgo wrappers. **Wire `Code` values
change
for every segcore pass-through error, not just the remapped ones.**
Named
sentinels remap (C++ `2003` → merr `2001`, `2033` → `2002`,
Folly/Knowhere codes
likewise); **all remaining pass-through codes (`2004`–`2043`, previously
surfaced to clients as raw C++ enum values) now serialize as `2000`**
(`ErrSegcore`), with the original C++ code preserved in the `Reason`
text
(`segcoreCode=...`); unknown/future codes collapse to `2000` as well
(pinned by
the `wire_code_projection` test). Transient segcore classes (object
storage /
file IO / OOM / mmap / FieldNotLoaded — 11 codes) now report
`Retriable=true`.
**Any client switching on raw segcore codes in the `2004`–`2043` range
must be
re-checked**; the in-Reason code remains available for diagnostics.
Signal
codes (PretendFinished / FollyCancel) are recognized centrally.
`errors.Is`-based
control flow on these (e.g. scheduler skip/retry) is preserved.

**E. InputError classification (25 sentinels + dynamic marks).**
25 sentinels in `errors.go` carry `WithErrorType(InputError)` (the
Collection /
ResourceGroup / Database families, `ErrIndexDuplicate`,
`ErrParameterInvalid`,
`ErrPrivilegeNotAuthenticated`, `ErrImportFailed`, `ErrQueryPlan`, ...),
plus dynamic
marks for the 8 segcore input codes (ExprInvalid, DimNotMatch,
MetricTypeInvalid, FieldIDInvalid, ...) and
`WrapErrAsInputError`. The widest blast radius is `ErrParameterInvalid`
(1100):
~2335 `WrapErrParameterInvalid*` callsites now classify as input /
non-retriable. Because of C1/C2 this changes retriability for
any path that returns these. **The audit to confirm no transient path
was
mis-marked is the single most important review item** (see Part 3). One
reverse
correction: storage field-stats parsing moved from `ErrParameterInvalid`
(input)
to `ErrDataIntegrity` — a corrupted stored stat is data corruption, not
user
input.

### Part 3 — Known risks & traps (called out proactively)

1. **`merr.Wrap` vs `WrapErr*Err` (code-masking).** `WrapErr*Err` builds
a
`wrappedMilvusError{sentinel: ErrServiceInternal}` whose `code()`
returns the
*outer* sentinel — it overwrites the inner typed code and hides the
`errors.Is`
chain. This is intentional (use it to *deliberately* downgrade), but it
was a
recurring conversion defect; the rule "add context with `merr.Wrap`,
downgrade
with `WrapErr*Err`" is enforced by convention and reviewed across the
diff.
2. **InputError × `retry.Do` blast radius.** Marking a sentinel
`InputError` makes
any `retry.Do(...)` without a `RetryErr` predicate stop retrying it.
Reviewers
should sanity-check that no transient use of the 19 newly-marked
sentinels
(especially `ErrParameterInvalid`) sits inside a retry loop that needed
to keep
   spinning. The known flusher cases were handled (see C2).
3. **The ~340 raw-error allowlist.** What remains as bare `errors.New`
is, by
design: package-level sentinels (caught by `errors.Is`), `//go:build
test`
sites, and out-of-band trees (`cmd/`, `tests/`, codegen, walimpls). The
linter
only bans the *direct-return* form; assignment-then-return escapes and
the full
no-exceptions ban are deferred to an AST-based linter (Tier 2,
documented).
4. **segcore C++ second step deferred.** This PR unifies classification
on the Go
side; splitting the dual-semantic C++ codes at the source is a
follow-up.

---

## Validation

- `make verifiers`: Go side clean (gofmt + static-check across modules,
including
  the new `rawmerrerror` rule with a 0-hit baseline repo-wide).
- `make test-go`: passing; the one real regression introduced (a
datanode
`invalid_task_type` assertion shifting `1` → `5` from a ParameterInvalid
  conversion) was fixed in-tree.
- go_client e2e CreateIndex assertions aligned to the new merr messages.

---------

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 15:04:51 -07:00
e617ecb22c test: add struct array filter correctness tests for query and search (index access, array_contains, array_length) (#49178)
## Summary

End-to-end correctness tests for struct array filter expressions in both
`query()` and `search()`, covering:

- **Index access** `structA[i][sub_field]` — newly introduced by PR
#48987
- **`array_contains` / `array_contains_all` / `array_contains_any`** —
exact
  ground-truth `==` assertions to detect false negatives (the existing
  Test Case 1 only checks no-false-positives via per-hit walking)
- **`array_length(structA[sub_field])`** — positive cases plus rejection
  of `array_length(structA[i][sub_field])` (grammar disallowed)

## Why

Every case asserts **exact ID sets** (no false negatives AND no false
positives) instead of subset / per-hit checks, so any drop in recall or
spurious match fails the test. Coverage spans:

- All comparison operators (`==`, `!=`, `>`, `>=`, `<`, `<=`)
- `IN` / `NOT IN`
- Forward / reverse range syntax (`a < x < b` and `b > x > a`)
- String sub-field
- Compound predicates across positions (`structA[0]... &&
structA[1]...`)
- Doc-level filter combination
- Out-of-bounds index (rows shorter than the requested index)
- Parser-error negative cases (missing field, swapped form, indexed-form
  passed to `array_length`)
- **Per-segment consistency**: mirrored controlled rows in sealed vs
  growing produce mirrored result sets
- **After `delete`** and **after `upsert`** (both matching and
  non-matching rows, cross-segment)

## Layout

- query: `TestMilvusClientStructArrayIndexAccess` (19 cases) in
  `test_milvus_client_struct_array_element_query.py`
- search: `TestMilvusClientStructArrayIndexAccessSearch` (12 cases) in
  `test_milvus_client_struct_array_element_search.py`
  - uses FLAT index + controlled vector layout (controlled rows share
    the query vector at COSINE=1.0) so search recall is 1.0 and exact
    ID set assertions are valid

Each case uses **>= 3500 rows** (3000 sealed + 500 growing inert
background plus controlled rows split across both segments).

## Test plan

- [x] verified locally on a Milvus standalone instance built from PR
  #48987 (commit `0c5a964ea0`): **31 / 31 passed in 387.54s**
- [x] each case scoped to `id < 100` so inert background rows cannot
  contaminate strict-equality assertions

issue: #42148

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 19:04:12 +08:00
yanliang567andGitHub eea56c4eed test: Fix flaky L2 search assertions (#49633)
## What this PR does

Fixes #49542

This updates stale L2 search test assertions without changing the
original test coverage points:
- keep the Bounded consistency range-search case on Bounded, but allow
already-visible growing data to be returned
- keep the duplicate-primary-key search case focused on PK
de-duplication, without requiring de-duplicated results to still equal
the requested limit
- keep distance ordering validation, but compare floating-point ordering
with a small epsilon instead of strict list equality

## Tests

Against Milvus `10.104.18.67:19530`:

- `pytest -q --host 10.104.18.67 --port 19530 --tag L2
milvus_client/test_milvus_client_range_search.py::TestRangeSearchIndependent::test_range_search_with_consistency_bounded`
- 2 passed
- `pytest -q --host 10.104.18.67 --port 19530 --tag L2
milvus_client/test_milvus_client_search_by_pk.py::TestSearchByPkIndependent::test_search_by_pk_with_duplicate_primary_key`
- 1 passed
- `pytest -q --host 10.104.18.67 --port 19530 --tag L2
milvus_client/test_milvus_client_search_pagination.py::TestMilvusClientSearchPagination::test_search_with_pagination_topk`
- 3 passed

Flaky check with `-n 10 --count=10`:
- Bounded consistency: 20 passed
- Duplicate primary key: 10 passed
- Pagination topK: 30 passed

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
2026-05-09 16:00:08 +08:00
3650408fc1 test: pay down ruff lint debt in func_check.py and test_index.py (#49544)
## Summary

- Mechanical cleanup of accumulated ruff lint debt in two
  `tests/python_client/` files so future PRs that touch them aren't
  blocked by file-scope lint failures unrelated to their actual change.
- Both files now pass `ruff check` and `ruff format --check` against
  `tests/ruff.toml`.
- No behavior change. `pytest --collect-only` still collects the
  same 265 tests in `test_index.py`.

issue: #49543

First follow-up cleanup for the framework introduced in #49130, which
intentionally deferred mass-reformatting the existing 316 `.py` files
under `tests/` ("will be cleaned up incrementally in follow-up PRs per
sub-directory").

## What changed

`tests/python_client/check/func_check.py` (25 errors)
- `ruff --fix --unsafe-fixes` handles `UP032` (`.format()` → f-string),
  `F541` (empty-placeholder f-strings), `F841` (one unused
  `error_code` local), `I001` (import order), `F401` (unused imports).
- Four `UP031` percent-format strings hand-converted to f-strings.
- One orphan subscript expression left behind by the unsafe-fix pass
  removed.

`tests/python_client/testcases/test_index.py` (60+ errors)
- Same auto-fix sweep, plus `E712` (`== True` → truthy check).
- Star imports made explicit:
  - `from common.constants import *` → dropped (only
    `default_entities` was referenced, and only inside a commented-out
    line).
  - `from utils.util_pymilvus import *` → narrowed to `MyThread`,
    `default_dim`, `default_float_vec_field_name`,
    `default_binary_vec_field_name`.
- The `time` module that the star import used to re-export is now
  imported directly.

## Test plan

- [x] `ruff check --config tests/ruff.toml` passes on both files.
- [x] `ruff format --check --config tests/ruff.toml` passes on both
      files.
- [x] `pytest --collect-only
tests/python_client/testcases/test_index.py`
      collects 265 tests (unchanged).
- [ ] CI's `Python Lint (tests/)` job goes green (was the original
      block).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 11:52:13 +08:00
wtandGitHub 7aa8f374c0 test: add warmup e2e tests covering async/sync/disable policies at field/collection/index levels (#48726)
## Summary

- Add `TestMilvusClientWarmupAsync` class with 30+ async warmup test
cases covering field/collection/index level warmup, hasRawData=false
paths (IVF_PQ, IVF_SQ8, sparse), cross-feature combos (mmap, partition,
multi_vector, dynamic, text_match, JSON, add_field), DQL/DML
correctness, and bug-finding cases (cancel/reload, priority chain,
sync-vs-async result identity)
- Fix index-before-insert ordering bug in 4 existing
`TestMilvusClientWarmup` cases
- Add `@pytest.mark.skip` guards for L3 tiered-storage-dependent cases
- Remove unused variables and dead code

## Test plan

- [x] Run `TestMilvusClientWarmup` + `TestMilvusClientWarmupAsync` with
`-n 6 --host=10.104.24.55`
- [x] Result: 131 passed, 6 skipped (L3), 1 failed
(`test_warmup_x_multi_replica` — requires multi-replica cluster,
unrelated to this PR)
- [ ] L3 cases verified separately on tiered storage instance and
`lowPriority=0` instance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2026-04-08 09:37:37 +08:00
21266e0cb7 test: fix nightly E2E test assertions for master #650/#651 failures (#48584)
## Summary

Fix 16 consistently failing test cases from nightly CI master build 650
(2026-03-26) and build 651 (2026-03-27). All failures reproduced across
all 3 configs (kafka / pulsar-mmap / woodpecker).

- Nightly build 650:
https://jenkins.milvus.io:18080/job/Milvus%20Nightly%20CI(new)/job/master/650/
- Nightly build 651:
https://jenkins.milvus.io:18080/job/Milvus%20Nightly%20CI(new)/job/master/651/

## Changes

### 1. `param_check.py` — fix `output_field_value_check` for sparse and
bfloat16 fields

**Fixes:**
- `test_search_by_pk_with_output_fields_all` — `KeyError: -1` (3/3
configs)
- `test_search_with_output_fields_all@TestMilvusClientSearchBasicV2` —
`TypeError: ufunc 'equal' not supported` (3/3 configs)

**Root cause:** Sparse vector comparison used hardcoded
`original[-1][_id]` instead of pk-based lookup. bfloat16/float16 vectors
returned as `bytes` cannot be compared via numpy `ufunc 'equal'`.

**Fix:** Use pk-based index for sparse comparison; skip value check for
`bytes` type (bfloat16/float16/binary).

### 2. `test_milvus_client_hybrid_search_v2.py` — fix expected output
fields

**Fixes:**
- `test_hybrid_search_with_diff_output_fields` — `AssertionError` (3/3
configs)

**Root cause:** Test excluded ALL sparse fields from expected output,
but only BM25-generated sparse fields cannot be output. User-inserted
nullable sparse vectors CAN be output.

**Fix:** Only exclude BM25 sparse fields
(`sparse_vector_field_name1/2`), keep `nullable_sparse_vec_field_name`.

### 3. `test_milvus_client_search_by_pk.py` /
`test_milvus_client_search_v2_new.py` — exclude DISKANN from mmap test

**Fixes:**
- `test_search_by_pk_each_index_with_mmap_enabled_search[DISKANN]` —
`alter_index_properties expect True, but got False` (3/3 configs)
- `test_each_index_with_mmap_enabled_search[DISKANN]` — same error (3/3
configs)

**Root cause:** DISKANN is a disk-based index and does not support mmap.
`alter_index_properties` correctly rejects it.

**Fix:** Exclude DISKANN from the `dense_float_index_types` parametrize
list in both files.

### 4. `test_milvus_client_search_group_by.py` — rewrite nonexistent
field group_by tests

**Fixes:**
-
`test_search_group_by_nonexistent_field_on_dynamic_enabled_collection[nonexist_field]`
— `AssertionError` (3/3 configs)
-
`test_search_group_by_nonexistent_field_on_dynamic_enabled_collection[21]`
— `TypeError: object of type 'Error' has no len()` (3/3 configs)

**Root cause:** With dynamic field enabled, group_by on a nonexistent
field name returns 1 result (all rows have null for that field → one
group), not `limit` results. Passing `21` (int) as field name is an
invalid type that the server rejects.

**Fix:** Split into 2 tests:
`test_search_group_by_nonexistent_field_on_dynamic_enabled_collection`
expects `limit=1`; new `test_search_group_by_invalid_field_type` expects
`err_res` with code 65535.

### 5. `test_milvus_client_search_pagination.py` — fix search params for
pagination topk

**Fixes:**
- `test_search_with_pagination_topk[100]` — `AssertionError` (3/3
configs)
- `test_search_with_pagination_topk[3000]` — `AssertionError` (3/3
configs)
- `test_search_with_pagination_topk[10000]` — `AssertionError` (3/3
configs)

**Root cause:** Missing `metric_type` and low `nprobe=10` caused poor
recall at high offset, leading to empty or incorrect pagination results.

**Fix:** Add `metric_type: COSINE` and increase `nprobe` to 128.

### 6. `test_milvus_client_search_v2.py` — fix recall, exists
expression, and metric type error

**Fixes:**
- `test_search_with_expression_auto_id` — `recall too low: got 313,
expected >= 800` (3/3 configs)
- `test_search_with_expression_exists[json_field]` — `TypeError: object
of type 'Error' has no len()` (3/3 configs)
- `test_search_with_expression_exists[float_array]` — same error (3/3
configs)
- `test_search_with_invalid_metric_type[JACCARD]` — `KeyError:
'err_msg'` (3/3 configs)
- `test_search_with_invalid_metric_type[HAMMING]` — `KeyError:
'err_msg'` (3/3 configs)
- `test_search_with_output_fields_all@TestSearchV2Shared` —
`AssertionError` (3/3 configs)

**Fixes applied:**
- **recall**: Add `metric_type: COSINE` + `nprobe: 64` to search params
- **exists expression**: Remove `json_field` and `float_array` from
parametrize (top-level field `exists` expression is not valid for these
types)
- **invalid metric type**: Add `err_msg` assertion (was only checking
`err_code`, `KeyError` when test accessed missing key)
- **output_fields_all**: Add `new_added_field` to expected output fields
list

### 7. `test_milvus_client_sparse_search.py` — update error message for
invalid metric type

**Fixes:**
- `test_sparse_search_invalid_metric_type[L2]` — `got 65535 ... expected
1100` (3/3 configs)
- `test_sparse_search_invalid_metric_type[COSINE]` — same error (3/3
configs)

**Root cause:** Server error message changed from `"only IP is
supported"` to `"metric type not match: invalid
parameter[expected=IP][actual=...]"`.

**Fix:** Update `err_msg` assertion to match new format.

## Test plan

- [x] `test_upsert_preserves_element_filter` — verified PASSED locally
- [x] `test_insert_without_connection` — verified PASSED locally
- [ ] Full nightly CI run to validate all 16 fixes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: nico <cheng.yuan@zilliz.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:57:32 +08:00
17532517c6 test: Fix sparse/group_by/text_match/diskann/invalid/by_pk (#48347)
## Summary
- GPU→L1, zip() for positional index, add metric_type, fix spelling
- IVF_SQ8→DISKANN, skip DISKANN mmap, fix INT8 metric, filter assertions
- Fix false-pass bypass in check_search_results for empty hits

## Code Review Fix (0e4ad86c13)

### check_search_results: remove empty-hits bypass

`check_search_results` in `func_check.py` had a `if len(hits) == 0:
continue` guard that silently skipped all verification (including limit
check) when search returned no results. This caused tests with filters
matching zero rows to false-pass instead of failing.

Removed the 2-line guard entirely. The existing `assert len(hits) ==
check_items["limit"]` already handles this correctly — when limit is
specified and hits is empty, the assertion properly fails.

**Impact:** `test_search_with_scalar_field` in
`test_milvus_client_search_diskann.py` filters `int64 in [1, 2, 3, 4]`
on randomly-generated INT64 data (non-PK field uses full-range random
values). This matches 0 rows, and previously false-passed due to the
bypass. After the fix, it correctly fails. The filter in the test should
also be updated to use values that exist in the data.

Verified: ran all 6 PR test files (L0+L1, `-n 4`) — 88 passed, 1
expected failure (`test_search_with_scalar_field`), 9 skipped.

issue: #48048
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 10:51:30 +08:00
61e929c2cf test: Fix json/string/iterator/pagination verification (#48346)
## Summary
- Add check_task/filter assertions, fix hardcoded COSINE, add pk_range
to func_check
- Reduce L0 iterator combos, fix bare field names, increase
overlap_ratio

issue: #48048
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 10:47:29 +08:00
yanliang567andGitHub 4bbc01324b fix: handle NaN vs None comparison in output field value check for nullable fields (#47841)
## Summary

- Fix `output_field_value_check` in `param_check.py` to handle NaN vs
None comparison for nullable fields
- When nullable fields contain `None` values, `pandas.DataFrame`
converts them to `NaN`. The check function then fails comparing `NaN`
(from DataFrame) with `None` (from Milvus search results). This fix
treats both as equivalent null values.
- Remove resolved issue reference comments (`#47065`)

**Related issue**: #47065

### Changes

**Modified Files:**
- `tests/python_client/check/param_check.py`: Add NaN/None equivalence
check before asserting output field values
- `tests/python_client/milvus_client/test_add_field_feature.py`: Remove
resolved `#47065` comment
- `tests/python_client/milvus_client/test_milvus_client_collection.py`:
Remove resolved `#47065` comment

### Test Plan

- [x]
`test_search_by_pk_with_output_fields_and_consistency_level[Strong/Session/Bounded/Eventually]`
- 4 previously failing tests now PASS
- [x] All 98 search_by_pk tests: 93 passed + 4 fixed + 1 skipped
- [x] 13 null/nullable vector related tests all PASS

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
2026-02-25 16:14:46 +08:00
9ef7e25778 test: enable search by ids on null vectors tests for #47065 (#47559)
## Summary
- Enable search by ids tests that were commented out pending fix of
#47065
- Fix syntax error in test: `for i in len()` → `for i in range(len())`
- Update test expectations to match the fixed behavior:
  - Search on null vectors now returns empty results instead of error
- Fix assertion to use correct `num_entities_with_not_null_vector` count

## Test plan
- [x] `test_milvus_client_add_nullable_vector_field_search` - verified
passing
- [x] `test_milvus_client_collection_null_vector_field_search` -
verified passing

issue: #47065

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Signed-off-by: Yan Liang <yanliang@zilliz.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 14:17:49 +08:00
zhuwenxingandGitHub a80e003731 test: add snapshot API test cases (#47309)
## Summary
- Add comprehensive test cases for Milvus snapshot API functionality
- Cover snapshot create, list, describe, drop, restore operations
- Test various data types, index types, partition scenarios
- Add concurrent operation and data integrity tests
- Update pymilvus to 2.7.0rc122 for snapshot API support

## Test Coverage
- L0 smoke tests: basic snapshot lifecycle
- L1 tests: data types, partitions, indexes
- L2 tests: boundary conditions, negative cases, concurrency

## Related Issue
https://github.com/milvus-io/milvus/issues/44358

## Test plan
- [x] Run tests with `-n 6` parallel execution
- [x] Verify all tests pass (67 tests: 62 passed, 4 xfail)

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2026-01-28 19:41:33 +08:00
yanliang567andGitHub bdb56f6053 enhance: Optimize Python test logging from 60MB to 3-5MB per run (#47253)
## Summary

This PR optimizes Python test logging to reduce log file size from 60MB+
to 3-5MB per test run (90%+ reduction) while preserving full debugging
information for failed tests.

related issue:  #47256

### Key Improvements

- **Conditional Logging**: PASSED tests save only metadata, FAILED tests
preserve complete logs
- **Memory Buffering**: Eliminates runtime disk I/O (no file writes
during test execution)
- **Unified Reports**: Generates 2 formats
  - `test_report.json`: AI-friendly structured data
  - `test_report.html`: Human-readable with color-coded logs
- **Worker Parallelism**: Full support for `pytest -n` with data merging

### Performance Impact

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| **Log Size** (15K tests) | 60 MB | 3-5 MB | **-90%+** |
| **Memory Usage** (6 workers) | ~50 MB | ~450 MB | +400 MB |
| **CPU Overhead** | N/A | 0.0004% | Negligible |
| **Runtime I/O** | 75 seconds | 75 ms | **-99.9%** |

### Changes

**New Files:**
- `plugin/log_filter.py`: Conditional log handler with per-test memory
buffers
- `plugin/__init__.py`: Plugin package initialization

**Modified Files:**
- `config/log_config.py`: Simplified config (only JSON/HTML paths)
- `conftest.py`: Register log filter plugin
- `utils/util_log.py`: Removed redundant file handlers
- `utils/api_request.py`: Optimized API request logging
- `check/param_check.py`: Truncate long lists in error messages
- `common/common_func.py`: Removed verbose logging
- `pytest.ini`: Added log filter plugin

**Cleanup:**
- `milvus_client/test_add_field_feature.py`: Removed unused variable

### Test Plan

- [x] Syntax validation: All Python files pass `py_compile`
- [x] Module imports: All modules load successfully
- [x] Data structures: Verified serialization without report/buffer
objects
- [x] Log configuration: Confirmed only necessary attributes present
- [x] Small-scale test (17 tests):  Generated correct JSON + HTML
reports
- [x] Performance analysis: Memory/CPU profiling for 15K test scenario

### Example Output

**test_report.html** (human-readable):
- Beautiful color-coded UI
- Failed tests with full error traceback
- Expandable log sections by level (debug/info/warning/error)

**test_report.json** (AI-friendly):
```json
{
  "metadata": { "total_tests": 15000, ... },
  "summary": { "passed": 13500, "failed": 1500, ... },
  "tests": {
    "failed": [
      {
        "id": "test.py::test_func",
        "error": { "type": "AssertionError", ... },
        "logs": { "debug": [...], "error": [...] }
      }
    ]
  }
}
```

### Memory Safety

For the standard scenario (15K tests, 10% failure, 6 workers):
- Worker memory: ~53 MB each (318 MB total)
- Main process peak: ~112 MB
- Total system: **< 450 MB**  Safe

High-risk scenarios (>30% failure or >50K tests) may require optional
streaming optimizations (documented in performance analysis).

### Backward Compatibility

-  No breaking changes to external APIs
-  Existing test commands work unchanged
-  Compatible with all pytest plugins

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2026-01-27 16:37:33 +08:00
marcelo-cjlandGitHub 9e726e82d9 fix: improve error messages for nullable vector field operations (#46798)
issue: #46779 
related: #45993 

Return clear error when using is null/is not null filter on vector
fields
Return clear error when search by IDs with all null vectors
Fix nq mismatch when search by IDs with mixed null/valid vectors

Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
2026-01-08 17:45:25 +08:00
624147740b test: fix timestamptz e2e case failure on Jenkins Weekly (#46210)
Issue: #46188 

Bug was caused by inconsistent version of tzdata as well as wrong month
assignment in convert_timestamptz function.
Also fix when debug_mode=True the compare function can correctly return
True or False.

---------

Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
2025-12-09 16:09:15 +08:00
nicoandGitHub 43fe215787 test: update sdk version and skip some debug log (#46040)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-12-04 10:33:11 +08:00
yanliang567andGitHub 13a52016ac test: Update hybrid search tests with milvus client (#46003)
related issue: https://github.com/milvus-io/milvus/issues/45326

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-12-02 18:11:10 +08:00
yanliang567andGitHub 1da75c0ee2 test: Update hybrid search tests to milvus client style (#45772)
related issue: #45326

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-11-24 17:55:07 +08:00
zhuwenxingandGitHub 1e130683be test: add geometry datatype in checker (#44794)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-10-24 11:28:04 +08:00
yanliang567andGitHub d45274512c test: Refactor search tests and remove useless common functions (#43608)
related issue: #40698

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-08-05 11:15:39 +08:00
yanliang567andGitHub abb3aeacdf test: Refactor diskann and hsnw index, and update gen data functions (#43452)
related issue #40698
1. add diskann and hnsw index test
2. update gen_row_data and gen_column_data functions

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-07-23 22:04:54 +08:00
yanliang567andGitHub e8011908ac test: Add tests for partition key filter issue and ttl eventually search (#43052)
related issue: #42918
1. add tests for ttl eventually search
2. add tests for partition key filter 
3. improve check query results for output fields 
4. verify some fix for rabitq index and update the test accordingly
5. update gen random float vector in (-1, 1) instead of (0,1)

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-07-02 11:02:43 +08:00
qixuanandGitHub 3b2ed5815f test:add test cases for add field (#42472)
issue: #42126

---------

Signed-off-by: qixuan <673771573@qq.com>
2025-06-11 17:06:39 +08:00
nicoandGitHub c827f4b948 test: update sdk version and cases (#42077)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-06-03 14:16:32 +08:00
binbinandGitHub eea6b50fbb test: add test cases for int8 vector (#41957)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-05-23 09:24:28 +08:00
yanliang567andGitHub ee659d50db test: [E2e Refactor] update search basic tests and add a pk_name instead of hard code (#41669)
related issue: https://github.com/milvus-io/milvus/issues/40698

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-05-09 21:58:54 +08:00
nicoandGitHub 5c11c4add1 enhance: update pymilvus version (#41695)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-05-08 10:04:54 +08:00
yanliang567andGitHub 70b311735b test: [E2e Refactor] use vector datatype instead of hard code dataype names (#41497)
related issue: #40698 
1. use vector datat types instead of hard code datatpe names
2. update search pagination tests
3. remove checking distances in search results checking, for knowhere
customize the distances for different metrics and indexes. Now only
assert the distances are sorted correct.

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-04-25 10:46:38 +08:00
yanliang567andGitHub 6a9ecb760f test: Update binary pagination search assertion for more stable (#41370)
related issue: #41362

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-04-17 14:16:32 +08:00
binbinandGitHub 345538d10a test: Add test cases for part of json path index (#40445)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-03-10 14:30:07 +08:00
yanliang567andGitHub 4437dae479 test: Add search iterator with external filter func neg tests (#40303)
related issue: https://github.com/milvus-io/milvus/issues/39985

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-03-05 10:58:00 +08:00
yanliang567andGitHub afbe02009e test: Add test for search iterator external filter function (#39986)
related issue: #39985
1. add some test for external filter function
2. combine search iterator tests into one test file

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-21 11:25:53 +08:00
a5217e9f36 test: add database milvus client testcases (#39505)
/kind improvement

---------

Signed-off-by: laurazhao0611 <laurazhao@zilliz.com>
Co-authored-by: laurazhao0611 <laurazhao@zilliz.com>
2025-01-23 18:45:07 +08:00
qixuanandGitHub 228bff5e7b test: modify search iterator V2 case about alias/db/rbac for milvus client (#39400)
issue: #37548

Signed-off-by: qixuan <673771573@qq.com>
2025-01-20 10:25:03 +08:00
yanliang567andGitHub 731e882d22 test: Refactor pymilvus client v2 testbase and add a test for search hint (#38939)
issue: https://github.com/milvus-io/milvus/issues/38877
1. refactor pymilvus client v2 testcasebase
2. add a test for search hint
3. update pymilvus to 2.6

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-01-06 11:24:55 +08:00
ThreadDaoandGitHub ea339c13c6 test: add cases for async milvus client (#38699)
issue: #38697

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-25 16:54:50 +08:00
yanliang567andGitHub 6884319d03 test: [CP] add alter tests (#38659)
related issue: https://github.com/milvus-io/milvus/issues/38471

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-12-23 19:42:49 +08:00
yanliang567andGitHub af433ffd68 test: Add expression template tests and use error msg only as assertion (#37618)
related issue: https://github.com/milvus-io/milvus/issues/37451

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
zhuwenxingandGitHub c8dd665bf6 test: supplementing case for text match (#36693)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:31:40 +08:00
yanliang567andGitHub 0d376f1a8f test: Add query iterator test and seperate a new test file (#37085)
related issue: #37084

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-24 11:47:29 +08:00
binbinandGitHub 0d57ff01a6 test: add null and default test cases (#36612)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-08 17:25:24 +08:00
wtandGitHub 14be89df29 test: add case to enable offset cache and mmap at the same time (#36505)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-25 19:59:13 +08:00
binbinandGitHub 5ca4d5977a test: Add bulk insert related test cases for default and null support (#36219)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-18 19:33:17 +08:00
yanliang567andGitHub 2e434b2358 test: Add group size tests (#36240)
related issue: #36146

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-13 16:13:08 +08:00
wtandGitHub cb49b32358 test: add bitmap index cases (#35909)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-03 16:49:03 +08:00
yanliang567andGitHub 019d76c79d test: Enable group by tests in e2e (#31903)
enable groupby tests in e2e

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-04-08 10:25:16 +08:00
binbinandGitHub a556671119 test: add hybrid search cases (#29830)
issue: #29799

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-02-20 11:58:51 +08:00
nicoandGitHub 168260cba3 test: update test cases about rbac (#29088)
issue: #29087
RBAC cases fail a lot.
1. some cases are out of date, for example, the default value of db_name
has changed from "default" to "" in some apis
2. add time sleep after the action of grant or revoke, for it costs time
to take effect

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-01-30 11:33:02 +08:00
binbinandGitHub a847d56ac0 test: add part of milvus client test cases (#30222)
issue: #30221

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-01-26 09:25:01 +08:00
zhuwenxingandGitHub 24bbb12166 test: add assert message if assert failed (#30186)
add assert message if assert failed

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-01-23 16:07:01 +08:00