Commit Graph
3 Commits
Author SHA1 Message Date
5def5ced4a enhance: enforce function-field binding principle for function DDL (#51360)
## What

Enforce a function–field binding principle for function DDL (add / drop
/ alter function on an existing collection), so a function and its
output field stay coupled and already-stored vectors can never be
silently invalidated by DDL.

BM25 and MinHash follow the strict binding: add/drop only via the
unified `add_function_field` / `drop_function_field`, output field
always coupled. **TextEmbedding is a temporary carve-out** — its legacy
`AddCollectionFunction` / `DropCollectionFunction` paths are retained,
so attach-over-existing-field and detach are still possible for it,
because `add_function_field` embedding backfill is not yet implemented.
It will be folded into the unified path in a follow-up. So the "always
coupled" invariant currently holds for BM25/MinHash, not TextEmbedding.

## Changes

- **AlterCollectionSchema invariant**: reject standalone add-function (a
function must be added together with its new output field), reject
detaching a function without dropping its output field, and reject
function cascade (a new function's input being another function's
output).
- **Legacy RPCs**: `AddCollectionFunction` / `DropCollectionFunction`
are rejected for BM25/MinHash (must use `add_function_field` /
`drop_function_field`) and retained only for TextEmbedding (see
carve-out). `AlterCollectionFunction` is kept and gains the whitelist
below. In the alter and legacy-add paths, function field IDs are always
re-derived from field names (never trusted from the request), so a
request cannot inject an unrelated field ID that a later
`drop_function_field` would delete.
- **alter_function whitelist**: only connection/runtime params may
change (TextEmbedding: `url`, `credential`, `timeout_ms`,
`max_client_batch_size`, `region`, `location`, `projectid`, `user`);
BM25/MinHash have no alterable params. Function identity (type, name,
input/output fields) and output-shaping params (`dim`, `model_name`,
`endpoint` — the TEI model identity, `normalize`, `truncate*`, prompts,
...) are immutable. Params are normalized (keys lowercased, duplicate
keys rejected) so a crafted duplicate key cannot bypass the diff.
- **drop_function_field**: allow any output-producing function (dropping
needs no backfill), removing the previous BM25/MinHash-only restriction.

## Not in scope / follow-up

- Extending `add_function_field` to TextEmbedding (post-creation add
would require backfilling every existing row through the external
embedding model) — deferred; folding the TextEmbedding legacy paths into
the unified binding follows this.
- MinHash input-field analyzer immutability lives on a different DDL
path (`AlterCollectionField`); tracked as a separate follow-up.

## Breaking changes

- `DropCollectionFunction` on a **MinHash** function (detach — remove
the function but keep its output field) is no longer supported; use
`drop_function_field` instead (drops the function together with its
output field). If a released version supported MinHash detach, migrate
any such call to `drop_function_field`.

issue: #51348

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 05:04:39 +08:00
c322c84c22 test: cover add-field default values on growing segments (#51303)
## What this PR does

- Remove the `xfail` marker from the existing drop-and-readd
analyzer-field regression for #50484.
- Add L1 E2E coverage for the live QueryNode growing-segment reopen path
when an `enable_match` VARCHAR field with `default_value` is added to
loaded growing data.
- Verify the default value is immediately searchable on pre-existing
unflushed rows, eventually searchable on a subsequent defaulted insert
after the growing text index catches up, and remains searchable after
flush/reload.

Related fix: #51201

issue: #50484

## Scope note

This PR validates the live reopen path:

```text
loaded collection
-> insert rows that stay in a QueryNode growing segment
-> add an `enable_match` VARCHAR field with `default_value`
-> query triggers LazyCheckSchema/Reopen
-> default values for pre-existing growing rows are indexed and searchable immediately
```

It does not cover the QueryNode recovery / LoadGrowing path. That path
requires forcing a QueryNode restart, replacement, or channel rewatch
while the data is still growing: the new QueryNode can create the
segment with the latest schema and then load old binlogs written before
AddField. Since the segment schema is already current in that path,
LazyCheckSchema/Reopen is not the mechanism being tested here. Recovery
coverage should be handled by a focused stability test.

## Test results

-
`test_milvus_client_add_match_field_with_default_value_on_growing_data`:
3/3 passed
- `test_drop_then_add_same_name_analyzer_field`: passed
- Ruff lint and format checks: passed
- `git diff --check`: passed

Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-13 20:12:37 +08:00
Li YiyangandGitHub d8d823e782 test: add schema evolution e2e coverage (#50826)
## What
- Add drop field e2e coverage for MilvusClient.
- Add function field e2e coverage.
- Refine schema evolution e2e coverage.
- Bump pymilvus test dependency to 3.1.0rc49.

## Test
- Not run in this turn; PR contains Python e2e test coverage changes.

---------

Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
2026-06-30 11:02:31 +08:00