mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
wei liuandGitHub
1b9c0b423d
issue: #49241 pr: https://github.com/milvus-io/milvus-proto/pull/586 design doc: https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260423-array_partial_update_ops.md ## Summary Phase-1 implementation of field-level partial update operators for Array fields. Lets clients mutate a single array without resending the full value on upsert. Supported ops (see `schemapb.FieldPartialUpdateOp.OpType`): - `REPLACE` — legacy semantics (full overwrite). Default when no op is set. - `ARRAY_APPEND` — concatenate payload to base (duplicates preserved, in-order). - `ARRAY_REMOVE` — delete **all** occurrences matching any value in payload. ## Changes ### Proto surface (via https://github.com/milvus-io/milvus-proto/pull/586) - `UpsertRequest.field_ops` carries per-field directives; `FieldData` stays a pure data carrier (no leakage into Insert/Query/Search/msgstream). ### Proxy (`internal/proxy/task_upsert*.go`) - `validateFieldPartialUpdateOps` rejects: empty field name, duplicate field, op on PK, unknown field, non-Array field, element-type mismatch, unsupported op enum, op without matching FieldData, ARRAY_APPEND payload + base > `max_capacity`. - Any non-REPLACE op auto-promotes `partial_update=true` (user does not need to set both). - In `queryPreExecute` merge step, dispatches per field: REPLACE → `UpdateFieldDataByColumn`; ARRAY_APPEND/REMOVE → `UpdateArrayFieldByColumnWithOp`. ### typeutil (`pkg/util/typeutil/schema.go`) - `UpdateArrayFieldByColumnWithOp` (Array-only entry point) + `ApplyArrayRowOp` + `appendArrayRow` / `removeArrayRow` cover all Array element types (Bool/Int32/Int64/Float/Double/VarChar). - REMOVE deletes every match (aligns with MongoDB `$pull`, Postgres `array_remove`). - Float/Double REMOVE: `NaN != NaN` (IEEE 754). - ARRAY_APPEND enforces `max_capacity` per merged row and returns a descriptive error. ### Go SDK (`client/v2`) - `columnBasedDataOption.WithArrayAppend(field)` / `WithArrayRemove(field)` / `WithFieldPartialOp(field, op)` build `UpsertRequest.FieldOps`. - REPLACE clears any prior directive for that field; non-REPLACE ops auto-promote `PartialUpdate=true`. - Unknown-field ops are forwarded as-is so the server can return the descriptive validation error (client does not silently drop). ### Integration test (`tests/integration/hellomilvus/upsert_partial_op_test.go`) End-to-end insert → op-upsert → query flow exercising: ARRAY_APPEND, ARRAY_REMOVE (incl. no-op row + multi-match), ARRAY_APPEND-over-capacity rejection. Does **not** set `partial_update=true` — asserts the server auto-promotes. ## Forward compatibility Older servers that don't understand `field_ops` treat the request as a plain upsert (proto3 unknown-field tolerance) → graceful fallback to REPLACE semantics with a client-side warning path available. ## Test plan - [x] typeutil UT: 40+ cases covering all element types, empty/nil inputs, capacity edge cases, NaN semantics, unsupported-type rejection - [x] proxy UT: 20+ validation cases (PK, unknown field, type mismatch, capacity precheck, duplicate, empty name, op without FieldData) - [x] go-sdk UT: 10 cases — auto-promote, REPLACE-clears-prior, row-based + column-based builders, multi-op emission - [x] integration test: append / remove / capacity-overflow end-to-end - [x] `go mod tidy` in root / pkg / client - [ ] CI green 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Go MilvusClient
Go MilvusClient for Milvus. To contribute code to this project, please read our contribution guidelines first.
Getting started
Prerequisites
Go 1.24.12 or higher
Install Milvus Go SDK
-
Use
go getto install the latest version of the Milvus Go SDK and dependencies:go get -u github.com/milvus-io/milvus/client/v2 -
Include the Go MilvusClient in your application:
import "github.com/milvus-io/milvus/client/v2/milvusclient" //...other snippet ... ctx, cancel := context.WithCancel(context.Background()) defer cancel() milvusAddr := "YOUR_MILVUS_ENDPOINT" cli, err := milvusclient.New(ctx, &milvusclient.ClientConfig{ Address: milvusAddr, }) if err != nil { // handle error } // Do your work with milvus client
API Documentation
Refer to https://milvus.io/api-reference/go/v2.5.x/About.md for the Go SDK API documentation.
Code format
The Go source code is formatted using gci & gofumpt. Please run make lint-fix before sumbit a PR.