mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
6ca0350944
issue: #51372 ### Problem `MergeOp.buildResultArrays` builds a zero-hit chunk's `$id` array with a hardcoded Int64 builder (`operator_merge.go:713-721` before this change), while chunks that do have hits are typed from the first actual ID (`buildInt64Results` / `buildStringResults`). All per-query chunks then go into `AddColumnFromChunks`, which builds one `arrow.NewChunked` column out of them. For a **VarChar-PK collection** where **one query of an nq >= 2 batch returns zero hits** while another returns hits, those chunks mix `utf8` and `int64` and arrow panics: ``` panic: invalid: arrow/array: mismatch data type int64 vs utf8 ``` `MergeOp` sits at chain index 0, so every request that goes through a function rerank chain — plain search with a rerank function as well as hybrid search — can hit it. The proxy has no recovery interceptor on its gRPC chains (`internal/distributed/proxy/service.go:295-307`, `416-424`) and no `recover()` in `internal/proxy/`, so the panic takes the proxy process down. The input side was never affected: `collectRRFScores` / `collectWeightedScores` read IDs row by row, so an empty chunk is simply iterated zero times. Only the output side was broken. ### Fix Resolve the `$id` arrow type once per merge, from the first input that actually carries IDs (`resolveIDType`), and build the zero-hit chunk with that type. Zero-row inputs are deliberately skipped when resolving: an empty result carries no ID type of its own and is materialized as an empty Int64 column regardless of the collection's PK type (`importEmptyIDs`, converter.go). When *every* input is empty, Int64 is kept — every chunk is then empty as well, so the column is self-consistent and the rerank operator's `allEmpty` early-return short-circuits before it anyway. An unexpected arrow ID type now returns a `merr` error instead of silently producing an Int64 array. ### Testing - `TestMergeWithEmptyChunkVarCharIDs` — single input, VarChar IDs, `topks=[2,0]`; asserts the merged `$id` column stays `utf8` with an empty second chunk. Written first and observed panicking with `mismatch data type int64 vs utf8` on unmodified master. - `TestMergeMultiInputEmptyLegVarCharIDs` — hybrid-search shape: one leg entirely empty, the other with a zero-hit query. - The existing `TestMergeWithEmptyChunk` (Int64) still passes, pinning the unchanged Int64 path. - `./internal/util/function/...` green, including `-race`; `gofmt` clean. ### Note Found while reviewing #51156 (fix for #50969). This panic is **not** introduced by that PR — `operator_merge.go` is byte-identical between master and its head, and the repro above does not touch the lines it changes. #51156 does widen the reachable surface, though: before it, a single-shard zero-hit result failed earlier in the converter with `unsupported ID type` and never reached `MergeOp`; after it, that input builds a DataFrame and walks into this panic. Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>