mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
sparknackandGitHub
8dad593ba5
- Replace `common.entryStream.streamBudgetRatio` with refreshable `common.loadTransientBudgetBytes`, a process-wide transient load budget shared by scalar index V3 entry streaming and storage v2/v3 field-data loading. `0` keeps the limit disabled. - Gate field-data batch reads on the shared budget, split batches by loading overhead, scale read parallelism from the effective batch budget, and finalize `GroupChunk` cells before pushing them so Arrow tables are released promptly. - Align MCL loading-overhead reservations for storage v1 scalar indexes and storage v2/v3 field data under one load-transient overhead group, with array-field overhead and mmap file usage accounted separately. - Propagate load cancellation into budget waits, field-data batch tasks, and V3 scalar `IndexEntryReader` stream/file reads so canceled loads do not stay blocked behind transient memory pressure. - Update segcore init/config watchers and add C++/Go coverage for budget sizing, cancellation, field-data batching/finalization, V3 entry streaming, and the new config. issue: #49499 --------- Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>