Cherry-pick of #49728 onto 2.6, adapted for the branch.
issue: #49585
pr: #49728
## Original change
- Move dropped segment index cleanup into the dropped segment recycle
path: when recycling a dropped segment, delete segment object files and
segment index files first, then remove segment-index meta, and finally
remove segment meta. If any step fails, the later state is preserved for
the next GC cycle to retry.
- Add `GetAllSegmentIndexes` so dropped segment GC can clean
segment-index meta even when the field index has already been marked
deleted and would be filtered by `GetSegmentIndexes`.
- Keep orphan index file/meta GC as a fallback for old data and partial
failure recovery.
## 2.6 adaptations (features absent on this branch)
- Dropped the snapshot-based GC protection path (`GetSnapshotMeta` /
`IsBuildIDGCBlocked` / `snapshotMeta`), which does not exist on 2.6.
- Dropped the `StorageV3` / packed manifest handling in
`removeDroppedSegmentFiles`; 2.6 only has `StorageV1`/`StorageV2`.
- `getDroppedSegmentIndexFiles` now returns `(segIndexes, indexFiles)`
without the snapshot-blocked flag.
- Backported only the PR-added tests that apply to 2.6 (snapshot/V3
tests dropped); index file path assertions use
`metautil.BuildSegmentIndexFilePath`.
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: #49991
pr: #50114
Target-size manual compaction should use the same candidate rules as
normal manual compaction. Share the candidate predicate so force merge
only plans healthy, flushed, visible, sorted, non-L0, non-L2 segments
and skips compacting, importing, or snapshot-protected segments.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #50098
pr: #50099
## Summary
2.6 backport of #50099. Cherry-pick of:
- `cf6dc32687` enhance: extend arrow IO thread pool config to DataNode
init
- `dcd82c723a` enhance: raise dataCoord.import.fileNumPerSlot default
from 1 to 4
Conflict resolved: import paths re-pointed from `pkg/v3/*` (master) to
`pkg/v2/*` (2.6). No behavior change beyond what's in the master PR.
## Why
See #50098 / #50099 for the full reproducer and analysis. Production
2.6.x cluster hit a 1.5M-row sort compaction stalled for >1 hour while
CPU sat at 12% — root cause is the DataNode Arrow IO thread pool being
stuck at Arrow's hard-coded default of 8 threads, with
`common.arrow.ioThreadPoolCoefficient` introduced in 2.6.16 (#49554)
having no effect on DataNode because the wiring was never extended
there.
## Test plan
- [x] Cherry-pick clean (only conflict was pkg/v3 → pkg/v2 import paths)
- [x] `gofmt` clean
- [ ] CI
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cherry-pick from master (PR still open)
pr: #50161
issue: #49996
## Summary
Pre-emptive cherry-pick of master PR #50161 to 2.6. Original PR is
currently still open on master; this CP PR is kept in sync with the
master PR.
Adjustments from master: `pkg/v3/log`, `pkg/v3/streaming/util/message`,
and `pkg/v3/util/merr` imports rewritten to `pkg/v2/...` to match the
2.6 branch convention. No logic changes.
See #50161 for the full root-cause writeup (CDC-replicated
`AlterLoadConfig` retry loop blocked by dropped-channel sentinel,
blocking rootcoord meta cleanup on the secondary).
This revision re-picks the latest master commit, which additionally adds
a `querycoordv2` unit test for `alterLoadConfigV2AckCallback`
(dropped-sentinel ack-with-no-op + generic-error propagation) and maps
`ErrChannelDroppedSentinel` to `NoReplicaAvailable` in `merr.oldCode`.
## Verification
- [x] Cherry-pick diff matches master PR diff (only `pkg/v3` → `pkg/v2`
import-path substitutions and hunk line-number drift; verified
line-by-line)
- [x] No conflict markers remain
- [x] File count and line counts match original commit (7 files,
+117/-7)
- [ ] make static-check (skipped by cherry-pick workflow)
- [ ] CI
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pr: #50180
## What
Backport #50180 to the 2.6 branch. Switch Python test-side Jieba usage
from the Python `jieba` package to `rjieba`, the Python binding for
`jieba-rs`.
## Changes
- Replace Python test requirements from `jieba==0.42.1` to
`rjieba==0.1.11`
- Update Python client and REST test helpers to call `rjieba.cut`,
`rjieba.cut_for_search`, and `rjieba.cut_all`
- Update analyzer expected-token helper to use `rjieba` for supported
mode/hmm cases
- Keep custom-dictionary validation focused on analyzer output because
`rjieba` does not expose dynamic dictionary APIs like Python
`jieba.add_word` or `set_dictionary`
- Format changed Python test files with Ruff
## Test
- `ruff check` on changed Python files
- `ruff format --check --diff` on changed Python files
- `python -m py_compile` on changed Python files
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
issue: #48115
pr: #48537
Cherry-pick of #48537 (master) to the 2.6 branch. Same one-line root
cause; production-confirmed on a `v2.6.12-hotfix4` cluster where REST
`/v2/vectordb/roles/grant_privilege_v2` is rejected by the Cloud
`denyAPI` hook because the handler hands the legacy `OperatePrivilege`
fullMethod into `wrapperProxy`.
Also adds a regression test `TestGrantPrivilegeV2_FullMethodMismatch`
that injects a recording `hook.Hook` via `hookutil.SetTestHook`, fires
the V2 grant route, and asserts the captured fullMethod is
`OperatePrivilegeV2` — without this guardrail a future copy-paste
regression would silently re-introduce the Cloud-blocking behavior.
Companion hotfix PR: #50133
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Bingyi Sun <sunbingyi1992@gmail.com>
issue: #50107
pr: #50141
Cherry-pick the StructArray vector search fix to 2.6.
This fixes StructArray vector searches where StructArray parent fields
create gaps in user field IDs, while query/sealed-segment bitsets were
sized by schema field count.
Validation:
- uvx ruff==0.15.11 format --check --diff
tests/python_client/milvus_client/test_milvus_client_struct_array.py
- python3 -m py_compile
tests/python_client/milvus_client/test_milvus_client_struct_array.py
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
## Summary
- Cherrypick nullable vector lazy offset mapping fixes from #50085 to
2.6.
- Keep growing nullable-vector offset mapping stable during valid-data
updates and use lock-free sealed offset mapping.
- Batch nullable-vector bitset/offset conversion and skip conversion for
fast-path/non-nullable cases.
- Store nullable fixed-width vector Arrow schema fields as binary with
dimension metadata.
pr: #50085
issue: #50122
Note: source master PR #50085 is currently open, so the release gate may
wait for it to merge.
## Test plan
- [x] `git diff --check upstream/2.6..HEAD`
- [x] `make cppcheck`
- [x] `env CONAN_CMD=/tmp/conan1 make build-cpp-with-unittest`
- [x] `env
LD_LIBRARY_PATH=cmake_build/src:cmake_build/lib:internal/core/output/lib
cmake_build/bin/all_tests
--gtest_filter=OffsetMapping.*:SchemaTest.NullableFixedWidthVectorUsesBinaryInArrowSchemas:ChunkedColumnInterfaceTest*.*:ChunkedColumnTest.*:ChunkedColumnGroupTest.*:SearchOnSealedColumnBitsetLifetime.*:SearchOnSealedIndexNullableNoFilter.*:SearchOnSealedIndexNullableIteratorNoFilter.*`
Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
## Summary
- Backport #49933 to the 2.6 branch.
- Bump `github.com/apache/pulsar-client-go` to v0.19.0.
- Replace `streamnative/pulsarctl` admin usage with `pulsar-client-go`'s
`pulsaradmin`.
issue: #49932
pr: #49933
## Test plan
- [ ] Not run per maintainer request; rely on CI.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
pr: #50146
## Summary
Copy the 2.6 test coverage for the struct array search features that
were backported:
- element-level search on struct array vector fields
- hybrid search combining element-level search and a normal vector field
- collapse correctness coverage adapted from the master PR
This PR intentionally does not add `element_filter` or `MATCH` cases
because those features are not included in the 2.6 backport scope.
## Verification
Tested against latest 2.6 daily build:
- Milvus image: `harbor.milvus.io/manta/milvus:2.6-20260528-beb5662`
- Server version: `2.6-20260528-beb56622c9`
Commands run:
```bash
python -m py_compile tests/python_client/milvus_client/test_milvus_client_struct_array_element_search.py
pytest -c pytest.ini milvus_client/test_milvus_client_struct_array_element_search.py --host <host> --port 19530 --token <token> --tb=short
```
Result:
```text
7 passed
```
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Cherry-pick from master PR #49986
pr: #49986
issue: #49985
## Summary
Cherry-picked from master PR #49986 (merged).
When a DataNode running an import task fails or returns `Retry`,
DataCoord
resets the task to `Pending` and reschedules it. Preallocated segment
IDs
are reused on retry, but `PickSegment` on the DataNode picks randomly,
so
the retried attempt can write to a different subset of preallocated
segments than the failed attempt. `NumOfRows` updates were
monotonic-only
and never cleared across attempts, leaving orphan segments with stale
`NumOfRows` but no persisted binlogs. Sort compaction later failed with
`unexpected row count` or EOF.
## Fix
- New operator `ResetImportingSegmentRows(segmentIDs ...int64)` in
`internal/datacoord/meta.go` — clears `NumOfRows` and `MaxRowNum` only
on
segments still in `Importing` state.
- In `(*importTask).QueryTaskOnWorker`, call it on every preallocated
segment **before** transitioning the task back to `Pending`.
- Orphan segments now have `NumOfRows = 0` after retry, so
`createSortCompactionTask`'s zero-rows guard drops them.
## Verification
- [x] File count matches original PR (3 files, +107/-2)
- [x] Code changes verified (identical line-by-line diff vs master PR)
- [x] No conflict markers
- [ ] make static-check (skipped by cherry-pick workflow)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
issue: #48185
pr: #50086
- chaos tests: add pytest options to override search and query timeout
values used by the chaos checker
Signed-off-by: chyezh <chyezh@outlook.com>
## Summary
- Cherrypick nullable vector compact-storage invariant fixes from #49924
to 2.6.
- Enforce the compact nullable vector contract: `ValidData` tracks
logical rows, while vector payloads contain only valid physical rows.
- Cover the affected storage, import, proxy, index build, and segcore
paths with regression tests.
pr: #49924
issue: #49881
## Test plan
- [x] `git diff --check HEAD`
- [x] `cd pkg && go test ./util/funcutil ./util/typeutil ./mq/msgstream`
targeted nullable compact vector tests
Local full build/static checks were not completed in this workspace:
- `make build-cpp` was blocked by generated v3 proto imports and an
existing CMake generator-cache mismatch.
- `make static-check` was blocked by stale/generated canalyzer C header
mismatch.
Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
- **Bug 1 (MixCoord panic):** `compactionInspector.analyzeScheduler` was
never initialized in production code. When FloatVector is used as
ClusteringKey, the `doAnalyze()` path calls `analyzeScheduler.Enqueue()`
on a nil pointer → panic. Fixed by passing `analyzeScheduler` to
`newCompactionInspector`.
- **Bug 2 (DataNode round-robin panic):** `validateClusteringKey()` had
no field type whitelist, so JSON/Bool/Array passed schema validation.
During clustering compaction, `NewScalarFieldValue()` panics on
unsupported types. Fixed by adding `IsClusteringKeyType()` check to
reject unsupported types at collection creation time.
- [x] `TestIsClusteringKeyType` — verifies supported/unsupported type
classification
- [x] `TestClusteringKey` — new sub-tests for JSON, Bool, Array as
ClusteringKey (all rejected)
- [ ] Existing `TestClusteringKey` sub-tests (normal, multiple keys,
vector key) still pass
- [ ] `TestCompactionPlanHandler*` tests pass with updated
`newCompactionInspector` signature
issue: #47540
master pr: #48184#48529
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
pr: #50011
related issue:
- fix: harden writer lifecycle transitions to reject late writes during
finalize/close/fence/storage-failure states
- enhance: support Azure managed identity authentication for the storage
client
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
fixes#50031
Record the base data/index loading span before starting text index
loading so loadTextIndexesSpan no longer includes earlier segment load
work. This makes QueryNode segment load latency logs more accurate for
diagnosis.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry-pick from master
pr: #49982
Related to #49981
The REST timeout middleware could return a timeout response while the
handler goroutine continued using the original Gin context. Once Gin
recycled that context, late handler writes could race with a later
request and trigger concurrent map writes.
Run timeout-wrapped handlers with a copied Gin context and a fully
buffered response recorder instead. The original context and real
response writer now remain owned by the middleware goroutine, while
normal completion explicitly propagates selected metadata and commits
the recorder. Timeout closes the recorder and writes the 408 response
through the real writer, so late handler writes are discarded safely.
Add tests for recorder isolation, normal buffered commits, metadata and
abort propagation, late writes after timeout, and race validation of the
late-write timeout path.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
pr: #48347
Related master PRs: #48347, #48345, #48584, #48786
Related issue: #49992
## What this PR does
Backports the master-side Python client test fixes needed by the 2.6
Nightly failures:
- fixes dynamic-field spelling and output-field check setup in
search-by-PK/search-v2 cases
- aligns wildcard output-field metric checks with master
- updates TTL default coverage to use generated schema rows and
consistency-level checks from master
- updates output-field value checking for sparse vectors and byte-backed
vector fields
## Verification
- `python -m py_compile` on the four touched Python files: passed
- Targeted pytest against the 2.6 Nightly environment with `--host
10.104.21.115`: 9 passed, 1 failed
Known remaining failure:
- `test_search_by_pk_nullable_vector_field` still fails with
`MilvusException: vector data is not
schemapb.VectorField_SparseFloatVector`; tracked by #49992.
Signed-off-by: nico <cheng.yuan@zilliz.com>
pr: #49649
Cherry-pick PR #49649 to 2.6.
Changes:
- Add nullable vector/search pagination coverage in 2.6 client v2 tests.
- Add nullable full-text search function input/output coverage.
- Add nullable struct array embedding-list boundary coverage.
Notes:
- Skipped the master sync merge commit from the source PR.
- The group-by nullable vector coverage already exists in the 2.6
milvus_client_v2 test file, so the deleted old-path conflict was
resolved by keeping 2.6 layout.
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
pr: #49689,
#49770
issue: #49241,
#49746
## Summary
Cherry-pick partial update array op test suite from master to 2.6:
- #49689: Core test coverage for ARRAY_APPEND/ARRAY_REMOVE across all
element types, error/validation scenarios, and boundary cases
- #49770: Sealed segment path tests with flush + release + load +
array_contains predicate verification
## Test plan
- [x] All 39 cases verified on master CI (e2e-default passed)
- [x] All 39 cases verified on 2.6 nightly (`2.6-20260514-bc760565`)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cherry-pick from master
pr: #49910
Related to #49909
Enable h2c for the Gin REST server and advertise HTTP/2 through ALPN on
TLS REST listeners. Route shared-port HTTP/2 traffic through the REST
HTTP server so non-gRPC REST requests and gRPC requests can coexist on
the same plaintext port.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
## What's changed
- Upgrade the 2.6 test Python runtime to Python 3.12 via
tests/.python-version, Dockerfiles, and CI shell env activation.
- Update Python client and REST v2 test dependencies for Python 3.12,
including numpy 2.x compatibility.
- Adapt test data generation and assertions for numpy 2.x API/overflow
behavior.
- Remove unused GitHub workflows and stale README links following the
master cleanup.
## Related master PRs
pr: #49378
pr: #49571
## Verification
- git diff --check
- bash -n tests/scripts/python-env.sh tests/scripts/ci-util.sh
tests/scripts/ci-util-4am.sh tests/scripts/ci_e2e.sh
tests/scripts/ci_e2e_4am.sh tests/scripts/ci_e2e_4am_without_restful.sh
tests/scripts/ci_e2e_with_restful.sh
- Parsed remaining .github/workflows/*.{yaml,yml}
- Python 3.12 pip check for python_client and restful_client_v2
environments
- py_compile for changed Python test files
- pytest --collect-only for representative python_client and
restful_client_v2 suites
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
pr: #49349
This cherry-picks the assert_statistic update from #49349 to 2.6. It
removes the strict total > 2 requirement for expected-success checkers,
so low-frequency chaos checkers can pass when they complete at least one
successful operation. total=0 remains a failure with a clearer
diagnostic.
Verification:
- python3 -m py_compile tests/python_client/chaos/chaos_commons.py
- uvx ruff@0.15.11 check tests/python_client/chaos/chaos_commons.py
- uvx ruff@0.15.11 format --check --diff
tests/python_client/chaos/chaos_commons.py
- inline dummy checker validation for total=1 success and total=0
failure
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
issue: #49959
pr: #49961
- streaming flusher: only observe truncate collection before flusher
handling, keeping drop collection on the deferred observe path
- streaming flusher tests: cover drop collection cleanup ordering and
truncate collection early observe behavior
Signed-off-by: chyezh <chyezh@outlook.com>
Cherry-pick from master
pr: #49587
## Summary
This PR cherry-picks QueryCoord ChannelDistManager filtering
optimizations to the 2.6 branch.
The change removes the global ChannelDistManager collection index and
reuses each node's collection-scoped channel index for collection
lookups. It also reduces temporary allocations in ChannelDist and
SegmentDist filtering paths.
## Code Changes
- Remove the global collectionIndex from ChannelDistManager.
- Use the unified GetByFilter path with WithCollectionID2Channel.
- Reuse per-node nodeChannels.collChannels for collection-scoped lookup.
- Reduce temporary allocations in dist manager filtering paths.
- Update QueryCoord callers and tests for the unified ChannelDist filter
path.
issue: #49511
## Test Plan
- git diff --check
- gofumpt -l $(git diff --name-only upstream/2.6..HEAD -- '*.go')
- CMAKE_POLICY_VERSION_MINIMUM=3.5 CONAN_CMD=conan-1 SKIP_3RDPARTY=1
make
- zsh -c 'source scripts/setenv.sh && go test -v -count=1 -tags
dynamic,test -gcflags="all=-N -l" -ldflags="-r
${MILVUS_WORK_DIR}/cmake_build/lib -r
${MILVUS_WORK_DIR}/internal/core/output/lib" ./internal/querycoordv2/...
-timeout 600s'
---------
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
Cherry-pick from master
pr: #48105
issue: #49934
## Summary
Backport the chaos checker fix from #48105 to the 2.6 branch.
This addresses the checker-side failures observed during the 2.6 chaos
run analysis:
- Hybrid search used one cached `self.dim` for all vector fields. After
`AddVectorFieldChecker` mutates the schema, a collection can contain
vector fields with different dimensions, so hybrid search must generate
vectors using each field's actual schema dimension.
- `AddVectorFieldChecker` hardcoded new FLOAT_VECTOR fields as `dim=32`,
which can diverge from the base collection schema.
- The previous vector-field-count precheck is unreliable under
concurrent schema mutation. This backport follows master and falls back
to insert-only when the server returns `maximum vector field`.
## Scope
This PR fixes only the Python chaos checker issue. The server crash
tracked in #49934 is a separate Milvus 2.6 server bug and should be
fixed by backporting #47151.
## Verification
- `python3 -m py_compile tests/python_client/chaos/checker.py`
- `git diff --check`
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
## What type of PR is this?
/kind bug
## What this PR does / why we need it:
Backport #47151 to 2.6.
Fix dangling reference in `IndexingRecord` after
`SegmentGrowingImpl::Reopen()` updates the schema object.
issue: #46656
pr: #47151
## Which issue(s) this PR fixes:
issue: #46656
## Special notes for your reviewer:
Cherry-picked from commit 882d7be8fe.
## Test report
- `git diff --check` passed for the backport changes.
- Checked changed files for unresolved conflict markers: none found.
- `make run-test-cpp filter='SchemaReopenTest.*'` could not complete
locally because `libmilvus-planparser.so` was not loadable from the
existing local C++ test binary.
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
## Summary
- Reject absent non-nullable vector fields during sealed segment reopen
instead of silently installing an unsupported schema.
- Keep non-nullable vectors out of default-fill during finish load,
while asserting that field data or a vector index carrier is present.
- Add focused segcore coverage for reopen rejection, indexed
finish-load, and missing-carrier finish-load failure.
issue: #49896
issue: #49470
## Test plan
- [x] `git diff --check`
- [x] `make -C cmake_build all_tests`
- [x]
`LD_LIBRARY_PATH="$PWD/cmake_build/lib:$PWD/internal/core/output/lib"
./cmake_build/bin/all_tests
--gtest_filter='test_chunk_segment.ReopenRejectsAbsentNonNullableVectorField:test_chunk_segment.FinishLoadSkipsNonNullableVectorWithIndex:test_chunk_segment.FinishLoadRejectsMissingNonNullableVectorCarrier'`
Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
