issue: #46308
## Summary
Normalize analyzer names before running multi-analyzer highlight and
analyzer requests: omitted names use default, one name is broadcast, and
per-text names must match the text count. Also add default analyzer
names for lexical highlight query texts and request the analyzer-name
field for multi-analyzer query-only highlight.
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
issue: #45881
## Summary
Add the `milvus-table` external format so external collection refresh
can consume Milvus snapshot metadata and StorageV3 segment manifests
directly.
This change lets refresh resolve source manifest row counts, rebuild
target StorageV3 manifests from source column groups, copy source
deltalogs with deterministic preallocated IDs, and keep real external
primary-key semantics for milvus-table segments.
## Why
Snapshot-backed external tables are not generic file tables. The refresh
path needs to preserve Milvus StorageV3 manifest structure, deltalogs,
and real PK behavior instead of falling back to virtual PK semantics or
format-agnostic fragment handling.
## Details
- Add milvus-table external spec parsing and snapshot manifest
resolution.
- Add StorageV3 manifest translation for source Milvus segment
manifests.
- Copy milvus-table deltalogs into target manifests with deterministic
LogIDs.
- Wire QueryNode real-PK lookup/load handling for milvus-table segments.
- Add unit and Go client coverage for spec parsing, manifest resolution,
refresh, and deltalog handling.
## Validation
- `git diff --cached --check`
- No proto changes detected
- No mockery patterns in changed test files
- `source ~/.profile && source scripts/setenv.sh && make milvus`
- Reset Milvus local standalone before Go tests
- `go test -tags dynamic,test -gcflags="all=-N -l" -ldflags="-r
${RPATH}" github.com/milvus-io/milvus/internal/datanode/external
github.com/milvus-io/milvus/internal/storagev2/packed -count=1 -timeout
300s`
- `source ~/.profile && source scripts/setenv.sh && make lint-fix`
Related: [#45881](https://github.com/milvus-io/milvus/issues/45881)
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
## Issue
Fixes#50678
## Problem
When `common.security.authorizationEnabled=true`, the Proxy's
`PrivilegeInterceptor` resolved the database for privilege checks solely
from the gRPC connection-context db header (set via
`useDatabase`/`using_database`, defaulting to `default`), via
`GetCurDBNameFromContextOrDefault(ctx)`. It **ignored the `DbName`
carried in the request body**.
Downstream handlers, however, operate on the request's `DbName` (after
`DatabaseInterceptor` normalizes it — request body takes precedence,
then context header, then default). So when a caller does not call
`useDatabase` and instead targets a database directly via the request's
`db_name`, the auth check and the actual operation run against
**different** databases. This produces two classes of problems for every
db-scoped privilege check (`CreateCollection`, `DropCollection`,
`CreateIndex`, `Insert`, `Search`/`Query`, `AlterDatabase`, …):
1. **False denial (functional bug):** a user holding a grant on
`db_target`, whose connection stays on `default` and who targets
`db_target` only through the request's `db_name`, is incorrectly checked
against `default` and receives `PermissionDenied`.
2. **Cross-db privilege escalation (security risk):** a user with a
grant on `default` can set the request's `db_name` to a different
database they are not authorized for; the check passes against `default`
while the operation actually lands on the target database.
## Fix
Resolve the db the request actually operates on: prefer the request-body
`DbName`, falling back to the connection-context db and finally the
cluster default. Privilege checks now use this resolution so
authorization runs against the same database as the operation.
- `internal/proxy/util.go`: add `GetCurDBNameFromRequestOrContext(ctx,
req)` (+ `dbNameGetter` interface). Requests without a top-level
`DbName` field (e.g. grant-management requests carrying `Entity.DbName`)
fall through to the previous context-based behavior, so their semantics
are unchanged.
- `internal/proxy/privilege_interceptor.go`: resolve `dbName` via the
new helper instead of `GetCurDBNameFromContextOrDefault(ctx)`.
## Tests
- `TestPrivilegeInterceptor`: a few assertions had encoded the old
"context decides the db" behavior (e.g. a user granted Global-All only
on `default` was expected to be *allowed* when sending `db_name=db_test`
— which is exactly the escalation bug). Realigned the positive-path
request db with the granted db so the assertions are self-consistent
under correct semantics.
- Added `TestPrivilegeInterceptorRequestDBName` covering the two issue
scenarios plus the fallback:
- granted on `db_target`, no `useDatabase`, request `db_name=db_target`
→ **allowed** (false-denial fixed);
- granted only on `default`, request `db_name=db_escalate` → **denied**
(escalation blocked);
- request omits `db_name` → falls back to the connection-context db
(unchanged behavior).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: https://github.com/milvus-io/milvus/issues/49879
## Summary
- Keep `limit + offset` candidates during reduce for common search with
`order_by_fields`.
- Apply final offset/limit after scalar order_by sorting in the search
pipeline.
- Add targeted proxy unit tests for row-aligned reorder, group
pagination, string IDs, and vector slicing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
- Replace `common.entryStream.streamBudgetRatio` with refreshable
`common.loadTransientBudgetBytes`, a process-wide transient load budget
shared by scalar index V3 entry streaming and storage v2/v3 field-data
loading. `0` keeps the limit disabled.
- Gate field-data batch reads on the shared budget, split batches by
loading overhead, scale read parallelism from the effective batch
budget, and finalize `GroupChunk` cells before pushing them so Arrow
tables are released promptly.
- Align MCL loading-overhead reservations for storage v1 scalar indexes
and storage v2/v3 field data under one load-transient overhead group,
with array-field overhead and mmap file usage accounted separately.
- Propagate load cancellation into budget waits, field-data batch tasks,
and V3 scalar `IndexEntryReader` stream/file reads so canceled loads do
not stay blocked behind transient memory pressure.
- Update segcore init/config watchers and add C++/Go coverage for budget
sizing, cancellation, field-data batching/finalization, V3 entry
streaming, and the new config.
issue: #49499
---------
Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
## What type of PR is this?
/test
## What this PR does / why we need it
Adds Python client coverage for struct array feature scenarios on
master:
- Covers struct-array vector subfields across JSON, JSONL, CSV, and
Parquet import files.
- Covers LocalBulkWriter generated JSON, JSONL, CSV, and Parquet files
for FLOAT_VECTOR, FLOAT16_VECTOR, BFLOAT16_VECTOR, INT8_VECTOR, and
BINARY_VECTOR struct subfields.
- Aligns BulkWriter test inputs with ordinary vector field input forms,
including FP16/BF16 ndarray inputs and INT8 ndarray inputs.
- Adds hybrid search, range, group-by, element_scope regression
coverage, and bitmap index coverage for struct array element fields.
- Updates Python client test dependency pin to pymilvus 3.1.0rc50 so the
merged BulkWriter JSONL and struct vector fixes are exercised.
## Which issue(s) this PR fixes
None.
## Special notes for your reviewer
The BulkWriter coverage intentionally uses native insert-format inputs
for non-float32 vectors to match ordinary vector field behavior.
## Tests
```bash
python -W ignore -m pytest -v -n 6 --tb=short \
milvus_client/test_milvus_client_struct_array.py::TestMilvusClientStructArrayImport::test_import_struct_array_vector_subfield_with_local_bulk_writer \
--host 10.100.36.207 --port 19530 --token root:Milvus \
--minio_host 10.100.36.206 --minio_bucket struct-array-master-ff3dbeb
```
Result: `20 passed in 120.02s`
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
issue: #50483
## Summary
- Record a stable reason when RollbackImport marks a 2PC import job as
Failed.
- Treat repeated AbortImport calls on user-aborted jobs as successful
no-ops.
- Keep AbortImport rejection for real failed, committing, or completed
import jobs.
## Test Plan
- [x] go test -tags dynamic,test -gcflags=\"all=-N -l\" -count=1
./internal/datacoord -run
'Test(CommitImport|AbortImport|RollbackImportCallback|CommitImportCallback)'
- [x] make static-check
---------
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
## Summary
Reject REST import create requests when `options.auto_commit` is present
but not `"true"` or `"false"`. The omitted option remains accepted and
keeps the default auto-commit behavior.
Fixes#50460
## Changes
- Add strict REST JSON validation for `options.auto_commit`.
- Add handler coverage for omitted, valid, null, empty, and invalid
values.
## Test Plan
- `GOTOOLCHAIN=auto make static-check`
- `GOTOOLCHAIN=auto go test ./util/merr` from `pkg/`
- `GOTOOLCHAIN=auto go test ./internal/json`
Note: local `static-check` used `milvus-cpp-share` to reuse the main
checkout C++ artifacts and a temporary ignored
`cmd/tools/migration/legacy/legacypb/legacy.pb.go` copied from the main
checkout, because this generated file is ignored but required by the
migration package during local lint loading.
---------
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
## Summary
- Add authorization for REST v2 import commit and abort endpoints.
- Resolve the import job collection via GetImportProgress before
checking PrivilegeImport.
- Cover denied non-privileged users, allowed root users, and
authorization-disabled behavior.
Fixes#50458
## Test Plan
- [x] make build-cpp-with-unittest
- [x] go test -tags dynamic,test -gcflags=\"all=-N -l\" -count=1
./internal/distributed/proxy/httpserver/... -run
'TestCommitImportJob|TestAbortImportJob'
- [x] go test -tags dynamic,test -gcflags=\"all=-N -l\" -count=1
./internal/distributed/proxy/httpserver/... -skip '^TestSearchV2$'
- [x] GOTOOLCHAIN=auto make static-check
## Notes
- Full httpserver package test currently has an unrelated existing
TestSearchV2/search#09 expectation mismatch: expected `Mismatch type
uint8`, actual contains `Mismatch type []uint8`.
---------
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
issue: #50750
Move sync task submission out of the writebuffer mutex. The writebuffer
still selects segments, yields buffered payload, records sync
checkpoints, and marks segment syncing while holding the lock, but
submits the already built tasks to syncmgr after releasing it.
This prevents ReleaseCollection from hanging in manual-flush preparation
when syncmgr dispatcher backpressure blocks SyncData submission, because
CheckReleaseManualFlushNeed can still acquire the writebuffer read lock.
Also move growing-source follow-up resync out of the callback critical
section and add a regression test that blocks SyncData submission while
verifying writebuffer readers are not blocked.
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
issue: #50749
Depends on #50527.
Route proxy PK/channel and partition-key hash helpers through the
routing table while preserving the existing hash modulo behavior.
Query/search are covered through assignPartitionKeys, which delegates to
internal/util/typeutil.HashKey2Partitions.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
issue: #50729
### What
Dropping a BM25 function field and re-adding it with the same function
name and output field left the new sparse index stuck `InProgress`. The
flow was blocked on two layers, fixed here:
- **DataCoord**: accept expanded pre-allocated segment ID ranges when
completing the schema-bump replacement compaction (previously only a
single-ID range was allowed, causing `BumpSchemaVersionCompaction failed
... compaction plan illegal`). Still require the replacement segment to
use the range begin, matching DataNode full-rewrite behavior.
- **QueryNode**: allow a BM25 function field to be dropped and re-added
with a new output field ID (reject only incompatible in-place changes to
a shared output field), and prune dropped BM25 stats from
current/growing/sealed state and local disk so reclaimed disk size stays
accurate.
### Tests
- Cover expanded-range completion in the schema-bump metadata regression
test.
- Cover drop/re-add validation and dropped-stats pruning
(current/growing/sealed + on-disk) in the delegator and idf_oracle
tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: #50452
Pulls in milvus-io/milvus-storage#567, which:
1. Lowers the default zstd writer compression level from 5 to 3.
At level 5 zstd switches from the dfast strategy to greedy with
row matchfinder; on random float embeddings that costs ~8% more
CPU for near-zero size benefit.
2. Emits FIXED_SIZE_BINARY and BINARY columns uncompressed with
statistics disabled. Vector payloads behave like incompressible
bytes; zstd spends CPU on them without shrinking the output.
Together these cut parquet write CPU on vector workloads without
hurting file size, addressing the storage v3 import slowdown
tracked in #50452.
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
## Issue
issue: #50102
## Problem
With etcd auth enabled and `mixCoord.enableActiveStandby: true`, a
long-running **standby** MixCoord panics when promoted to active:
```
panic: rpc error: code = Unauthenticated desc = etcdserver: invalid auth token
ProxyWatcher.startWatchEtcd (proxy_watcher.go:146)
```
This is **not** a credential misconfiguration (the same config works for
the active node and all other components, and a restart of the same pod
recovers).
Root cause:
- etcd's default `simple` auth token is stateful and server-local: GC'd
after idle TTL (`--auth-token-ttl`, default 300s), member-local, and
lost on member restart. A long-idle standby's token gets invalidated
server-side without the client knowing.
- When an **already-established watch stream** then hits
`Unauthenticated`, clientv3 classifies it as a halt error (`isHaltErr` —
anything that isn't `Unavailable`/`Internal`) and tears the stream down
**without** refreshing the token. clientv3's token refresh only covers
unary calls and a stream's first recv, not a live watch stream.
- `ProxyWatcher.startWatchEtcd` and `sessionutil.handleWatchErr` only
handled `ErrCompacted` and `panic()`ed on every other watch error.
## Fix
Treat auth-token watch errors as recoverable and re-establish the watch
with a bounded retry, instead of panicking. Re-watching issues a unary
request first (`getSessionsOnEtcd` / `GetSessions`), which refreshes the
auth token via clientv3's unary retry interceptor, so the new watch
stream recovers.
- New shared predicate `etcd.IsRetriableWatchErr` classifies recoverable
watch errors (`ErrCompacted` + auth-token errors: `ErrInvalidAuthToken`
/ `ErrUserEmpty` / `ErrAuthOldRevision` / raw `Unauthenticated`).
- `ProxyWatcher.startWatchEtcd` and `sessionutil.handleWatchErr` now
re-watch on recoverable errors via `retry.Do(...,
retry.RetryErr(etcd.IsRetriableWatchErr))`. The retry is bounded by
`retry.Do`'s default attempt count / backoff; non-recoverable errors
still fail fast.
The CDC controller and streaming session discoverer already self-heal
(return-and-re-list on watch error), so they are unchanged.
> Operational note for affected users: configuring etcd with **JWT**
auth tokens (`--auth-token jwt,...`) instead of the default `simple`
token avoids the unexpected server-side invalidation entirely (JWT is
stateless — valid across members, survives restart, no idle GC).
## Test
- Added `TestIsRetriableWatchErr` (pkg/util/etcd) — predicate
classification.
- Added subtests in `TestWatcherHandleWatchResp` (sessionutil) —
auth-token error triggers re-watch (not channel close), non-retriable
error closes the channel.
- Existing `TestProxyManager_ErrCompacted` still passes (non-retriable
bad-session-data fails fast).
> Note: the watcher integration tests link the C++ core; they were not
runnable in my local/remote env at PR time and will be validated in CI.
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This PR removes Code Checker MacOS from the Mergify ci-passed gate.
The macOS checker will no longer be required for ci-passed, and macOS
checker failures will no longer remove ci-passed. It can still run
separately or on demand.
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
Follow-up to #50515 — documentation only, no code change.
## Why
CLAUDE.md already ritualizes how an agent *enters* the code (the
subsystem reading procedure) and the Go/merr error-handling rules, but
has no symmetric ritual for *proving a change achieves its purpose*. In
practice this lets a behavioral change (error-code propagation, retry
classification, routing) reach a "compiles + unit tests pass +
success-path e2e is green" state while missing its goal — e.g. a
boundary classifier that is correct in isolation but fed a value the
upstream already destroyed (`throw
std::runtime_error(status.ToString())`) or mis-set (an IOError rewritten
as Invalid, inverting transient vs permanent). Success-path e2e never
exercises the failure modes such a change exists for, so "all green"
reads as done when it is not.
The error-handling rules are also Go/merr-centric, so the same
discipline was not stated for the C++ side (segcore / milvus-storage /
cgo boundary), which is where the final `ErrorCode` is actually decided.
## What
- New **Verification gate** section (symmetric in weight to the reading
procedure):
- **G1** verify the data, not just your transform — audit every upstream
construction/throw/rewrite site of the value, not only edited lines, and
grep the escape hatches.
- **G2** trace each real failure mode end-to-end (trace or fault-inject;
success-path e2e is not evidence).
- **G3** do not over-claim in commit / PR body.
- **G4** adversarial self-review before human review.
- **Error handling** section: a C++ rule (segcore / milvus-storage /
cgo) — the class is decided at the `ThrowInfo` / `AssertInfo` /
`SegcoreError` / arrow-`Status` / `LOON_*` construction sites, not the
cgo translator; a boundary helper is correct only if its inputs carry
the right category; apply the blame test at every such site.
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
relate: #49716
pr: https://github.com/milvus-io/milvus/pull/49717
## Summary
Split FunctionRunnerManager lifecycle references between WAL and
delegator keys while still sharing runners by function signature. Add
latest-version lookup support for schemaVersion 0 or nil schema in the
manager; master call sites still pass the current schema version
explicitly.
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This reverts PR #50711 (commit ce6cc762fd), which added a `cubefs` note
to `configs/milvus.yaml` documenting CubeFS as a supported cloud storage
provider.
Reverting per maintainer request.
Reverts #50711
issue: #26189
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
## What
Add CubeFS as a supported `cloudProvider` option in Milvus object
storage configuration.
## Changes
- Add `CloudProviderCubeFS = "cubefs"` constant to
`pkg/objectstorage/util.go`
- Add CubeFS case in `NewMinioClient` switch: forces path-style bucket
lookup (`BucketLookupPath`) and static V4 credentials (IAM not supported
by CubeFS ObjectNode)
- Document `cubefs` as a valid `cloudProvider` value in
`configs/milvus.yaml`
- Add integration test `TestNewMinioClientCubeFS` (skipped unless
`CLOUD_PROVIDER=cubefs` env is set, consistent with all other provider
tests)
## Why path-style?
CubeFS ObjectNode requires `S3ForcePathStyle` — it does not support
virtual-host style bucket addressing (`bucket.host/key`).
issue: #26189
Signed-off-by: Ayush KAshyap <kashyap11ayush02@gmail.com>
issue: #50145
## Summary
Make `mixCompactionTask` build text indexes inline for `enable_match`
fields (mirroring `sortCompactionTask`) so that mix-compaction output
segments arrive at QueryNode already carrying `TextStatsLogs`. This
eliminates the CGO_LOAD `CreateTextIndex` fallback path in segcore —
observed in production to redundantly rebuild text indexes during
segment load for tens of seconds up to 13+ minutes per segment.
## Background
For a freshly mix-compacted segment with text-match fields, the load
pipeline currently races:
1. Vector index task completes within ~10s (event-triggered)
2. QueryCoord treats the segment as loadable once the vector index is
ready
3. The `TextIndexJob` (stats task) is event-less — only the 60s polling
ticker submits it
4. The job itself takes ~2 minutes for typical text-heavy segments
5. QueryNode receives the segment before its persistent text index is
written; segcore detects `text_stats_logs == nil` and calls
`CreateTextIndex` in-place at load time (`[CGO_LOAD]` scope)
`sortCompactionTask` already avoids this by building the text index
inline as part of compaction
(`internal/datanode/compactor/sort_compaction.go:469-503`).
`mixCompactionTask` did not — this PR closes the gap.
## What changes
- **`internal/datanode/compactor/mix_compactor.go`** — Add `cm
storage.ChunkManager` field; constructor takes one extra arg; add a thin
`createTextIndex` method wrapper that delegates to the existing
package-level `createTextIndex` helper (in `compactor_common.go`,
already used by sort compaction); after `applyLOBCompaction`, loop over
each non-empty **sorted** output segment, build text indexes, update the
V3 manifest, and assign `TextStatsLogs`. Emits a `create_text_index`
stage latency metric. (No new shared helper is introduced — this PR
reuses `compactor_common.go:createTextIndex` and `sort_compaction.go` is
untouched.)
- **`internal/datanode/compactor/namespace_compactor.go`** — Forward
`cm` to the inner `mixCompactionTask`.
- **`internal/datanode/services.go`** — Updated call sites (pass `cm`).
- **`internal/datacoord/compaction_task_mix.go`** — Extend the
`FileResources` plumbing condition to include `MixCompaction` so custom
analyzers (ref mode) reach the inline build. Previously only
`SortCompaction` got `FileResources`, which would have caused
mix-compacted text indexes to use default tokenization → silent search
regressions.
- **`internal/datacoord/compaction_task_clustering.go` /
`compaction_inspector.go`** — Namespace-enabled `ClusteringCompaction`
is routed on the DataNode to `NewNamespaceCompactor`, which delegates to
`mixCompactionTask.Compact()` and now builds the text index inline for
sorted-by-namespace outputs. Wire `IndexEngineVersionManager` into
`clusteringCompactionTask` and, in `BuildCompactionRequest()`, set
`CurrentScalarIndexVersion` (otherwise the inline text index metadata is
emitted with version 0) and fetch `FileResources` for namespace-enabled
clustering plans in ref mode (otherwise the custom analyzer resources
are not downloaded and the text index falls back to the default
analyzer). Addresses review feedback from @aoiasd.
- **`internal/datanode/compactor/mix_compactor_test.go` /
`namespace_compactor_test.go`** — Updated existing constructor call
sites to pass the new `cm` arg.
- **`internal/datanode/compactor/mix_compactor_text_index_test.go`
(new)** — Mockey-based unit tests covering: the wrapper's identifier
propagation to the package helper, error propagation, and real
`Compact()` inline-loop semantics (empty + unsorted segments skipped,
`TextStatsLogs` assigned for sorted, V3 manifest rewritten,
build/manifest errors abort).
- **`internal/datacoord/compaction_task_mix_test.go`** — Test that
`MixCompaction` plans receive `FileResources` in ref mode.
## Test plan
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1 -run
"TestMixCompaction|TestNamespace|TestSort"
./internal/datanode/compactor/...` — PASS (post-rebase)
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1 -run
"TestMixCompaction|TestCompactionTask" ./internal/datacoord/...` — PASS
- [x] New unit tests pass:
`TestMixCompaction_createTextIndex_Delegates`,
`TestMixCompaction_createTextIndex_PropagatesError`,
`TestMixCompaction_Compact_InlineTextIndex`,
`TestMixCompaction_Compact_InlineTextIndex_BuildError`,
`TestMixCompaction_Compact_InlineTextIndex_ManifestError`, and
(datacoord)
`TestMixCompactionTaskSuite/TestBuildCompactionRequest_MixFileResourcesInRefMode`,
`TestClusteringCompactionTaskSuite/TestBuildCompactionRequest_NamespaceFileResourcesInRefMode`
- [ ] CI builds against fresh C++ artifacts (local worktree build was
skipped because the C++ shared libs in this workspace are stale relative
to upstream's recent cgo changes — unrelated to this PR's diff)
- [ ] Manual / integration: verify a freshly mix-compacted segment with
text-match fields lands on QueryNode with non-empty `text_stats_logs`
and does NOT trigger the `[CGO_LOAD] CreateTextIndex` slow path in
QueryNode logs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: https://github.com/milvus-io/milvus/issues/42148
1. Support range search in hybrid search
Enable element-level vector fields to participate in hybrid range search
while preserving element identity and offsets.
2. Support group-by in hybrid search, with constraints
Enable element-level hybrid search with PK group-by and preserve element
offsets. For group-by, sub-searches that need element identity merging
must stay within the same struct array semantic scope.
Also, it fixes 2 related bugs:
1. Fix group-by related issues
Fix lost element indices and incorrect merging behavior in query/search
group-by paths, so multiple matching elements under the same PK can
still be returned with correct offsets.
2. Fix aggregation output issues
Fix aggregation/count output corruption when a collection contains
struct array fields. Proxy struct-field reconstruction now preserves
virtual aggregation fields such as `count(*)` instead of treating
`field_id=0` as a struct sub-field. As a result, `id + count(*)`,
element-level `count(*)`, and group-by `count(*)` return correctly named
fields.
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
## Description
The proxy access log recorded the collection field as `Unknown` for any
request that does not expose a singular `GetCollectionName() string`.
This made it impossible to attribute traffic to a specific collection
from access logs — most notably when diagnosing per-collection flush
rate-limit rejections
(`milvus_proxy_rate_limit_req_count{msg_type="DDLFlush",
status="fail"}`, default limit
`quotaAndLimits.flushRate.collection.max=0.1`).
Affected proxy-facing requests:
| Request | Collection field | Before | After |
|---------|------------------|--------|-------|
| `Flush` | `collection_names []string` | `Unknown` | `[coll_a coll_b]`
|
| `ShowCollections` | `collection_names []string` | `Unknown` | `[coll_a
coll_b]` |
| `RenameCollection` | `OldName` / `NewName` | `Unknown` | `old_coll` |
| `BatchDescribeCollection` | `collection_name []string` | `Unknown` |
`[coll_a coll_b]` |
## Root cause
`GrpcAccessInfo.CollectionName()` / `RestfulInfo.CollectionName()` only
used `requestutil.GetCollectionNameFromRequest`, which relies on the
singular `CollectionNameGetter` (`GetCollectionName() string`).
`FlushRequest`/`ShowCollectionsRequest` only have the repeated
`GetCollectionNames() []string`, so the assertion failed and the field
fell back to `Unknown`. `RenameCollectionRequest` and
`BatchDescribeCollectionRequest` carry the collection under non-standard
fields.
## Fix
- Add `CollectionNamesGetter` + `GetCollectionNamesFromRequest` to
`requestutil` (mirrors the existing `PartitionNamesGetter`).
- In `CollectionName()`, fall back to the plural getter, then to the
request-specific fields, mirroring the existing `PartitionName()` plural
handling.
- Unit test covering nil / singular / `Flush` / `RenameCollection` /
`BatchDescribe`.
Internal coordinator RPCs (`DescribeSegment`, `ShowSegments`) only carry
`CollectionID` and do not flow through the proxy access log, so they are
out of scope.
Fixes#50741🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
issue: #50698Fixes#50698
## What this PR does
This PR fixes NULL/UNKNOWN validity handling in boolean filter execution
paths:
- Makes conjunctive filter short-circuit decisions preserve rows that
are still UNKNOWN and may be resolved by later predicates.
- Treats UNKNOWN predicate results as filtered out when converting
predicate results to filter bitsets.
- Preserves nullable parent JSON validity for `!=` instead of producing
`data=true, valid=false` that can leak through raw-bit consumers.
- Fixes chunked sealed variable-length offset filtering so skipped NULL
rows keep invalid validity instead of becoming definite false.
## Verification
- `ninja unittest/all_tests` linked successfully.
- Focused C++ test run passed 109/109:
- `ConjunctExprTest.*`
- `*TestUnaryRangeWithJSONNullable*`
- `*TestUnaryRangeJsonNullable*`
- `git diff --check` passed.
- Staged diff check passed.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
Add missing fileNum increments in garbage collector walkers so
walkFileNum metrics/logs are accurate for text and JSON cleanup paths.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #46308
## Summary
Keep lexical highlight corpus texts aligned with search result rows when
highlighted string fields are nullable or empty. Null and empty rows now
still produce highlight data with an empty fragments list.
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Allow mixed normal/import compaction results to use output binlog
timestamp ranges even when the fallback start position is earlier than
an import commit timestamp.
- Normalize fallback-only start/dml positions to the max input commit
timestamp when output timestamps are unavailable.
- Add regression coverage for manual mix compaction with normal rows
preceding committed import rows.
Fixes#50464
## Test Plan
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/datacoord -run '^TestMeta$' -ldflags="-r ${RPATH}"`
- [x] `make static-check`
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
## Summary
issue: #50579
Cache estimated index load resources on `LoadIndexInfo` so segment load
paths can reuse the same estimate for raw-data checks and resource
accounting.
## Changes
- Store the cached load resource estimate as
`std::optional<LoadResourceRequest>`.
- Reuse cached load-resource estimates in sealed index loading and V1
translator paths.
- Keep `CheckIndexHasRawData` inline with a direct `IndexFactory`
fallback instead of a separate helper.
- Add unit coverage for optional assignment, copy, and reset behavior.
## Test Plan
- [x] `/opt/homebrew/opt/llvm@18/bin/clang-format --dry-run --Werror
internal/core/src/segcore/SegmentLoadInfo.cpp
internal/core/src/segcore/SegmentLoadInfo.h`
- [x] `git diff --check upstream/master..HEAD`
- [x] `rg "GetIndexLoadResource|has_load_resource_request"
internal/core/src/segcore internal/core/unittest/test_loading.cpp`
- [x] `make run-test-cpp
filter=IndexLoadTest.LoadResourceRequestCacheIsOptional`
- [x] `make SKIP_3RDPARTY=1 build-cpp-with-unittest` passed before the
final inline cleanup; after the cleanup, the same target compiled
`SegmentLoadInfo.cpp` successfully and was then stopped before
completion.
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
## Summary
Fix QueryNode schema freshness checks by separating two ordering
domains:
- `CollectionSchema.version` is the logical schema freshness version.
QueryNode uses it to prevent old schema payloads from overwriting newer
fields/functions.
- `schema_barrier_ts` is the DDL/update barrier timestamp. QueryNode
uses it to fence stale load results and to refresh same-version schema
payloads, such as collection properties (`ttl_field`) that do not bump
`schema.version`.
Fixes#50364
## Update rules
When QueryNode receives a schema payload:
- If incoming `schema.version < current schema.version`, skip the update
even if `schema_barrier_ts` is larger. This prevents schema rollback
from out-of-order replay/channel delivery.
- If incoming `schema.version > current schema.version`, apply the
update.
- If incoming `schema.version == current schema.version`, apply the
update only when `schema_barrier_ts` is newer than the current barrier.
This allows properties-only schema snapshots, for example TTL field
changes, to refresh the runtime schema.
- If the schema payload is absent, fall back to `schema_barrier_ts` for
legacy/rolling-upgrade compatibility.
Segcore still receives a single schema update token, but QueryNode now
derives that token with collection-local monotonic ordering across both
logical schema versions and barrier timestamps. This keeps segcore
accepting updates when a high-barrier same-version refresh is followed
by a higher logical schema version that carries a smaller barrier
timestamp.
## Delegator side effects
`shardDelegator.UpdateSchema` now runs the same stale/no-op schema check
before updating workers, load barriers, function runtime state, IDF
oracle state, or local collection metadata. This keeps stale schema
messages from producing delegator side effects when the collection
manager would skip the schema payload.
The delegator load barrier remains timestamp-based and monotonic. A
newer logical schema version with a smaller barrier timestamp must not
reopen older load results after a previous same-version property refresh
advanced the barrier.
## Changes
- Rename the legacy QueryCoord fields to `schema_barrier_ts` to reflect
their timestamp-barrier semantics.
- Track both logical schema version and schema barrier timestamp in
QueryNode collection schema snapshots.
- Use `CollectionSchema.version` as the logical QueryNode collection
schema freshness version for load, sync, pipeline replay, and
UpdateSchema paths.
- Allow same-version schema payload refresh only when the barrier
timestamp advances, so properties-only changes such as `ttl_field` take
effect without release/load.
- Generate a monotonic segcore schema update token from the current and
incoming logical/barrier values.
- Skip stale/no-op schema payloads in delegator before worker updates
and other schema side effects.
- Keep `schema_barrier_ts` in delegator/pipeline load fencing so stale
load results are rejected after schema-changing DDL.
- Update logs, comments, and tests to distinguish `schemaVersion` from
`schemaBarrierTs`.
## Test Plan
- [x] `make generated-proto-without-cpp`
- [x] `git diff --check upstream/master...HEAD`
- [x] `cd pkg && go test -count=1 ./proto/querypb`
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/querycoordv2/task -run
'TestUtils/TestPackLoadMetaSchemaVersions'`
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/querynodev2/segments -run
'TestCollectionManager/(TestUpdateSchema|TestPutOrRefKeepsFreshCollectionInSchemaVersionDomain|TestLoadMetaSchemaVersionCompatibility)'`
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/querynodev2/delegator -run
'TestDelegatorSuite/TestUpdateSchema|TestUpdateSchema'`
- [x] `env 'etcd.auth.enabled=false' go test -tags dynamic,test
-gcflags="all=-N -l" -count=1 ./internal/querynodev2 -run
'TestQueryNodeService/TestUpdateSchema'`
- [x] `make static-check` with Go 1.26.4 and shared C++ build
environment before the final upstream rebase
Notes:
- Full `./internal/querynodev2/delegator` was also attempted locally. It
currently fails in unrelated
`TestDelegatorDataSuite/TestLoadPartitionStats` because partition stats
deserialization reports `json: cannot unmarshal into Go value of type
storage.VectorFieldValue`; the schema/update-focused tests above pass.
- Per maintainer request, the final post-rebase `make static-check` run
was skipped before pushing the latest commit.
---------
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
## Summary
This PR fixes a DataCoord GC race where `recycleUnusedSegIndexes` could
delete segment index metadata using a stale `SegmentIndex` candidate
with no recorded index files, leaving the actual index files orphaned.
Changes:
- Reload the latest segment index metadata by build ID before GC removes
files/meta.
- Skip segment index GC for non-terminal index tasks.
- Keep finished segment index metadata when no index files are recorded,
instead of deleting meta with an empty file set.
- Add v1 orphan index file prefix cleanup for cases where metadata is
already missing.
issue: #50658
## Test Plan
- `GOTOOLCHAIN=auto make static-check`
- `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/datacoord -run
'TestGarbageCollector_recycleUnusedSegIndexes|TestGarbageCollector_recycleUnusedIndexFilesV1'`
- `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/datacoord -run 'TestGarbageCollector'`
Note: full `./internal/datacoord` was also run locally and failed only
on the existing unrelated
`TestCompactionTriggerManagerSuite/TestGetExpectedSegmentSize/all_DISKANN`
assertion (`expected 209715200`, `actual 104857600`).
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
issue: #50377
## Problem
Standalone crashes with SIGSEGV (exit 139) when an `ST_WITHIN` query
runs on a nullable GEOMETRY field of a **growing** segment concurrently
with schema-evolution (add field).
The reported stack:
```
GISFunctionFilterExpr.cpp -> segment_->bulk_subscript(...)
SegmentGrowingImpl.cpp -> insert_record_.get_valid_data(field_id)
/usr/include/c++/12/bits/unordered_map.h:880 (SIGSEGV)
```
## Root cause
`InsertRecordGrowing` keeps the per-field columns in two plain
`std::unordered_map`s (`data_` / `valid_data_`).
- The query path (`SegmentGrowingImpl::bulk_subscript` → `get_data_base`
/ `get_valid_data`, and every other growing-segment reader that funnels
through them) reads these maps **without any lock**.
- Schema evolution (`Reopen` → `fill_empty_field` → `append_field_meta`
→ `append_data` / `append_valid_data`) `emplace`s into the maps, which
can **rehash** the bucket array.
- A rehash concurrent with a lock-free `find()` / `at()` on a query
thread walks freed bucket memory → SIGSEGV. The GIS `ST_WITHIN` path
just happens to be a frequent reader (`bulk_subscript`).
Only `Insert` (shared) and `Reopen` (unique) took `sch_mutex_`; the
query path took nothing, so nothing serialized a reader against the
rehash.
## Fix
Add a dedicated `field_map_mutex_` to `InsertRecordGrowing`:
- structural writers (`append_*`, `drop_field_data`, `clear`) take it
**unique**;
- lookups (`get_data_base`, `get_valid_data`, `is_data_exist`,
`is_valid_data_exist`) take it **shared**.
The `append_data` paths resolve their reader-dependent values before
taking the unique lock to avoid recursive locking on the non-recursive
`shared_mutex`. The mutex is kept separate from the existing
`shared_mutex_` (which serializes pk inserts) so the hot read path does
not contend with inserts. Because all growing-segment readers funnel
through these accessors, this covers `bulk_subscript`, vector search and
expression chunk reads uniformly.
## Test
`internal/core/unittest/test_growing_concurrent_reopen.cpp` races
single-offset `bulk_subscript` readers against `Reopen`-driven field
additions. It reliably fails before the fix (failed map lookup under
debug/ASan; SIGSEGV in Release) and passes after.
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Issue
issue: #50549
## What this PR does
`tombstoneScheduler.AddPending` panicked with
`unreachable: tombstone scheduler is closing when adding pending
tombstone`
when `s.notifier.Context()` was already `Done()`.
That path **is** reachable under concurrent shutdown: an in-flight
`ackCallbackScheduler.doAckCallback` goroutine (spawned by
`triggerAckCallback`)
can call `AddPending` after `Close()` cancels the tombstone scheduler's
context.
The panic in a background goroutine crashes the whole process —
surfacing as a
flaky `ci-v2/ut-go` failure
(`TestForcePromoteFailover/no_incomplete_tasks`,
`panic: unreachable: tombstone scheduler is closing ...`).
## Why dropping the enqueue on shutdown is safe
`AddPending` only adds the broadcastID to the **in-memory** GC tracking
list.
The durable state is already written before we get here:
1. `doAckCallback` calls `bt.MarkAckCallbackDone(...)` which sets the
task state to
`BROADCAST_TASK_STATE_TOMBSTONE` and persists it (`saveTaskIfDirty`) —
*then*
calls `AddPending`.
2. On the next startup, `broadcast_manager` recovers all `TOMBSTONE`
tasks into
`tombstoneIDs` and re-seeds the GC list via `Initialize`.
So skipping the enqueue during shutdown only defers GC of that one
tombstone to
the next restart, which is already how GC behaves (lifetime + count
thresholds).
No data/cleanup is lost.
## Change
Replace the `panic` with a log + `return` when the context is already
cancelled.
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#### What this PR does / why we need it:
Improve QueryCoord balance stability by making scheduler segment task
deltas aware of current segment distribution.
When pending task deltas are not aligned with actual segment
distribution, balance policies can repeatedly shift plans based on stale
workload information. This PR tracks segment task delta effects with
action records and filters effects that have already been reflected in
distribution.
This also caches collection row count and delegator count in score
workload status to avoid repeated distribution scans, and applies the
updated segment delta snapshot to score and row-count balance paths.
Round-robin segment assignment keeps the lightweight segment-count only
ordering.
#### Which issue(s) this PR fixes:
Fixes#49860
#### Special notes for your reviewer:
MultiTargetBalancer is intentionally unchanged because it does not
consume scheduler segment task deltas in the original design.
#### Tests
- `gofumpt -l internal/querycoordv2/task/scheduler.go
internal/querycoordv2/assign/assign_policy_score.go
internal/querycoordv2/assign/assign_policy_rowcount.go
internal/querycoordv2/assign/assign_policy_roundrobin.go`
- `git diff --check`
- `go test -v -count=1 -tags dynamic,test -gcflags="all=-N -l"
-ldflags="-r ${MILVUS_WORK_DIR}/cmake_build/lib -r
${MILVUS_WORK_DIR}/internal/core/output/lib"
./internal/querycoordv2/assign -run
"Test(ScoreBasedAssignPolicy|RowCountBasedAssignPolicy|RoundRobinAssignPolicy)"
-timeout 300s`
- `go test -v -count=1 -tags dynamic,test -gcflags="all=-N -l"
-ldflags="-r ${MILVUS_WORK_DIR}/cmake_build/lib -r
${MILVUS_WORK_DIR}/internal/core/output/lib"
./internal/querycoordv2/task -run
"TestTask/(TestCalculateTaskDelta|TestSegmentTaskDeltaWithDistFilter|TestChannelTaskDeltaCache)"
-timeout 300s`
---------
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
related: #46442
This commit adds proper RBAC (Role-Based Access Control) support for the
/expr HTTP endpoint, replacing the previous root-only authentication.
Changes:
- Add new rbac.go with CheckPrivilege function for HTTP endpoints
- Support HTTP Basic Auth only (removed non-standard Bearer token
format)
- Integrate with existing Casbin RBAC framework
- Add PrivilegeExpr to GlobalLevelPrivileges and ClusterAdminPrivileges
- Register GetUserRoleFunc callback in meta_cache.go
- Update tests for new RBAC behavior
Security features:
- Authentication required when authorization is enabled
- Root user bypass when RootShouldBindRole is false
- Proper 401/403 status code differentiation
- Integration with privilege result cache
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
