Files
milvus/internal/agg/operators.go
T
e2787d3981 enhance: standardize error handling on merr + Sys/Input classification (#50221)
issue: #47420

## What this PR does

Project-wide migration of raw `fmt.Errorf` / `errors.New` in function
bodies onto
the `merr` framework, plus the Sys-vs-Input error classification and the
machinery it drives (retriability, fine-grained metrics, segcore
unification),
plus the convention docs and a linter that keeps it from regressing.

Scope: storage, proxy, coordinators (root/data/query), query node, data
node,
`pkg/util` & `internal/util`, expression parser, message queue,
streaming, and
misc packages. Bare raw-error usages went from ~3000 to a ~340 allowlist
(package-level sentinels / build-tag / test sites).

---

## How to review this PR

It is large but the vast majority is mechanical. Changes fall into three
tiers;
spend review budget on Part 2 and Part 3.

### Part 1 — Mechanical standardization (low risk, verify by rule)

Each converted call follows one of a small fixed set of rules. To
review, check
that each site obeys the matching rule rather than reading every line:

| Pattern | Rule |
|---|---|
| `fmt.Errorf("...")` originating a new error | →
`merr.WrapErrXxxMsg("...")` with a code matching the failure's meaning |
| Adding context to an existing typed error | → `merr.Wrap(err, "...")`
/ `merr.Wrapf(...)` — **preserves** the inner code (never `WrapErr*Err`,
which overwrites it) |
| Errors inside the streaming subsystem | → `status.New*` factories
(StreamingError), **not** merr — this is the component-internal dialect
(see `docs/dev/error_handling_guide.md`) |
| Low-level / control-flow signal caught by `errors.Is` | → kept as a
package-level `errors.New` sentinel (lowercase, same-package) |

Conventions are documented in `docs/dev/error_handling_guide.md`
(how-to) and
`docs/dev/error_sentinel_convention.md` (rules + audit). A
`gocritic`/`ruleguard`
rule (`rawmerrerror`, in `rules.go`) enforces "no raw `return
errors.New/fmt.Errorf`"
under `make verifiers`.

### Part 2 — Behavior changes (review these closely)

These are the sites where the wire contract or runtime behavior changes,
not just
the source text. Listed by category; representative locations given,
full set in
the diff.

**A. gRPC wire-code shifts: `UnexpectedError(1)/Code 65535` → typed
code.**
Where a handler previously returned a raw error (collapsed to
`Code=65535` on the
wire), it now returns a typed merr, so the client sees a real code. The
most
common shift is to `IllegalArgument(5)/Code 1100` (ParameterInvalid).
Touch
points include datanode task handlers (CreateTask/Query/Drop), proxy
Upsert,
querynode GetMetrics, datacoord CreateIndex, httpserver query-response
builder,
and typeutil schema validation. One code refinement: an index-param
validation
moved `1100` → `1101` (ParameterMissing). **Client/SDK assertions and
any code
that switched on `Code=65535` for these paths must be re-checked** (the
go_client
e2e assertions were already aligned in this PR).

**B. Prometheus `status` label contract change (externally visible).**
The proxy metric's coarse `fail` / `rejected` values are split into
`fail_input` / `fail_system` and `rejected_user` / `rejected_system` (in
`requestutil.ParseMetricLabel`; auth/privilege rejections count as
`rejected_user`), so dashboards can attribute a failure to caller vs
operator.
**Dashboards/alerts querying `status="fail"` must migrate to
`status=~"fail_.*"`, and `status="rejected"` to
`status=~"rejected_.*"`.** The
in-repo Grafana dashboard is already migrated; external dashboards built
on the
old values silently go empty after upgrade. This is the one change that
requires an ops-side migration.

**C. Retriability semantics.**
- C1: `merr.Status(err)` now forces `Retriable=false` when the error is
an
`InputError` — a malformed request can never succeed on blind retry, so
clients
never get the self-contradictory "your input is wrong but you may
retry".
- C2: `retry.Do` short-circuits an `InputError` (non-retriable) — **but
only when
  the caller did not pass a `RetryErr` predicate**. The check is an
`if c.isRetryErr != nil { ... } else if InputError { ... }` *mutually
exclusive*
branch (`pkg/util/retry/retry.go`): an explicit `RetryErr` takes
precedence and
bypasses the InputError abort. `retry.Handle` deliberately does **not**
apply
the InputError abort (its callers signal abort via `shouldRetry=false`).
Four
flusher startup callsites that must retry through transient "not ready"
errors
  were given explicit `RetryErr` escape hatches.

**D. segcore (C++→Go) error classification.**
A single shared Go-side table (`pkg/util/merr/segcore.go`) maps each
segcore code
to a merr sentinel + InputError/signal category, replacing scattered
hand-written
`if errorCode == ...` switches in the cgo wrappers. **Wire `Code` values
change
for every segcore pass-through error, not just the remapped ones.**
Named
sentinels remap (C++ `2003` → merr `2001`, `2033` → `2002`,
Folly/Knowhere codes
likewise); **all remaining pass-through codes (`2004`–`2043`, previously
surfaced to clients as raw C++ enum values) now serialize as `2000`**
(`ErrSegcore`), with the original C++ code preserved in the `Reason`
text
(`segcoreCode=...`); unknown/future codes collapse to `2000` as well
(pinned by
the `wire_code_projection` test). Transient segcore classes (object
storage /
file IO / OOM / mmap / FieldNotLoaded — 11 codes) now report
`Retriable=true`.
**Any client switching on raw segcore codes in the `2004`–`2043` range
must be
re-checked**; the in-Reason code remains available for diagnostics.
Signal
codes (PretendFinished / FollyCancel) are recognized centrally.
`errors.Is`-based
control flow on these (e.g. scheduler skip/retry) is preserved.

**E. InputError classification (25 sentinels + dynamic marks).**
25 sentinels in `errors.go` carry `WithErrorType(InputError)` (the
Collection /
ResourceGroup / Database families, `ErrIndexDuplicate`,
`ErrParameterInvalid`,
`ErrPrivilegeNotAuthenticated`, `ErrImportFailed`, `ErrQueryPlan`, ...),
plus dynamic
marks for the 8 segcore input codes (ExprInvalid, DimNotMatch,
MetricTypeInvalid, FieldIDInvalid, ...) and
`WrapErrAsInputError`. The widest blast radius is `ErrParameterInvalid`
(1100):
~2335 `WrapErrParameterInvalid*` callsites now classify as input /
non-retriable. Because of C1/C2 this changes retriability for
any path that returns these. **The audit to confirm no transient path
was
mis-marked is the single most important review item** (see Part 3). One
reverse
correction: storage field-stats parsing moved from `ErrParameterInvalid`
(input)
to `ErrDataIntegrity` — a corrupted stored stat is data corruption, not
user
input.

### Part 3 — Known risks & traps (called out proactively)

1. **`merr.Wrap` vs `WrapErr*Err` (code-masking).** `WrapErr*Err` builds
a
`wrappedMilvusError{sentinel: ErrServiceInternal}` whose `code()`
returns the
*outer* sentinel — it overwrites the inner typed code and hides the
`errors.Is`
chain. This is intentional (use it to *deliberately* downgrade), but it
was a
recurring conversion defect; the rule "add context with `merr.Wrap`,
downgrade
with `WrapErr*Err`" is enforced by convention and reviewed across the
diff.
2. **InputError × `retry.Do` blast radius.** Marking a sentinel
`InputError` makes
any `retry.Do(...)` without a `RetryErr` predicate stop retrying it.
Reviewers
should sanity-check that no transient use of the 19 newly-marked
sentinels
(especially `ErrParameterInvalid`) sits inside a retry loop that needed
to keep
   spinning. The known flusher cases were handled (see C2).
3. **The ~340 raw-error allowlist.** What remains as bare `errors.New`
is, by
design: package-level sentinels (caught by `errors.Is`), `//go:build
test`
sites, and out-of-band trees (`cmd/`, `tests/`, codegen, walimpls). The
linter
only bans the *direct-return* form; assignment-then-return escapes and
the full
no-exceptions ban are deferred to an AST-based linter (Tier 2,
documented).
4. **segcore C++ second step deferred.** This PR unifies classification
on the Go
side; splitting the dual-semantic C++ codes at the source is a
follow-up.

---

## Validation

- `make verifiers`: Go side clean (gofmt + static-check across modules,
including
  the new `rawmerrerror` rule with a 0-hit baseline repo-wide).
- `make test-go`: passing; the one real regression introduced (a
datanode
`invalid_task_type` assertion shifting `1` → `5` from a ParameterInvalid
  conversion) was fixed in-tree.
- go_client e2e CreateIndex assertions aligned to the new merr messages.

---------

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 15:04:51 -07:00

361 lines
9.1 KiB
Go

package agg
import (
"github.com/milvus-io/milvus/pkg/v3/proto/planpb"
"github.com/milvus-io/milvus/pkg/v3/util/merr"
)
func terminateSingleSlot(slots []*FieldValue) (any, error) {
if len(slots) != 1 {
return nil, merr.WrapErrParameterInvalidMsg("aggregate expects 1 accumulator slot, got %d", len(slots))
}
if slots[0] == nil || slots[0].IsNull() {
return nil, nil
}
return slots[0].Value(), nil
}
func terminateAvg(slots []*FieldValue) (any, error) {
if len(slots) != 2 {
return nil, merr.WrapErrParameterInvalidMsg("avg expects 2 accumulator slots, got %d", len(slots))
}
sum := slots[0]
count := slots[1]
if sum == nil || count == nil || sum.IsNull() || count.IsNull() {
return nil, nil
}
sumF, err := fieldValueToFloat64(sum.Value())
if err != nil {
return nil, err
}
countF, err := fieldValueToFloat64(count.Value())
if err != nil {
return nil, err
}
if countF == 0 {
return nil, nil
}
return sumF / countF, nil
}
func fieldValueToFloat64(v any) (float64, error) {
switch value := v.(type) {
case int:
return float64(value), nil
case int32:
return float64(value), nil
case int64:
return float64(value), nil
case float32:
return float64(value), nil
case float64:
return value, nil
default:
return 0, merr.WrapErrParameterInvalidMsg("avg expects numeric accumulator, got %T", v)
}
}
func newSingleSlotState() []*FieldValue {
return []*FieldValue{NewNullFieldValue()}
}
type SumAggregate struct {
fieldID int64
originalName string
isAvg bool
}
func (sum *SumAggregate) Name() string {
return kSum
}
func (sum *SumAggregate) Update(target *FieldValue, new *FieldValue) error {
return AccumulateFieldValue(target, new)
}
func (sum *SumAggregate) NewState() []*FieldValue {
return newSingleSlotState()
}
func (sum *SumAggregate) UpdateState(slots []*FieldValue, new *FieldValue) error {
if len(slots) != 1 {
return merr.WrapErrParameterInvalidMsg("aggregate expects 1 accumulator slot, got %d", len(slots))
}
return sum.Update(slots[0], new)
}
func (sum *SumAggregate) Terminate(slots []*FieldValue) (any, error) {
return terminateSingleSlot(slots)
}
func (sum *SumAggregate) ToPB() *planpb.Aggregate {
return &planpb.Aggregate{Op: planpb.AggregateOp_sum, FieldId: sum.FieldID()}
}
func (sum *SumAggregate) FieldID() int64 {
return sum.fieldID
}
func (sum *SumAggregate) OriginalName() string {
return sum.originalName
}
type CountAggregate struct {
fieldID int64
originalName string
isAvg bool
}
func (count *CountAggregate) Name() string {
return kCount
}
func (count *CountAggregate) Update(target *FieldValue, new *FieldValue) error {
return AccumulateFieldValue(target, new)
}
func (count *CountAggregate) NewState() []*FieldValue {
return newSingleSlotState()
}
func (count *CountAggregate) UpdateState(slots []*FieldValue, new *FieldValue) error {
if len(slots) != 1 {
return merr.WrapErrParameterInvalidMsg("aggregate expects 1 accumulator slot, got %d", len(slots))
}
if new == nil || new.IsNull() {
return nil
}
return count.Update(slots[0], NewFieldValue(int64(1)))
}
func (count *CountAggregate) Terminate(slots []*FieldValue) (any, error) {
return terminateSingleSlot(slots)
}
func (count *CountAggregate) ToPB() *planpb.Aggregate {
return &planpb.Aggregate{Op: planpb.AggregateOp_count, FieldId: count.FieldID()}
}
func (count *CountAggregate) FieldID() int64 {
return count.fieldID
}
func (count *CountAggregate) OriginalName() string {
return count.originalName
}
type AvgAggregate struct {
fieldID int64
originalName string
sum *SumAggregate
count *CountAggregate
}
func NewAvgAggregate(fieldID int64, originalName string) *AvgAggregate {
return &AvgAggregate{
fieldID: fieldID,
originalName: originalName,
sum: &SumAggregate{fieldID: fieldID, originalName: originalName, isAvg: true},
count: &CountAggregate{fieldID: fieldID, originalName: originalName, isAvg: true},
}
}
func (avg *AvgAggregate) Name() string {
return kAvg
}
func (avg *AvgAggregate) Update(target *FieldValue, new *FieldValue) error {
return merr.WrapErrServiceInternalMsg("avg aggregate updates require aggregate state slots")
}
func (avg *AvgAggregate) NewState() []*FieldValue {
return append(avg.sum.NewState(), avg.count.NewState()...)
}
func (avg *AvgAggregate) UpdateState(slots []*FieldValue, new *FieldValue) error {
if len(slots) != 2 {
return merr.WrapErrParameterInvalidMsg("avg expects 2 accumulator slots, got %d", len(slots))
}
if err := avg.sum.Update(slots[0], new); err != nil {
return err
}
if new == nil || new.IsNull() {
return nil
}
return avg.count.Update(slots[1], NewFieldValue(int64(1)))
}
func (avg *AvgAggregate) Terminate(slots []*FieldValue) (any, error) {
return terminateAvg(slots)
}
func (avg *AvgAggregate) ToPB() *planpb.Aggregate {
return &planpb.Aggregate{Op: planpb.AggregateOp_avg, FieldId: avg.FieldID()}
}
func (avg *AvgAggregate) FieldID() int64 {
return avg.fieldID
}
func (avg *AvgAggregate) OriginalName() string {
return avg.originalName
}
func updateOrderedState(target *FieldValue, new *FieldValue, op string) error {
if target == nil || new == nil {
return merr.WrapErrServiceInternalMsg("target or new field value is nil")
}
if new.IsNull() {
return nil
}
if target.IsNull() {
target.val = new.val
target.isNull = false
return nil
}
if target.val == nil {
target.val = new.val
return nil
}
replace, err := shouldReplaceOrderedValue(target.val, new.val, op)
if err != nil {
return err
}
if replace {
target.val = new.val
}
return nil
}
func shouldReplaceOrderedValue(target, new any, op string) (bool, error) {
switch targetVal := target.(type) {
case int:
newVal, ok := new.(int)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is int, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
case int32:
newVal, ok := new.(int32)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is int32, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
case int64:
newVal, ok := new.(int64)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is int64, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
case float32:
newVal, ok := new.(float32)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is float32, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
case float64:
newVal, ok := new.(float64)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is float64, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
case string:
newVal, ok := new.(string)
if !ok {
return false, merr.WrapErrParameterInvalidMsg("type mismatch: target is string, new is %T", new)
}
return shouldReplaceOrdered(newVal, targetVal, op), nil
default:
return false, merr.WrapErrParameterInvalidMsg("unsupported type for %s aggregation: %T", op, target)
}
}
// Direct comparisons preserve existing NaN behavior for float min/max.
func shouldReplaceOrdered[T ~int | ~int32 | ~int64 | ~float32 | ~float64 | ~string](newVal, targetVal T, op string) bool {
if op == kMin {
return newVal < targetVal
}
return newVal > targetVal
}
type MinAggregate struct {
fieldID int64
originalName string
}
func (min *MinAggregate) Name() string {
return kMin
}
func (min *MinAggregate) Update(target *FieldValue, new *FieldValue) error {
return updateOrderedState(target, new, kMin)
}
func (min *MinAggregate) NewState() []*FieldValue {
return newSingleSlotState()
}
func (min *MinAggregate) UpdateState(slots []*FieldValue, new *FieldValue) error {
if len(slots) != 1 {
return merr.WrapErrParameterInvalidMsg("aggregate expects 1 accumulator slot, got %d", len(slots))
}
return min.Update(slots[0], new)
}
func (min *MinAggregate) Terminate(slots []*FieldValue) (any, error) {
return terminateSingleSlot(slots)
}
func (min *MinAggregate) ToPB() *planpb.Aggregate {
return &planpb.Aggregate{Op: planpb.AggregateOp_min, FieldId: min.FieldID()}
}
func (min *MinAggregate) FieldID() int64 {
return min.fieldID
}
func (min *MinAggregate) OriginalName() string {
return min.originalName
}
type MaxAggregate struct {
fieldID int64
originalName string
}
func (max *MaxAggregate) Name() string {
return kMax
}
func (max *MaxAggregate) Update(target *FieldValue, new *FieldValue) error {
return updateOrderedState(target, new, kMax)
}
func (max *MaxAggregate) NewState() []*FieldValue {
return newSingleSlotState()
}
func (max *MaxAggregate) UpdateState(slots []*FieldValue, new *FieldValue) error {
if len(slots) != 1 {
return merr.WrapErrParameterInvalidMsg("aggregate expects 1 accumulator slot, got %d", len(slots))
}
return max.Update(slots[0], new)
}
func (max *MaxAggregate) Terminate(slots []*FieldValue) (any, error) {
return terminateSingleSlot(slots)
}
func (max *MaxAggregate) ToPB() *planpb.Aggregate {
return &planpb.Aggregate{Op: planpb.AggregateOp_max, FieldId: max.FieldID()}
}
func (max *MaxAggregate) FieldID() int64 {
return max.fieldID
}
func (max *MaxAggregate) OriginalName() string {
return max.originalName
}