3 people d6982c74c4 enhance: record and expose DataNode task cost_time / cost_cpu_num via QueryTask (#51443) ...

issue: #49156

## Summary

- Record `cost_time` / `cost_cpu_num` on DataNode index build tasks and
surface them through `QueryTask` response `Properties`.
- Reserve the unified `cost_time` / `cost_cpu_num` keys under
`pkg/taskcommon` so future PRs can wire up the other
`CreateTask`-dispatched task types.
- Introduce small `internal/datanode/taskcost` helper (`NowMs`,
`ElapsedMs`, `EstimateIndexBuildCPUNum`) for index build task cost
accounting.
- Final state/failReason and execution-end cost are written in a single
`TaskManager` critical section, and `QueryTask` reads them from a single
snapshot, so state and cost in one response are always consistent.

## Notes

This PR is producer-side only. DataCoord / proxy / metrics exporter do
not consume these properties yet; downstream consumption should land in
a follow-up linked from #49156.

`cost_cpu_num` is a coarse, statistics-only estimate for observability
(vector index ≈ knowhere build pool size = NumCPU in cluster mode;
scalar index = 1). It is not used for coordinator-side scheduling, so
the standalone pool-ratio deviation (NumCPU × buildIndexThreadPoolRatio)
and pool sharing across concurrent builds are intentionally not modeled.

Supersedes #49157 (that PR's head branch was rebased onto latest master
and force-pushed, so GitHub no longer allows reopening it). Content is
identical, rebased to a single commit on top of current master.

## Test plan

- [x] `go test ./taskcommon/...` from `pkg/`
- [x] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/datanode/taskcost`
- [ ] `go test -tags dynamic,test -gcflags="all=-N -l" -count=1
./internal/datanode/index -run
'TestIndexTaskSchedulerRecordsIndexTaskCost|Test_statsTaskInfoSuite/Test_IndexTaskCostMethods'`
- [ ] Manual sanity: `QueryTask` returns `cost_time` / `cost_cpu_num`
for DataNode index build task success and failure paths.

---------

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

2026-07-17 17:38:40 +08:00

..

cost_test.go

enhance: record and expose DataNode task cost_time / cost_cpu_num via QueryTask (#51443)

2026-07-17 17:38:40 +08:00

cost.go

enhance: record and expose DataNode task cost_time / cost_cpu_num via QueryTask (#51443)

2026-07-17 17:38:40 +08:00