mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 18:25:44 +00:00
7ef2d32c49
## What this PR does Adds an integration test (`TestWpRetentionTruncateRead`) for the Woodpecker (wp) WAL implementation pinning the read semantics around truncation that Milvus relies on but had no explicit coverage for. Milvus truncates the wp WAL aggressively right after a flush, but truncation only moves a metadata watermark — it does **not** delete data. Data at/under the watermark stays readable until the retention-TTL GC physically removes it. The test pins this with three sub-cases: 1. **TruncatedDataStillReadable** — after truncating to the latest id, an explicit `StartFrom` at any position in `(earliest, latest]` (spanning segment boundaries) still returns every message, including the watermark id itself. As a contrast, a fresh `DeliverPolicy_All` reader is parked past the watermark and returns nothing — proving the watermark really moved while the data underneath is intact. 2. **SeekForwardWithinSegment** / **SeekForwardCrossSegment** — a reader asking for data older than the earliest deliverable position (`DeliverPolicy_All` after truncation) is moved forward to the first available position instead of erroring or stalling. The two sub-cases exercise both branches of `adjustPendingReadPointIfTruncated`: a truncation point inside segment 0 (within-segment) vs. in a later segment (cross-segment id jump). ## Notes - **Deterministic by design**: large retention (`72h`) so GC never fires, a small `segmentRollingPolicy.maxSize` so writes span several segments, single-threaded writes so ids are monotonic, and all assertions are message-id based. Run logs confirm each sub-case hits its intended adjustment branch (`truncatedSegmentId` 0 vs >0). - **Scope**: covers the open-time read-position adjustment after truncation. It does NOT exercise the physical retention-TTL GC + runtime skip-forward path — that needs waiting for the async GC sweep and is intentionally left out to keep the test deterministic. - Runs against local fs storage; the truncate/read logic under test lives in the woodpecker log handle/reader and is storage-backend agnostic. issue: #43638 --------- Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>