mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-20 09:45:43 +00:00
b9621d7d32
issue: #51497 After a successful WAL switch via `POST /management/wal/alter`, `GET /management/config/get?keys=mq.type` kept returning the pre-switch value with `"source":"RuntimeSource"` until the coordinator restarted, even though the ack callback had already persisted the new mq.type into etcd (and linearizably refreshed the local EtcdSource). External operators polling that endpoint to confirm switch completion never saw the new value. ### Root cause `InitAndSelectWALName()` saved the startup-resolved WAL name into the config manager's **runtime overlay**, and `Manager.GetConfig` serves the overlay with top priority — permanently shadowing the EtcdSource for every reader in the process. The overlay write existed only to feed `ProcessImmutableConfigs`, which pins `GetConfig`'s value into etcd create-if-absent at first boot (without it, the literal `"default"` would be persisted). ### Fix Replace the overlay bridge with an explicit per-key **renderer** on the persistence path, so mq.type reads fall through to the etcd source and naturally follow a WAL switch: - `ProcessImmutableConfigs(renderers map[string]func(raw string) string)`: a renderer converts a placeholder raw value (mq.type's literal `"default"`) into the concrete value at create-if-absent persist time. It never runs when etcd already holds the key (a completed switch survives restart), and it also covers the key-absent-from-all-sources case (raw is passed as `""`). Generic: any future immutable config with a placeholder value can plug in its own renderer. - `InitAndSelectWALName()` no longer writes the runtime overlay and returns the resolved `message.WALName`; the coordinator startup passes a mq.type renderer backed by that name. - `dependency.HealthCheck` resolves the literal `"default"` to the actually-enabled MQ before probing (this gap predates the overlay Save: since #36822, deployments running with `mq.type: default` probed nothing and always reported unhealthy). This also fixes two other readers of the shadowed value: - the `HandleAlterWAL` same-type short-circuit: re-posting the completed target no longer re-broadcasts, and a rollback to the original MQ is no longer silently swallowed with "already configured"; - the proxy MQ health check (`/_cluster/dependencies`) no longer keeps probing the old MQ forever after a switch. ### Behavior note `MQCfg.Type.GetValue()` now follows etcd at runtime instead of being frozen at process start: the coordinator sees a switch immediately (linearizable refresh in the ack callback); other nodes converge within one EtcdSource poll cycle (default `refreshInterval` 5s). Audited readers: `mustSelectMQType` / `MustSelectWALName` resolve `"default"` themselves and are startup-scoped; the AlterWAL idempotency check and the proxy health check are exactly the readers this change fixes. ### Verification - New tests (all watched failing first): renderer converts placeholder before first persist / existing etcd value never overwritten nor re-rendered / no-renderer keys unchanged / key absent from all sources still pinned via renderer (`pkg/config`); `InitAndSelectWALName` leaves mq.type served from its original source, not `RuntimeSource` (`streamingutil/util`); `HealthCheck("default")` resolves to the enabled MQ (`dependency`). - Full package runs green locally: `pkg/config`, `internal/util/streamingutil/util`, `internal/util/dependency` (with `-tags dynamic,test -gcflags="all=-N -l"`). - `cmd/roles`, `internal/coordinator`, `internal/distributed/streaming` type-checked via `go vet` locally (no current-master segcore build on this machine); relying on CI for full compilation. A live-cluster alter → config/get end-to-end pass is still pending — the switch-visibility chain (etcd write → `GetConfig` returns the new value) is covered at unit level by the existing `TestAlterConfigsInEtcd`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>