Zhen YeandGitHub
6b3eabb1ff
issue: #51446 - message properties: add an explicit `Unreplicable` marker for concrete WAL messages and expose builder opt-in through `WithUnreplicable` - replication: skip unreplicable messages in the CDC sender and secondary replicate interceptor before callback replay - DDL producers: mark unsupported snapshot, manifest backfill, and external collection refresh broadcasts as unreplicable - streaming docs: document the message-level skip marker and current unsupported DDL replication behavior Signed-off-by: chyezh <chyezh@outlook.com>
4.0 KiB
Message Model
Every WAL entry is a Message — the fundamental data unit flowing through all WAL components. A message consists of a typed payload (protobuf-encoded header + body) and key-value properties (map[string]string, reserved keys prefixed with _).
Message Lifecycle
Messages transition through three stages:
-
BroadcastMutableMessage: Created by the Broadcaster for messages that target multiple VChannels (DDL/DCL/WALInternal broadcasts). Carries a
BroadcastHeaderwith the list of target VChannels and ResourceKeys.SplitIntoMutableMessage()splits it into per-VChannel MutableMessages for individual WAL append. -
MutableMessage: The pre-append state. Properties can be modified by the WAL interceptor chain (attaching TimeTick, LastConfirmed, TxnContext, WALTerm, etc.). Created either by client-side builders (DML) or by splitting a BroadcastMutableMessage. Transitions to ImmutableMessage via
IntoImmutableMessage(msgID)after WAL persistence. -
ImmutableMessage: The post-append, read-only state. Carries a backend-assigned MessageID and LastConfirmedMessageID. Returned by WAL Read/Consume operations. For transactions, multiple ImmutableMessages are assembled into an ImmutableTxnMessage (Begin + body messages + Commit) by the consumer-side TxnBuffer.
-
ReplicateMutableMessage: Created from a source cluster's ImmutableMessage for cross-cluster replication. Attaches a
ReplicateHeaderpreserving the source cluster's original Message Properties, then re-enters the local WAL as a MutableMessage. Replication honors message-level compatibility markers such asUnreplicable.
Key Properties
| Property | Description |
|---|---|
| Version | Payload format version, bound at build time for compatibility. VersionOld(0): legacy format before StreamingNode, to be removed in future. V1(1): payload still uses msgstream serialization. V2(2): payload fully independent of msgstream. |
| MessageType | The kind of message. Determines how payload is decoded. |
| VChannel | Target virtual channel. Empty means visible to all VChannels on the PChannel. |
| IsPChannelLevel | Marks cluster-level broadcast messages handled at PChannel scope. |
| BroadcastHeader | Broadcast metadata for messages targeting multiple VChannels. See Broadcaster. |
| TimeTick | PChannel-level log sequence number. See TimeTick. |
| LastConfirmedMessageID | Reading from this MessageID guarantees all subsequent messages have TimeTick greater than this message's TimeTick (including txn messages). |
| TxnContext | Links the message to a transaction. Nil if non-transactional. |
| ReplicateHeader | Source cluster's original message metadata for cross-cluster replication. |
Unreplicable (_ur) |
Marks this concrete message as unsafe for cross-cluster replication. Replication skips only messages carrying this property; absence means replicable, including old WAL messages written before the marker existed. |
| MessageID | Backend-assigned unique identifier. |
| PChannel | The PChannel this message belongs to. |
Message Semantic Docs
- Collection Messages — DDL, partition, index, snapshot, import, DML, segment, load config
- Alias Messages — alias create/drop
- Database Messages — database lifecycle
- RBAC Messages — users, roles, privileges
- Transaction Messages — begin, commit, rollback
- Cluster Messages — global barriers, replication config, resource groups
- TimeTick Message — visibility barrier
Adding a New Message Type
New message types MUST be defined via codegen/reflect_info.json and pkg/streaming/util/message/codegen/. Do not manually write builder or type-conversion functions.
Key Packages
pkg/streaming/util/message/— Message types, builders, properties, codegen, legacy adaptor