Skip to content

Chain-driven VM reconcile: per-root tokenId order is non-deterministic (#15) #936

Description

@branarakic

Follow-up from review of #934 (Codex finding #15 on finalization-handler.ts).

Problem

On the chain-driven reconcile path, promoteSharedMemoryToCanonical assigns each KA's on-chain tokenId positionally from the rootEntities array:

for (let tokenIdx = 0; tokenIdx < rootEntities.length; tokenIdx++) {
  const rootEntity = rootEntities[tokenIdx];
  ... tokenId: startKAId + BigInt(tokenIdx) ...
}

findSwmSnapshotInNamespace now sorts the candidate WorkspaceOperations by URI (PR #934, #15 partial), so all replicas pick the same op. But the per-op roots list is still built in SPARQL binding order (?op dkg:rootEntity ?root), which is store-dependent. The merkle root can't pin the order either: V10MerkleTree sorts+dedupes its leaves, so the merkle is order-insensitive and any root permutation verifies.

Consequence: two replicas reconciling the same KC from the chain can map the same rootEntity to different tokenIds, so a given UAL/tokenId resolves to different content on different nodes.

Why it's not a one-line fix

The authoritative root→tokenId mapping lives in the publisher's manifest (gossip-publish-handler.ts, KAManifestEntry.tokenId; manifest order = kaMap insertion order in canonical-publish-payload.ts). The gossip/finalization path gets that order from the wire (msg.rootEntities). The chain-driven path has no wire, and the SWM workspace-meta graph it reads persists per-root privateMerkleRoot, publisherPeerId, ownership, and keepRootCopyOnLabel — but not a per-root tokenId or sequence index. So the originator's order is unrecoverable from RDF.

A naive roots.sort() makes chain-driven replicas mutually consistent but may diverge from the originator + gossip-path replicas.

Options

  1. Persist the index at publish time (preferred): publisher writes <root> dkg:kaIndex N (or the tokenId offset) into SWM workspace meta; chain-driven path orders roots by it. Exact, matches gossip path. Cross-package (publisher + agent).
  2. Shared canonical sort: define a deterministic root ordering (e.g. by privateMerkleRoot, or by root IRI) and adopt it on BOTH the publisher manifest builder and the chain-driven path so they agree. No new persisted field, but touches publish-side ordering.

Option 1 is cleaner and lower-risk to the publish path. Needs a design call before implementation.

Scope

  • packages/agent/src/finalization-handler.ts (consumer)
  • packages/publisher/src/canonical-publish-payload.ts / workspace-handler.ts (producer of the order / meta)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpre-mainnetMust land before mainnet launchpriority:highMust-fix: protocol correctness, security, economics, or headline flow broken

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions