Edge sync with a clustered hub

Edge sync ships Parquet files from an edge Arc (the spoke) to a central Arc (the hub). The spoke is normally a single node. The hub, in an Enterprise deployment, is usually not — and the two transports touch a cluster at different points.

This page covers what changes when the hub is a cluster. Everything in the OSS edge sync guide still applies.

The short version

Network transport: Any node works

A spoke has a single hub_url, and every request goes to whatever that resolves to. There is no writer-role gate on the receive path — unlike retention or continuous queries, which check IsPrimaryWriter() on every tick, a node receiving a file simply accepts it.

That is safe because the manifest write forwards:

receiving node is the Raft leader   → applies locally
receiving node is a follower        → forwards the command to the current leader

So a spoke can push at any node and the file still lands in the shared Raft manifest, becoming queryable from your readers.

Which node should a spoke point at?

Any of them. In practice:

• A writer is the natural choice — it already holds storage credentials and is sized for write throughput.
• A load balancer across the writers works and survives a node going down. The spoke retries on the next pass regardless, and re-delivery of an already-received file is a no-op, so a mid-transfer failover costs one repeated transfer at worst.
• A reader works too, though it means a follower forwarding every manifest write to the leader — an extra hop for no benefit.

The compactor is invisible to edge sync. Synced files land as ordinary Parquet under {spoke_id}/{database}/{measurement}/... and are compacted like anything else.

Cluster TLS

If cluster.tls_enabled is set, that governs Raft RPC and peer fetch — not the spoke's connection. A spoke connects over the public API listener, so its transport security keys off server.tls_enabled. These are independent flags; see Cluster Security.

Air-gap transport: Designate one import node

This is the part that needs an operational decision.

Two pieces of edge-sync state live in node-local SQLite (the shared metadata database), not in the Raft manifest:

• sync_received — the index reconcile answers from, so it can respond without re-hashing storage.
• sync_imported_bundles — the air-gap dedup ledger, keyed (spoke_id, bundle_id).

Neither is replicated. So on a multi-node hub:

That last row is worth being precise about: the re-import is safe, not harmful. Every file resolves to already_present, the §6.1 identity rule still refuses to overwrite differing content, and nothing is corrupted. What you lose is the refusal — and GET /api/v1/bundle-import/history/{spoke_id} on node B will not show what node A imported.

Raft proposal batching

Import registers manifest entries in batches of 1000 operations per Raft proposal.

This matters more than it might sound. The network path is naturally rate-limited to one proposal per HTTP request; an import is a tight loop over a whole drive. Without batching, a 10,000-file bundle would fire 10,000 individual proposals at your leader as fast as the disk allows. Batching makes that 10 proposals.

If a batch fails — a quorum loss, say — the import aborts rather than continuing. Files already committed to storage stay there, but the bundle is not recorded as imported, so re-importing it re-registers everything. That is the intended recovery.

One import at a time

Imports are serialized per node by a mutex. A second concurrent import returns 409 with "reason": "import in progress" rather than queuing behind the first — an import can legitimately run for hours, and a request hanging that long is indistinguishable from a dead hub.

Scheduling from a clustered spoke (planned for 27.01.1)

Spoke scheduling is separate from hub routing. The planned 27.01.1 scheduler requires a valid paid license on the spoke and checks IsPrimaryWriter() on every tick. Demotion cancels its active scheduled pass on the next tick; a later tick can start a pass when eligibility returns. It uses the eligible node's local ledger and does not replicate that ledger.

See scheduled network sync for configuration and monitoring. Earlier releases require manual triggers or an external timer; manual sync remains available without a license.

Failover

For a failover on the hub, the spoke can keep the same URL if it points at a load balancer:

• Network — if the spoke's hub_url points at a load balancer, the next pass simply lands on a healthy node. If it points at a specific node that goes down, the pass fails and retries on the next trigger; the ledger keeps its state, so nothing is lost or double-sent.
• Air gap — if your designated import node is down, import on another node. You lose dedup and history for that bundle, per the table above; the data itself is correct.

What is not cluster-aware yet

Stated plainly so it is not a surprise:

• The received-files index and dedup ledger are node-local. Replicating them would make any node a valid import target and give fleet-wide dedup. It is a candidate for a later release, not a defect — the current behaviour is safe, just node-scoped.
• There is no automatic reconciliation between the manifest and storage. Orphaned manifest entries are logged, not self-healed.