High Availability (HA)

Syncplify Server! supports an active/active High Availability (HA) deployment model, where two (or more) nodes run simultaneously and continuously replicate data between them. This ensures that if one node goes down, the other(s) can continue serving clients without interruption.

How It Works

Active/Active Replication

In an HA cluster, every node is a fully functional Syncplify Server! instance. Each node:

• Accepts client connections independently
• Maintains a full copy of the configuration, virtual sites, users, and settings
• Continuously replicates changes to all other nodes in the cluster

There is no "primary" or "secondary" distinction: all nodes are equal peers. A change made on any node (e.g. creating a new user, modifying a virtual site, updating global configuration) is automatically replicated to the other node(s).

Peer to Peer Communication

Nodes communicate with each other over a dedicated IPC (Inter-Process Communication) channel. This channel uses HTTPS on a configurable port (set during initial setup). Each node must be able to reach the other's IPC endpoint.

Heartbeat and Health Monitoring

Each node periodically sends heartbeat messages to its peers. The heartbeat includes:

• The node's unique ID
• Its software version
• A timestamp (used to detect clock drift between nodes)
• Its current online/offline status

If a node stops sending heartbeats, the other nodes mark it as offline after a configurable timeout.

Clock Drift Detection

Accurate timekeeping is essential for replication consistency. Syncplify Server! monitors the clock difference between nodes and will warn administrators if the drift exceeds a safe threshold. Large clock drift can cause replication ordering issues and should be corrected promptly.

Version Compatibility

All nodes in a cluster must run the same major version of Syncplify Server!. When a node detects that a peer is running an incompatible version, it will refuse the join and report a version mismatch error.

Terminology

Term	Meaning
Node	A single Syncplify Server! instance participating in the cluster
Cluster	Two or more nodes that replicate data between each other
Heartbeat	A periodic health check message sent between nodes
Clock Drift	The time difference in seconds between two nodes' system clocks
Drain	Gracefully stop accepting new connections on a node while allowing existing ones to finish
Replication Cycle	A complete round of data synchronization between nodes

Requirements

• All nodes must be able to reach each other's IPC endpoint (HTTPS)
• All nodes must run the same major version of Syncplify Server!
• System clocks should be synchronized (use NTP)
• Each node must have a unique Node ID (assigned automatically during setup)
• The HA cluster is configured through the Setup UI on each node

What Gets Replicated

All configuration data is replicated across nodes:

• Virtual sites and their settings
• Users, groups, and permissions
• Admin and SuperAdmin accounts
• Global configuration settings
• Certificates and security settings
• Scripts and automation rules

What is NOT replicated:

• Transferred files (file storage replication is outside Syncplify Server!'s scope)
• Log files (each node maintains its own logs)
• Node specific settings (e.g. the IPC binding address)

Hardening the cluster channel

Two protections govern inter node traffic. Both ship in their permissive setting, because a cluster that half refuses to talk to itself is worse than the problem either one solves, and both are meant to be turned up once an operator has watched a cluster run without complaint. Neither has to be turned up for a cluster to work.

Channel verification

Every node holds an identity key, and a connection to a peer is expected to prove that the far end holds the key the membership roster records for it. The first time a link verifies, that fact is remembered, and from then on the link must keep verifying. A link that has never verified, which is what an unfinished rolling upgrade looks like, is used anyway with a warning.

--hachannelpolicy selects how strictly this is applied:

Value	Meaning
empty, or permissive	The default described above. Required once a link has ever verified.
strict	Always required. A peer that cannot prove its identity is not talked to at all.
off	Not checked. Only for a deployment that terminates TLS between nodes, which is unsupported for HA and means every replicated secret is readable at that hop.

Move to strict once the cluster status shows channelVerified true for every peer. Note that a load balancer or a proxy that terminates TLS between nodes can never satisfy this, by design.

Replicated collection policy

Replication carries records identified by a database and a collection name, and both are chosen by the sending node. The receiving node checks that pair against the set of collections this product actually replicates. Anything else is counted and reported, and by default still applied.

--hacollectionenforce turns that from counted into dropped.

Read the deniedWrites census in the cluster status before turning it on: whatever it lists is exactly what enforcement will discard. On a healthy cluster the census is empty, and turning enforcement on changes nothing. A census that is not empty is worth understanding before enforcing, because the honest explanations are very different from each other: a node running a build that replicates something this one does not, or a peer sending data it has no business sending.

The same policy governs the snapshot a node receives while joining a cluster, which is the one moment the answering peer has the least standing to be trusted. Anything the snapshot carried that this node does not replicate is reported as a notification on the joining node, whether or not enforcement is on.

Setting them

Both are flags of the serve command, so they can equally be given as SS_HACHANNELPOLICY and SS_HACOLLECTIONENFORCE in the environment, or set in the configuration file. They take effect on restart, and every node decides independently: a cluster part way through the change keeps working.