Fencing

Fencing levels and durable proof

Fencing makes sure the site being given up can no longer run the application: its host powered off, its VM stopped, its traffic cut. qfence promotes another site only once that is proven, never merely requested. If it cannot prove it, it stops (spec §17).

Targets, per site

A plan lists its fencing targets: controller capabilities bound to the plan, each at a site. A failover to dc2 fences every site but dc2: dc1’s targets, and any target that names no site (ADR 0002). With unfence_on_promote, the promoted site’s own targets are then reversed: its host powered on, its VM started. That is how the new owner is started, once nothing else can run.

Each target is required or secondary. Required targets decide whether the site counts as fenced. Secondary ones are tried too, and their result is reported, but never relied on.

Levels

Every capability declares how strong its fence is (spec §14):

Level	What it stops	Examples
Compute	What the application runs on	Redfish or IPMI power off, a Proxmox VM stopped, a Docker service scaled to its fenced shape
Network	Traffic to and from it	A FortiGate policy disabled, an interface or VLAN taken down, a VM’s network link cut
Resource	Its access to something it uses, such as storage	None of the built-in controller types has one yet
Application	The application, through its own interface	A PostgreSQL primary demoted, a systemd unit stopped

    flowchart LR
  ac["Application controller<br/>db.readonly, unit.stop"] -->|application| app
  fw["Firewall<br/>policy.disable"] -->|network| net
  hv["Hypervisor or Docker<br/>vm.stop, service.shape"] -->|compute| vm
  bmc["BMC<br/>power.off"] -->|compute| host
  subgraph site["The site being fenced"]
    app["The application"]
    net["Its network path"]
    vm["Its VM or container"]
    host["Its host"]
  end

Where a compute fence exists, it must be required. A connection already open, or a write in flight, can outlive a firewall rule, so a network fence is not proof that the old owner stopped.
Network-only fencing is allowed only when the plan says allow_network_only_fencing: true. That is audited, and the dry run shows it as a weaker guarantee.
An application-level fence never proves a site stopped. It is always secondary: it acts through the application before its host is switched off, and the compute or network fence is the proof (ADR 0005 §7).

Verified, or not safe

A fence is FENCED only once qfence has read back that it holds: the power state reads Off, the VM reads stopped. A fence that cannot be read back is UNKNOWN, and UNKNOWN is treated as not fenced. A failover whose required fences do not all verify stops at BLOCKED, and nothing is promoted.

    flowchart TB
  send["Send the fence"] --> read["Read it back"]
  read -->|"off, stopped, disabled"| fenced["FENCED"]
  read -->|"still running"| failed["FAILED"]
  read -->|"no answer, a timeout, a state it does not know"| unknown["UNKNOWN<br/>treated as not fenced"]
  fenced --> lasts{"Does it survive<br/>a restart?"}
  lasts -->|yes| durable["Durable<br/>the site is confirmed fenced"]
  lasts -->|no| transient["Transient<br/>the site owes a fence"]

How many required targets must succeed is the plan’s policy: ALL, the default, or ANY, where one verified target fences its site. Under ANY, a site’s targets are tried in the order listed, gentlest first, and qfence stops at the first that proves the site fenced: least force (ADR 0009 §2). A site’s secondary application-level fences run before its required ones, for at most 10 seconds, so they can act while the host is still up.

The path a fence takes matters as well. A BMC on its own management network can still be reached when the production network is cut; a fence that shares the workload’s network may not be. Each controller declares its network_domain, the network it is reached through, and the dry run warns of a compute or network fence that declares none (spec §15).

Durable proof

A fence is durable when it survives a restart of what it fenced: a host powered off whose BMC will keep it off when power returns (Redfish PowerRestorePolicy set to AlwaysOff), a VM stopped and not set to start on boot. It is transient when a restart undoes it: a host powered off that the facility will power on again. qfence reads which it is from the target, and does not assume it (ADR 0009 §1, §3).

A site is confirmed fenced only by a durable proof. A site fenced only transiently owes a fence: qfence keeps trying to make it durable, and keeps the survivor’s shields up until it is (returning sites). qfence’s Redfish and IPMI power fences set the restore policy to stay off before powering off, and set it back when the site is promoted.

When a fence cannot be reached

When a whole site is dark, its BMCs do not answer, so its fences read UNKNOWN and the failover is BLOCKED. That is the right answer for software that cannot see the site. A person who can see it has two ways forward:

Attest it. Someone with the force permission (EmergencyOperator or Administrator) states, in at least 20 characters, how they know the site is down, and names the targets that statement covers. qfence still tries those fences, and counts the statement only where a fence still cannot answer. The failover is audited as forced, with the statement, and the attested site owes a fence when it comes back (ADR 0003). If the statement is wrong, two sites may run: an attestation is the one way past a fence, and it rests on the person making it.
Accept data loss. When the target’s replica is older than the plan’s readiness allows, someone with the force permission can accept up to a stated lag. qfence promotes only if the replica is within it, and records the lag it measured (ADR 0006).

A plan can require a second person to approve either one (approvals: 2). Automatic failover never attests and never accepts data loss.

The witness Returning sites