Operate
Operate
Day-to-day operation of a running cluster. Everything here can be done in the console, on a plan’s page, as well as through the API; the pages show both.
- The dry run: what a failover would do, before it does anything.
- Failover and switchover: moving a plan between sites, and back.
- Manual fence and forced operations: acting outside the usual sequence, audited as forced.
- The attention list: what needs a person now, and what to do about each item.
- Runbooks: a site lost, a failover BLOCKED, a site coming back, and more.
Calling the API
The commands use the orchestrator’s own client, qfence-orchestrator api, as a shell function. Step 6 of the install guide sets it up on a site:
api() { /opt/qfence/qfence-orchestrator api -server https://127.0.0.1:8443 -token-file admin.token -ca-file /var/lib/qfence-orchestrator/api-cert.pem "$@"; }Get started sets up the same for the demo lab, where every command on these pages was run.
Point it at the raft leader: only the leader changes ownership, and a follower refuses with node is not the leader. api GET /api/v1/cluster says whether you are on it ("is_leader":true), and the witness’s status names it.
Who may do what
| Role | May |
|---|---|
| Viewer | Read everything but secrets’ values, which nobody can |
| Operator | Also dry runs, failovers, switchovers, restarts, return to standby |
| EmergencyOperator | Also the forced operations: manual fence, force release, attested failover, accepted data loss, marking a site being prepared |
| Administrator | Everything, including configuration, controllers, secrets and tokens |
People get a role from your identity provider’s groups, through -oidc-role-map. Each orchestrator’s administrator token is break-glass.