Upgrading Steward consoles across clients¶
Tenantry already polls every client's Steward console (GET /api/fleet/v1/status, see Clients and consoles),
and the status carries the console's version (instance.version). Fleet upgrades build on that: you set the target
version, Tenantry shows which consoles are behind, generates the exact update commands for each host, and follows the next
polls to see the version move.
Tenantry never runs anything on a client host and never touches a Docker socket. It only produces commands and scripts; a technician runs them on the console host, like any other generated Steward script.
1. Target version (Settings > Steward versions)¶
/settings/versions, administrators only (technicians read the target on the Consoles page, they cannot change it). Every
change is audited as upgrades.settings (detail: what changed).
| Field | Meaning |
|---|---|
| Recommended image tag | The Steward image tag consoles should run, for example 1.4.0. Empty: versions are not compared. latest and any tag with whitespace or shell characters are refused (the console's own upgrade.sh refuses latest too). |
| Image repository (optional) | For example registry.example.com/steward, without a tag. Empty: each host keeps the repository found in its own .env and only the tag changes. |
| Alert after (days) | Delay after which a console still behind its planned upgrade raises an alert (default 7, 1 to 365). |
2. Drift: current, behind, unknown¶
The Consoles page (and the consoles of a client) shows, next to the reported version, a badge:
| Badge | Rule |
|---|---|
| Up to date | The console version is equal to or above the target. |
| Behind | The console version is older than the target. |
| Unknown | No successful poll yet (no version), or the versions cannot be ordered. |
Ordering follows semantic versioning (1.10.0 is above 1.9.0, a leading v is ignored, 1.4.0-rc.1 is below 1.4.0, build
metadata is ignored). When the console or the target is not a version number (a branch or build tag such as main-abc123,
or 1.4), Tenantry does not guess: an identical tag is up to date, anything else is unknown. A console that failed its last
poll keeps the version of its last successful poll.
The page shows the target, the global counts (behind, up to date, unknown), the number of consoles behind per client (each a link
to the client), and a filter Only consoles behind. The dashboard has a card N consoles to update linking to
/consoles?drift=behind.
3. Upgrade pack¶
On the Consoles page, tick the consoles (or Select all behind) and click Generate upgrade pack, or use Upgrade pack for all consoles behind. Nothing is generated before you confirm. The pack contains, grouped by client:
- the copy-paste view in the dialog: for each console, its version change and the commands to run on its host:
cd /path/to/the/console/folder # the folder with .env, compose.yml and upgrade.sh
./upgrade.sh 'registry.example.com/steward:1.4.0'
Without an image repository in the settings, the image argument keeps the repository already present in the host's .env;
- a zip (one folder per client): README.md (every console of the client with its commands), upgrade-<host>.sh (one wrapper per
console: ./upgrade-<host>.sh /path/to/the/console/folder) and upgrade.sh (the generic console update script, generated by the same
generator as the Steward bundle, used by the wrapper if the host no longer has its own).
The update itself is the console's own upgrade.sh: backup first, pin the new image in .env (keeping .env.before-upgrade-<date>),
pull or docker load, recreate, wait for healthy, with the rollback instructions printed on failure. See
Deploying Steward.
Consoles already at the target are left out of the pack; a pack for nothing but up-to-date consoles answers 409 nothing_to_upgrade.
"Upgrade planned" events¶
Generating a pack records, for each console, an upgrade planned plan (and the audit entry upgrade.planned, filed under the
client, detail id=…; from=…; to=…). The plan is idempotent: generating the zip after the copy-paste view, or regenerating the pack,
does not record it again and does not restart the delay of the alert. A new target replaces the open plan of a console.
4. Progress and the overdue alert¶
After each polling round Tenantry closes the plans whose console now reports the target version (or above): the console becomes upgraded. The Upgrade progress panel (Consoles page, and per client) shows the counters planned, upgraded, pending and overdue, and one line per plan (version change, when planned, state). Closed plans are forgotten after 30 days, plans of removed consoles immediately.
A pending plan older than the configured delay raises the alert console_upgrade_overdue (warning, key console:<id>:upgrade,
on the client of the console) through the usual channels (Alerts). It resolves when the console reaches the target. It
is only evaluated while the console answers its polls: an unreachable console keeps the alert state it had (and fires
console_down).
5. Scoping and audit¶
| Action | Who |
|---|---|
| Read the target, the plans and the progress | Every user; technicians only see the plans of their clients |
| Change the target version, repository, delay | Administrators |
| Generate a pack | Administrators; technicians with the operator level on the console's client (a viewer gets 403 insufficient_level; a console outside the scope answers 404 like an unknown id) |
| "All behind" | Same rule: only the consoles the caller may operate are included |
Routes: GET /api/upgrades, PUT /api/upgrades/settings, POST /api/upgrades/pack (body { consoleIds: [...] } or
{ allBehind: true }, format: "json" | "zip"). Audit actions: upgrades.settings, upgrade.planned, auth.forbidden for refusals.
Plans and settings are stored in upgrades.json of the data volume.
Troubleshooting¶
| Symptom | Cause |
|---|---|
| No badge, "No target version is set" | Set the target in Settings > Steward versions (administrator). |
| A console is always unknown | It never answered a poll, or its version is a branch/build tag that is not the target. Use the exact tag as target, or a semver tag. |
| A plan stays pending after the upgrade | The next poll has not run yet (60 s), the console is unreachable, or it reports an older version (the update failed: see its upgrade.sh output and rollback instructions). |
409 target_not_set when generating |
No target configured. |