From b8fdb62b410a808c8b3565e9bb37bfcb4a179185 Mon Sep 17 00:00:00 2001 From: Daniel Maksymilian Syrnicki Date: Thu, 16 Apr 2026 14:10:07 +0200 Subject: [PATCH] =?UTF-8?q?fix(furtka):=20pre-ISO=20audit=20fixes=20?= =?UTF-8?q?=E2=80=94=20chmod,=20Caddyfile=20refresh,=20unit=20linking?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five issues surfaced by the Phase-2 audit before the next ISO rebuild: P1 (real blockers for a fresh install / self-update): 1. chmod +x furtka/assets/bin/furtka-status, furtka-welcome. They were mode 644 in git, so the tarball shipped them non-executable and every ExecStart referencing /opt/furtka/current/assets/bin/furtka-* would have failed on first boot with Permission denied. 2. apply_update now refreshes /etc/caddy/Caddyfile from the new version when the content differs, then reloads caddy. Without this, a release that changes Caddy routes silently stays on the old config. 3. apply_update now systemctl-links any new unit files shipped by the update, not just the five linked at install time. A future release that adds furtka-foo.service would otherwise never appear in /etc/systemd/system/. P2 (hardening, not blockers today): 4. _resource_manager_commands now aborts the install if the tarball's VERSION file is empty — otherwise `mv "$staging" /opt/furtka/versions/` would move the staging dir in as a subdirectory and the symlink target would be invalid. 5. _extract_tarball passes filter='data' to tarfile.extractall on Python 3.12+ to catch symlink-escape / setuid / device-node tricks that the regex path-check can't see. Falls back silently on older interpreters. Plus the CHANGELOG [Unreleased] section got filled in with the whole Phase-1 + Phase-2 + UI-uplevel body so a 26.1-alpha tag cut off main has meaningful release notes. Test additions / updates: - test_refresh_caddyfile_{copies_when_different,noops_if_source_missing} - test_link_new_units_only_links_missing - test_extract_tarball_uses_data_filter_when_available - test_apply_update_happy_path now verifies the Caddyfile gets copied. - test_resource_manager_extracts_to_versioned_slot verifies the empty-VERSION guard is present in the install command. Paths now overridable via FURTKA_CADDYFILE_PATH + FURTKA_SYSTEMD_DIR so tests can pin a tmpdir for these new fs operations. Co-Authored-By: Claude Opus 4.6 (1M context) --- CHANGELOG.md | 18 ++++-- furtka/assets/bin/furtka-status | 0 furtka/assets/bin/furtka-welcome | 0 furtka/updater.py | 47 +++++++++++++- tests/test_updater.py | 100 +++++++++++++++++++++++++++++- tests/test_webinstaller_assets.py | 3 + webinstaller/app.py | 4 ++ 7 files changed, 164 insertions(+), 8 deletions(-) mode change 100644 => 100755 furtka/assets/bin/furtka-status mode change 100644 => 100755 furtka/assets/bin/furtka-welcome diff --git a/CHANGELOG.md b/CHANGELOG.md index 42de94c..def89fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,24 +9,30 @@ This project uses calendar versioning: `YY.N-stage` (e.g. `26.0-alpha` = 2026, r ### Added +- **Furtka self-update** (Phase 2). Tagging a release on main fires `.forgejo/workflows/release.yml`, which packages `furtka/` + `apps/` + a root-level `VERSION` file as `furtka-.tar.gz`, uploads it plus a `.sha256` + `release.json` to the Forgejo releases page, and makes the release available to running boxes. New CLI: `furtka update [--check]` + `furtka rollback`. New endpoints: `POST /api/furtka/update/check` + `/apply` + `GET /api/furtka/update/status`. UI: "Furtka updates" card on `/settings` shows installed vs latest, Update button runs the apply flow detached via `systemd-run`, progress polls `/update-state.json` served by Caddy so the mid-update API restart doesn't interrupt reporting. Atomic `/opt/furtka/current` symlink flip, auto-rollback on health-check failure post-restart, SHA256-verified downloads. +- **Per-app container image updates** (Phase 1). `POST /api/apps//update` runs `docker compose pull`, compares the running container's image digest to the just-pulled local image digest per service, and only restarts containers whose image actually changed. Update button on each installed-app row in `/apps`. Keeps `image: :latest` pins simple — no compose-file mutations. +- **Per-version install layout** on `/opt/furtka/`. Install now extracts the resource-manager payload to `/opt/furtka/versions//` and creates `/opt/furtka/current` as an atomic symlink; updates flip the symlink in place and `systemctl link` every unit from the shipped `assets/systemd/` tree. Runtime JSON (`status.json`, `furtka.json`, `update-state.json`) moved to `/var/lib/furtka/` so self-updates never clobber it. +- **On-box UI uplevel** across three pages sharing one design system (`/style.css` served by Caddy). Redesigned landing page with a "Your apps" tile grid driven by `/api/apps`, a `fileshare` app tile that deep-links to `smb://.local/files`, status tiles, and subtle "Coming next" links to `furtka.org`. `/apps` page renders real app icons inlined from each manifest's `icon.svg` (defensive SVG sanitiser — strips script/on*/javascript: content, 16 KB cap). New `/settings` page with About-this-box, Appearance, Furtka-updates, and Coming-next sections. Persistent top nav (Jakob's Law) on every page. Light-mode support via `prefers-color-scheme`. +- **Webinstaller step 2 (boot drive)** now shows size / type / health chips plus a "Recommended" badge on the auto-selected drive instead of a raw numeric score. +- **Forgejo branch protection on `main`** — no direct pushes except owner-whitelisted, required status checks (`CI / lint*`, `CI / test*`, `CI / validate-json*`), applied via the idempotent `ops/forgejo/apply-branch-protection.sh` script. + - **In-browser app settings**, so users no longer need SSH + `vim` to configure an app before first install. Manifest gains optional `settings` (name/label/description/type/required/default) and `description_long` fields. Installing a bundled app opens a form rendered from the manifest; installed apps grow a "Settings" button that edits merged values (password fields blank = keep current). API: `POST /api/apps/install` now accepts a `settings` object in the JSON body; new `GET`/`POST /api/apps//settings` for inspecting and updating an installed app. Password values never leave the server. - `nano` added to the installer package list so users have a beginner-friendly editor at the console/SSH (was `vim`-only, which `command not found`'d under Arch 4.x because it was actually missing from the package set too). - `openssh` added explicitly to the installer package list and `sshd` added to enabled services. `archinstall: true` in archinstall 4.x did not actually install openssh-server, so the documented recovery path (SSH → edit `.env`) silently failed. +- **Forgejo Actions runner** live on Proxmox VM (`forge-runner-01`, Ubuntu 24.04) with DinD sidecar — CI green end-to-end. Setup scripts in `ops/forgejo-runner/`. +- **Walking-skeleton live ISO** (`iso/build.sh`). Overlays an Arch `releng` profile with Flask + the webinstaller, bakes a systemd unit that auto-starts the wizard on boot, produces a hybrid BIOS/UEFI ISO via `mkarchiso` in a privileged `archlinux:latest` container. Tested booting under OVMF in Proxmox — wizard screens 1–3 respond at `http://:5000`. +- **Public website at [furtka.org](https://furtka.org)** (`website/`). Hugo static site, English + German, served from `/var/www/furtka.org` on `forge-runner-01` via nginx. Upstream openresty proxy handles TLS. Intentionally minimal single-page copy while the project is pre-alpha. Deploy is `./website/deploy.sh` (rsync + remote Hugo build); one-time VM setup in `ops/nginx/setup-vm.sh`. ### Changed +- Every on-box asset (landing page, settings page, style.css, status/welcome scripts, systemd units, Caddyfile) moved from inline Python string constants in `webinstaller/app.py` into real files under `furtka/assets/`. The installer reads them from disk at install time; the self-updater ships them in the release tarball. +- Settings-button label went from "Einstellungen" (prototyping leftover) to "Settings" — rest of the UI chrome is English. - Keyboard layout at the TTY now follows the chosen installer language (`de` → `de`, `pl` → `pl`, `en` → `us`) instead of hardcoding `us`. Previously German users couldn't type `/`, `-`, or `=` at the recovery console. - `fileshare` app: `description_long` + `settings` (SMB_USER, SMB_PASSWORD) for the new settings form. Docker-level healthcheck from `dperson/samba` is disabled in the compose override — it timed out under normal operation and marked a working share "unhealthy" in `docker ps`. - **Project name finalized: Furtka.** Working title "Homebase" retired. Domain `furtka.org` registered via Strato 2026-04-13. - Managed gateway NS hostnames updated from `ns1.homebase.cloud` / `ns2.homebase.cloud` to `ns1.furtka.org` / `ns2.furtka.org`. - Python package renamed from `homebase` → `furtka` in `pyproject.toml`. -### Added - -- **Forgejo Actions runner** live on Proxmox VM (`forge-runner-01`, Ubuntu 24.04) with DinD sidecar — CI green end-to-end. Setup scripts in `ops/forgejo-runner/`. -- **Walking-skeleton live ISO** (`iso/build.sh`). Overlays an Arch `releng` profile with Flask + the webinstaller, bakes a systemd unit that auto-starts the wizard on boot, produces a hybrid BIOS/UEFI ISO via `mkarchiso` in a privileged `archlinux:latest` container. Tested booting under OVMF in Proxmox — wizard screens 1–3 respond at `http://:5000`. -- **Public website at [furtka.org](https://furtka.org)** (`website/`). Hugo static site, English + German, served from `/var/www/furtka.org` on `forge-runner-01` via nginx. Upstream openresty proxy handles TLS. Intentionally minimal single-page copy while the project is pre-alpha. Deploy is `./website/deploy.sh` (rsync + remote Hugo build); one-time VM setup in `ops/nginx/setup-vm.sh`. - ## [26.0-alpha] - 2026-04-13 First tagged snapshot. Pre-alpha — the installer does not yet boot, but the design is locked and the prototype components are shaped. diff --git a/furtka/assets/bin/furtka-status b/furtka/assets/bin/furtka-status old mode 100644 new mode 100755 diff --git a/furtka/assets/bin/furtka-welcome b/furtka/assets/bin/furtka-welcome old mode 100644 new mode 100755 diff --git a/furtka/updater.py b/furtka/updater.py index b2322ad..c78f488 100644 --- a/furtka/updater.py +++ b/furtka/updater.py @@ -45,6 +45,8 @@ FORGEJO_HOST = os.environ.get("FURTKA_FORGEJO_HOST", "forgejo.sourcegate.online" FORGEJO_REPO = os.environ.get("FURTKA_FORGEJO_REPO", "daniel/furtka") _FURTKA_ROOT = Path(os.environ.get("FURTKA_ROOT", "/opt/furtka")) _STATE_DIR = Path(os.environ.get("FURTKA_STATE_DIR", "/var/lib/furtka")) +_CADDYFILE_LIVE = Path(os.environ.get("FURTKA_CADDYFILE_PATH", "/etc/caddy/Caddyfile")) +_SYSTEMD_DIR = Path(os.environ.get("FURTKA_SYSTEMD_DIR", "/etc/systemd/system")) class UpdateError(RuntimeError): @@ -187,13 +189,49 @@ def _extract_tarball(tarball: Path, dest: Path) -> str: for member in tf.getmembers(): if member.name.startswith(("/", "..")) or ".." in Path(member.name).parts: raise UpdateError(f"refusing tarball entry {member.name!r}") - tf.extractall(dest) + # Python 3.12+ grew a stricter default filter; opt into it where + # available to catch symlink-escape / device-node / setuid tricks + # that our regex check can't see. Older Pythons fall back to the + # historical permissive behaviour. + try: + tf.extractall(dest, filter="data") + except TypeError: + tf.extractall(dest) version_file = dest / "VERSION" if not version_file.is_file(): raise UpdateError("tarball has no VERSION file at root") return version_file.read_text().strip() +def _refresh_caddyfile(source: Path) -> bool: + """Copy the shipped Caddyfile to /etc/caddy/ iff it differs. Returns True + if the file changed (so caddy needs more than a bare reload).""" + if not source.is_file(): + return False + if _CADDYFILE_LIVE.is_file() and source.read_bytes() == _CADDYFILE_LIVE.read_bytes(): + return False + _CADDYFILE_LIVE.parent.mkdir(parents=True, exist_ok=True) + shutil.copy(source, _CADDYFILE_LIVE) + return True + + +def _link_new_units(unit_dir: Path) -> list[str]: + """`systemctl link` any unit file in unit_dir that isn't already symlinked + into /etc/systemd/system/. Returns the list of newly-linked unit names.""" + if not unit_dir.is_dir(): + return [] + linked = [] + for unit_file in sorted(unit_dir.iterdir()): + if unit_file.suffix not in (".service", ".timer"): + continue + target = _SYSTEMD_DIR / unit_file.name + if target.exists() or target.is_symlink(): + continue + _run(["systemctl", "link", str(unit_file)]) + linked.append(unit_file.name) + return linked + + def write_state(stage: str, **extra) -> None: state_path().parent.mkdir(parents=True, exist_ok=True) tmp = state_path().with_suffix(".tmp") @@ -296,7 +334,14 @@ def apply_update(tarball: Path, version: str) -> None: write_state("restarting", latest=version) try: + # Copy new Caddyfile into /etc/caddy/ if the release changed routes. + # reload always runs afterwards to flush the file-handle cache so the + # symlink flip takes effect even when Caddyfile itself didn't change. + _refresh_caddyfile(target / "assets" / "Caddyfile") _run(["systemctl", "reload", "caddy"]) + # Pick up any new systemd unit files added by this release. Existing + # linked units don't need relinking — daemon-reload rereads them. + _link_new_units(target / "assets" / "systemd") _run(["systemctl", "daemon-reload"]) _run(["systemctl", "try-restart", "furtka-reconcile.service"]) _run(["systemctl", "restart", "furtka-api.service"]) diff --git a/tests/test_updater.py b/tests/test_updater.py index 03e6ece..11edc2f 100644 --- a/tests/test_updater.py +++ b/tests/test_updater.py @@ -22,6 +22,9 @@ def updater(tmp_path, monkeypatch): monkeypatch.setenv("FURTKA_ROOT", str(tmp_path / "opt_furtka")) monkeypatch.setenv("FURTKA_STATE_DIR", str(tmp_path / "var_lib_furtka")) monkeypatch.setenv("FURTKA_LOCK_PATH", str(tmp_path / "update.lock")) + monkeypatch.setenv("FURTKA_CADDYFILE_PATH", str(tmp_path / "etc_caddy" / "Caddyfile")) + monkeypatch.setenv("FURTKA_SYSTEMD_DIR", str(tmp_path / "etc_systemd_system")) + (tmp_path / "etc_systemd_system").mkdir() # Reload the module so the path constants pick up the env vars. import importlib @@ -107,6 +110,25 @@ def test_write_and_read_state_round_trip(updater): assert "updated_at" in s +def _make_release_tarball(path: Path, version: str, caddyfile_body: str = "# caddy\n"): + """Richer tarball with assets/Caddyfile + assets/systemd/ — enough for + apply_update's post-swap integration (caddy refresh, unit linking).""" + buf = io.BytesIO() + with tarfile.open(fileobj=buf, mode="w:gz") as tf: + for name, content in [ + ("VERSION", f"{version}\n"), + ("furtka/__init__.py", ""), + ("apps/fileshare/manifest.json", "{}"), + ("assets/Caddyfile", caddyfile_body), + ("assets/systemd/furtka-api.service", "[Service]\nExecStart=/bin/true\n"), + ]: + data = content.encode() + info = tarfile.TarInfo(name=name) + info.size = len(data) + tf.addfile(info, io.BytesIO(data)) + path.write_bytes(buf.getvalue()) + + def test_apply_update_happy_path(tmp_path, updater, monkeypatch): # Set up an existing "26.0-alpha" current symlink so apply_update has # something to swap out. @@ -118,7 +140,7 @@ def test_apply_update_happy_path(tmp_path, updater, monkeypatch): current.symlink_to(versions / "26.0-alpha") tar = tmp_path / "t.tar.gz" - _make_tarball(tar, "26.1-alpha") + _make_release_tarball(tar, "26.1-alpha", caddyfile_body="# new caddy config\n") # Stub the shell-out + health check — both succeed. monkeypatch.setattr(updater, "_run", lambda cmd: None) @@ -128,6 +150,8 @@ def test_apply_update_happy_path(tmp_path, updater, monkeypatch): assert current.resolve() == (versions / "26.1-alpha").resolve() assert (versions / "26.1-alpha" / "VERSION").read_text().strip() == "26.1-alpha" + # P1-2: Caddyfile was copied into /etc/caddy/ from the new version. + assert updater._CADDYFILE_LIVE.read_text() == "# new caddy config\n" state = updater.read_state() assert state["stage"] == "done" assert state["version"] == "26.1-alpha" @@ -163,6 +187,80 @@ def test_apply_update_rolls_back_on_health_check_failure(tmp_path, updater, monk assert state["failed_version"] == "26.1-alpha" +def test_refresh_caddyfile_copies_when_different(updater, tmp_path): + # Fresh /etc/caddy/ — source wins. + src = tmp_path / "src" + src.write_text("# new\n") + assert updater._refresh_caddyfile(src) is True + assert updater._CADDYFILE_LIVE.read_text() == "# new\n" + + # Same content — no-op. + assert updater._refresh_caddyfile(src) is False + + +def test_refresh_caddyfile_noops_if_source_missing(updater, tmp_path): + assert updater._refresh_caddyfile(tmp_path / "does-not-exist") is False + + +def test_link_new_units_only_links_missing(updater, tmp_path, monkeypatch): + unit_dir = tmp_path / "assets_systemd" + unit_dir.mkdir() + (unit_dir / "furtka-foo.service").write_text("[Service]\nExecStart=/bin/true\n") + (unit_dir / "furtka-bar.timer").write_text("[Timer]\nOnBootSec=1s\n") + (unit_dir / "ignored.txt").write_text("not a unit") + # Pretend furtka-foo is already linked — it must be skipped. + (updater._SYSTEMD_DIR / "furtka-foo.service").symlink_to("/dev/null") + + seen = [] + monkeypatch.setattr(updater, "_run", lambda cmd: seen.append(cmd)) + + linked = updater._link_new_units(unit_dir) + assert linked == ["furtka-bar.timer"] + # Only one systemctl link call — for the new timer, not the existing service. + assert len(seen) == 1 + assert seen[0][:2] == ["systemctl", "link"] + assert seen[0][2].endswith("furtka-bar.timer") + + +def test_extract_tarball_uses_data_filter_when_available(tmp_path, updater, monkeypatch): + # Confirm we pass filter='data' to extractall on Python 3.12+; fall back + # cleanly on older runtimes. Capture the kwarg via a stub. + calls = [] + real_open = updater.tarfile.open # capture before monkeypatching + + class _Recorder: + def __init__(self, tarball): + self._tb = real_open(tarball, "r:gz") + + def __enter__(self): + return self + + def __exit__(self, *a): + self._tb.close() + + def getmembers(self): + return self._tb.getmembers() + + def extractall(self, *args, **kwargs): + calls.append(("extractall", args, kwargs)) + # Force the TypeError branch when filter is passed, then re-run + # without — matches the older-Python fallback. + if "filter" in kwargs: + raise TypeError("old python") + return self._tb.extractall(*args) + + tar = tmp_path / "t.tar.gz" + _make_release_tarball(tar, "26.9-alpha") + monkeypatch.setattr(updater.tarfile, "open", lambda *a, **kw: _Recorder(tar)) + + dest = tmp_path / "dest" + updater._extract_tarball(tar, dest) + # First call had filter=, second (fallback) didn't. + assert len(calls) == 2 + assert calls[0][2] == {"filter": "data"} + assert calls[1][2] == {} + + def test_apply_update_rejects_version_mismatch(tmp_path, updater, monkeypatch): versions = updater.versions_dir() versions.mkdir(parents=True) diff --git a/tests/test_webinstaller_assets.py b/tests/test_webinstaller_assets.py index dc7dd91..e24604b 100644 --- a/tests/test_webinstaller_assets.py +++ b/tests/test_webinstaller_assets.py @@ -75,6 +75,9 @@ def test_resource_manager_extracts_to_versioned_slot(install_cmds): assert "/opt/furtka/versions" in extract_cmd assert "staging-" in extract_cmd # mktemp -d pattern assert 'cat "$staging/VERSION"' in extract_cmd + # An empty VERSION file must abort the install instead of silently + # moving the staging dir into versions/ as a subdir. + assert '[ -n "$ver" ]' in extract_cmd assert 'ln -sfn "/opt/furtka/versions/$ver" /opt/furtka/current' in extract_cmd diff --git a/webinstaller/app.py b/webinstaller/app.py index 38ab52f..f2d25a5 100644 --- a/webinstaller/app.py +++ b/webinstaller/app.py @@ -255,6 +255,10 @@ def _resource_manager_commands(): "staging=$(mktemp -d /opt/furtka/versions/staging-XXXXXX) && " f'printf %s {payload_b64} | base64 -d | tar -xzf - -C "$staging" && ' 'ver=$(cat "$staging/VERSION") && ' + # Guard against an empty VERSION file: without this, `mv "$staging" + # "/opt/furtka/versions/"` would move the staging dir into versions/ + # as a subdir and the symlink target would be invalid. + '[ -n "$ver" ] || { echo "empty VERSION in payload" >&2; exit 1; } && ' 'mv "$staging" "/opt/furtka/versions/$ver" && ' 'ln -sfn "/opt/furtka/versions/$ver" /opt/furtka/current' )