diff --git a/CHANGELOG.md b/CHANGELOG.md index b576eb6..5b7f08a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,18 @@ This project uses calendar versioning: `YY.N-stage` (e.g. `26.0-alpha` = 2026, r ## [Unreleased] +### Fixed + +- **Self-update starts newly added timers immediately and links units + through `current`.** `_link_new_units()` enabled a timer a release + introduced but never started it, so after the 26.19 → 26.20 update the + bench sat with `furtka-welcome.timer` `inactive (dead)` — and the console + banner without its IP line — until the next reboot. It also linked the + unit from `/opt/furtka/versions//…` instead of `/opt/furtka/current/…`, + so the link would have stopped following later updates and rollbacks + (installer-created links use `current`). Now: link via `current`, enable, + start. + ## [26.20-alpha] - 2026-08-25 First release shaped by a real-hardware install (old laptop mainboard, BIOS boot, diff --git a/README.md b/README.md index 3059244..9f6aee0 100644 --- a/README.md +++ b/README.md @@ -102,6 +102,7 @@ None of these nail the "your dad can set this up" experience. The installer wiza - [x] Installer webapp prototype — device reader + form → JSON (Robert) - [x] Arch running on Proxmox, custom image builds in progress (Robert) - [x] Competitor analysis — see [docs/competitors.md](docs/competitors.md) +- [x] First bare-metal install (old laptop board, BIOS, SSD with a previous Proxmox VE) — 4 bugs found and shipped in 26.20-alpha, report in [docs/hardware-test-2026-08.md](docs/hardware-test-2026-08.md) - [x] Wizard flow spec — see [docs/wizard-flow.md](docs/wizard-flow.md) - [x] Release process + CI — CalVer tags, conventional commits, Forgejo Actions (ruff, pytest, JSON, link checks), `26.0-alpha` tagged - [x] Forgejo runner live on Proxmox VM (`forge-runner-01`, Ubuntu 24.04) — docker-outside-of-docker with host-mode jobs for ISO builds, setup captured in [docs/runner-setup.md](docs/runner-setup.md) + [ops/forgejo-runner/](ops/forgejo-runner/) diff --git a/docs/hardware-test-2026-08.md b/docs/hardware-test-2026-08.md new file mode 100644 index 0000000..ead34c0 --- /dev/null +++ b/docs/hardware-test-2026-08.md @@ -0,0 +1,75 @@ +# Hardware test report — "junk bench", 2026-08-24/26 + +First end-to-end run of Furtka on real hardware that was not a Proxmox VM +and not the Medion box: a bare old-laptop mainboard lying on the desk, a +1 TB Crucial BX SATA SSD that had previously lived in a Proxmox VE host, a +USB hub for keyboard + boot stick, LAN cable, HDMI monitor. Tester: Daniel. +Release under test: **26.19-alpha**, then self-updated to **26.20-alpha**. + +Why it mattered: every bug below is invisible on the smoke VM. A VM has a +fresh empty disk, a fast virtual NIC, UEFI firmware and no USB stick to pull. + +## Timeline + +| When | What happened | Outcome | +|---|---|---| +| 08-24 | ISO flashed with Fedora Media Writer, bench booted, browser opened `proski.local:5000` | "Server Not Found" — **typo** (`proski` vs `proksi`). Bench powered off. | +| 08-25 | `proksi.local:5000` — wizard came up (BIOS/syslinux boot path, stock Arch splash) | OK | +| 08-25 13:07 | Install run 1 | **Failed at 12 %**: `Partition(s) 2, 3 on /dev/sda have been written, but we have been unable to inform the kernel of the change … in use` | +| 08-25 | Root shell over SSH (`passwd` + `ssh-copy-id` on the live ISO): SSD still carried a Proxmox VE `pve` volume group (swap, root, thin pool); udev had activated it, six device-mapper nodes sat on `sda3` | Cleared by hand: `dmsetup remove` ×6, `wipefs -a /dev/sda`, `blockdev --rereadpt` | +| 08-25 13:16 | Install run 2 | `Installation completed without any errors`; BIOS detected → `Grub`, MBR table, `sda1` 1 G vfat `/boot`, `sda2` ext4 `/` | +| 08-25 | "Reboot now" clicked | Box **booted the USB stick again** (BIOS order). Stick pulled, rebooted → `furtka.local` = 192.168.178.142, Caddy answering | +| 08-25 16:22 | First boot of the installed system | `furtka-catalog-sync.service` failed: `Network is unreachable`. `network-online.target` had fired on the IPv6 SLAAC address ~1 s before DHCPv4 landed. State file stuck on `checking`, apps page empty; next timer run would have been 18:26 | +| 08-25 ~17:00 | Catalog synced via the "Sync apps catalog" button | Catalog 26.13-alpha, 7 apps listed | +| 08-25 17:47 | 26.20-alpha released with the fixes below | Box reports `Update available: 26.19-alpha → 26.20-alpha` | +| 08-26 09:34 | Self-update to 26.20-alpha from `/settings` | `current → 26.20-alpha`, no failed units. **New finding:** `furtka-welcome.timer` linked + enabled by the updater but `inactive (dead)` — `enable` alone only arms it for the next boot | + +## Findings and fixes + +| # | Finding | Root cause | Fix | Shipped in | Validated on hardware | +|---|---|---|---|---|---| +| 1 | Console banner showed only `proksi.local` / `furtka.local`, never the IP fallback | Banner script was a oneshot after `network-online.target` and dropped the IP line when no address existed yet. On the live ISO releng's root autologin on tty1 also made `agetty --reload` a no-op | `furtka-issue.timer` (ISO) / `furtka-welcome.timer` (installed) re-run the script every 5 s; fallback line is always present ("no IP address yet — check the cable / DHCP" until one arrives); ISO no longer auto-logs root in on tty1 | 26.20-alpha (`0e05bef`) | Installed side: pending reboot or next update. ISO side: pending reinstall from the 26.20 ISO | +| 2 | Install failed at partitioning on a disk with a previous OS | Live ISO ships lvm2/mdadm/dmraid/cryptsetup; udev auto-activates whatever the disk carries; parted cannot re-read the partition table while dm nodes hold the partitions | `webinstaller/diskprep.py`: `release_disk()` runs before archinstall — swapoff/umount leaves-first, `dmsetup remove` / `mdadm --stop`, `wipefs -a`, `blockdev --rereadpt`; steps logged at the top of the install log; progress page names this failure explicitly | 26.20-alpha (`5ddda53`) | Pending reinstall from the 26.20 ISO (the disk now carries Furtka itself, which exercises the same path minus LVM) | +| 3 | "Remove the USB stick" was easy to miss — the box booted the installer again | Hint lived on the *rebooting* page and in a `confirm()` popup, i.e. after the click, with the machine restarting 3 s later | Done state shows a two-step list (pull the stick → restart) above the button; button reads "USB stick is out — restart now" | 26.20-alpha (`60404ae`) | Pending reinstall | +| 4 | No apps after first boot | Boot-time catalog sync raced DHCPv4 (IPv6 made `network-online` fire early), no retry, timer jitter pushed the next attempt 2 h out, state file stuck on `checking` | `furtka-catalog-sync.service` retries on failure (20 s, up to 8× in 15 min); failures recorded as stage `error`; apps page shows "last sync failed: …" | 26.20-alpha (`32ba522`) | Retry config confirmed live on the box after the update; a cold boot with the new unit is still pending | +| 5 | New timer dead after self-update | `_link_new_units()` linked + enabled but never started the timer; also linked from the versioned dir instead of `current` | Link via `current`, enable, start | Unreleased (this branch) | — | + +Not a bug, but worth knowing: on this board the BIOS boots the USB stick +before the SSD, so every reinstall needs the stick pulled at the reboot +prompt. The stock Arch splash on the BIOS/syslinux boot path is unbranded +(only menu labels are) — cosmetic. + +## What worked first time + +- Fedora Media Writer → hybrid ISO boots on BIOS firmware. +- mDNS: `proksi.local` (live ISO) and `furtka.local` (installed) resolved + from a Fedora client once spelled correctly. +- BIOS detection → `Grub` + MBR layout; the installed system booted from + the SSD without intervention. +- Login, `/settings`, catalog sync via the UI button, `furtka update --check`, + self-update 26.19 → 26.20 with symlink flip and clean unit state. +- Time sync, Docker 29.7.2, Caddy on :80, sshd with key auth for the wizard + user (`wheel` + `docker` groups as configured). + +## Still open from this run + +- Reinstall from the 26.20-alpha ISO on the bench to validate findings 1–3 + on the ISO side (the installed side of #1 validates on the next reboot). +- Cold boot with the retrying catalog-sync unit (#4) on a dual-stack LAN. +- Rebrand the syslinux boot splash. +- `smoke-latest.yml` is `workflow_dispatch` only and has not run since + April; the smoke VM would not have caught any of the above anyway, but it + is currently not running at all. + +## Reproducing the bench conditions in a VM + +Most of this can be provoked on Proxmox if you want a regression check +without the hardware: + +- Finding 2: attach a disk that already carries an LVM install (e.g. a + spare Proxmox VE or Ubuntu LVM disk image) before booting the ISO. +- Finding 3: leave the ISO attached as CD-ROM with boot order CD-first. +- Finding 4: give the VM's bridge IPv6 RA before DHCPv4 — harder; the + retry logic is unit-tested instead. +- Finding 1: use SeaBIOS instead of OVMF (the README notes SeaBIOS fails on + `ldlinux.c32` from this ISO, so this one really needs hardware). diff --git a/furtka/updater.py b/furtka/updater.py index 77a14c0..96a3073 100644 --- a/furtka/updater.py +++ b/furtka/updater.py @@ -226,12 +226,17 @@ def _link_new_units(unit_dir: Path) -> list[str]: """`systemctl link` any unit file in unit_dir that isn't already symlinked into /etc/systemd/system/. Returns the list of newly-linked unit names. - Newly-linked `.timer` units are additionally `systemctl enable`d so that - a self-update introducing a timer (e.g. 26.5 → 26.6 adding - furtka-catalog-sync.timer) activates it automatically — the installer's - enable list only applies to fresh installs. A linked-but-disabled timer - never fires on its own, so without this step catalog sync would never - happen on upgraded boxes. + Newly-linked `.timer` units are additionally `systemctl enable`d *and + started* so that a self-update introducing a timer (e.g. 26.5 → 26.6 + adding furtka-catalog-sync.timer, 26.19 → 26.20 adding + furtka-welcome.timer) activates it right away — the installer's enable + list only applies to fresh installs, and `enable` alone only takes + effect at the next boot (the bench sat on a dead welcome timer after the + 26.20 update). + + Units are linked through the `/opt/furtka/current` symlink, not the + versioned directory that `unit_dir` scans, so the link keeps following + later updates and rollbacks the same way the installer-created links do. """ if not unit_dir.is_dir(): return [] @@ -242,9 +247,10 @@ def _link_new_units(unit_dir: Path) -> list[str]: target = _SYSTEMD_DIR / unit_file.name if target.exists() or target.is_symlink(): continue - _run(["systemctl", "link", str(unit_file)]) + _run(["systemctl", "link", str(current_symlink() / "assets" / "systemd" / unit_file.name)]) if unit_file.suffix == ".timer": _run(["systemctl", "enable", unit_file.name]) + _run(["systemctl", "start", unit_file.name]) linked.append(unit_file.name) return linked diff --git a/iso/README.md b/iso/README.md index 8dce49b..e5cbb3d 100644 --- a/iso/README.md +++ b/iso/README.md @@ -51,6 +51,27 @@ mDNS is wired: `avahi-daemon` + `nss-mdns` come from `packages.extra`, the live 5. Find its IP in Proxmox's VM summary (or your router's DHCP table) 6. Open `http://:5000` — the existing 3-screen wizard should be there +## Real hardware (what differs from the VM flow) + +Learned on the first bare-metal run, see [docs/hardware-test-2026-08.md](../docs/hardware-test-2026-08.md): + +- **BIOS boxes work.** The hybrid ISO boots via syslinux on legacy firmware + (you'll see the stock Arch splash — only the menu labels are branded), and + the installer detects the missing `/sys/firmware/efi` and picks GRUB + MBR. +- **Pull the USB stick before "restart now".** Most BIOSes boot USB before + the disk; the done page says so above the button since 26.20-alpha. +- **Disks with a previous OS are fine since 26.20-alpha.** The live ISO + auto-activates old LVM/RAID/swap; `webinstaller/diskprep.py` releases the + disk before archinstall partitions it. On 26.19 and earlier you'd hit + `unable to inform the kernel of the change … in use` at 12 % — clear it + from the tty1 root shell with `dmsetup remove ` for every dm node + shown by `lsblk`, then `wipefs -a /dev/sdX` and retry. +- **Slow NICs.** The console banner's IP fallback line is timer-driven + since 26.20-alpha; on 26.19 it only appeared if DHCP beat getty. +- **Shell on the live ISO**: since 26.20-alpha tty1 stays at the login + prompt (banner refreshes live); log in as `root`, no password. sshd runs; + set a root password with `passwd` to reach it over the network. + ## What you see after install + reboot Once `archinstall` finishes and you click **Reboot now**, the VM comes up into the installed system. No more port `:5000` — the wizard ISO is gone. Instead: diff --git a/tests/test_updater.py b/tests/test_updater.py index acbaac1..d78120d 100644 --- a/tests/test_updater.py +++ b/tests/test_updater.py @@ -316,14 +316,17 @@ def test_link_new_units_only_links_missing(updater, tmp_path, monkeypatch): linked = updater._link_new_units(unit_dir) assert linked == ["furtka-bar.timer"] - # Two calls for the newly-linked timer: systemctl link + systemctl enable. - # The already-linked service is untouched. Timers need the follow-up - # `enable` so self-updates that introduce new timers don't leave them - # dormant — fresh installs get their enable via the webinstaller. - assert len(seen) == 2 + # Three calls for the newly-linked timer: link + enable + start. The + # already-linked service is untouched. `enable` alone only arms the + # timer for the next boot; `start` makes it fire on the running box + # (26.20 left furtka-welcome.timer dead until reboot without it). + assert len(seen) == 3 assert seen[0][:2] == ["systemctl", "link"] - assert seen[0][2].endswith("furtka-bar.timer") + # Linked via the `current` symlink, not the versioned dir being scanned, + # so the link survives the next symlink flip / rollback. + assert seen[0][2] == str(updater.current_symlink() / "assets" / "systemd" / "furtka-bar.timer") assert seen[1] == ["systemctl", "enable", "furtka-bar.timer"] + assert seen[2] == ["systemctl", "start", "furtka-bar.timer"] def test_extract_tarball_uses_data_filter_when_available(tmp_path, updater, monkeypatch):