Daniel Maksymilian Syrnicki
8b07b57242
After the 26.19 → 26.20 self-update the bench had furtka-welcome.timer linked + enabled but inactive (dead): enable only arms a timer for the next boot. The link also pointed at /opt/furtka/versions/<ver>/ instead of /opt/furtka/current/, unlike the installer-created links, so it would have stopped following later updates and rollbacks. Link via current, enable, start. Adds docs/hardware-test-2026-08.md (timeline, five findings with root causes / fixes / validation status, what worked, open items) and a 'Real hardware' section in iso/README.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MScAinbyMdeNc7H2BZdnnG
75 lines
6.8 KiB
Markdown
75 lines
6.8 KiB
Markdown
# Hardware test report — "junk bench", 2026-08-24/26
|
||
|
||
First end-to-end run of Furtka on real hardware that was not a Proxmox VM
|
||
and not the Medion box: a bare old-laptop mainboard lying on the desk, a
|
||
1 TB Crucial BX SATA SSD that had previously lived in a Proxmox VE host, a
|
||
USB hub for keyboard + boot stick, LAN cable, HDMI monitor. Tester: Daniel.
|
||
Release under test: **26.19-alpha**, then self-updated to **26.20-alpha**.
|
||
|
||
Why it mattered: every bug below is invisible on the smoke VM. A VM has a
|
||
fresh empty disk, a fast virtual NIC, UEFI firmware and no USB stick to pull.
|
||
|
||
## Timeline
|
||
|
||
| When | What happened | Outcome |
|
||
|---|---|---|
|
||
| 08-24 | ISO flashed with Fedora Media Writer, bench booted, browser opened `proski.local:5000` | "Server Not Found" — **typo** (`proski` vs `proksi`). Bench powered off. |
|
||
| 08-25 | `proksi.local:5000` — wizard came up (BIOS/syslinux boot path, stock Arch splash) | OK |
|
||
| 08-25 13:07 | Install run 1 | **Failed at 12 %**: `Partition(s) 2, 3 on /dev/sda have been written, but we have been unable to inform the kernel of the change … in use` |
|
||
| 08-25 | Root shell over SSH (`passwd` + `ssh-copy-id` on the live ISO): SSD still carried a Proxmox VE `pve` volume group (swap, root, thin pool); udev had activated it, six device-mapper nodes sat on `sda3` | Cleared by hand: `dmsetup remove` ×6, `wipefs -a /dev/sda`, `blockdev --rereadpt` |
|
||
| 08-25 13:16 | Install run 2 | `Installation completed without any errors`; BIOS detected → `Grub`, MBR table, `sda1` 1 G vfat `/boot`, `sda2` ext4 `/` |
|
||
| 08-25 | "Reboot now" clicked | Box **booted the USB stick again** (BIOS order). Stick pulled, rebooted → `furtka.local` = 192.168.178.142, Caddy answering |
|
||
| 08-25 16:22 | First boot of the installed system | `furtka-catalog-sync.service` failed: `Network is unreachable`. `network-online.target` had fired on the IPv6 SLAAC address ~1 s before DHCPv4 landed. State file stuck on `checking`, apps page empty; next timer run would have been 18:26 |
|
||
| 08-25 ~17:00 | Catalog synced via the "Sync apps catalog" button | Catalog 26.13-alpha, 7 apps listed |
|
||
| 08-25 17:47 | 26.20-alpha released with the fixes below | Box reports `Update available: 26.19-alpha → 26.20-alpha` |
|
||
| 08-26 09:34 | Self-update to 26.20-alpha from `/settings` | `current → 26.20-alpha`, no failed units. **New finding:** `furtka-welcome.timer` linked + enabled by the updater but `inactive (dead)` — `enable` alone only arms it for the next boot |
|
||
|
||
## Findings and fixes
|
||
|
||
| # | Finding | Root cause | Fix | Shipped in | Validated on hardware |
|
||
|---|---|---|---|---|---|
|
||
| 1 | Console banner showed only `proksi.local` / `furtka.local`, never the IP fallback | Banner script was a oneshot after `network-online.target` and dropped the IP line when no address existed yet. On the live ISO releng's root autologin on tty1 also made `agetty --reload` a no-op | `furtka-issue.timer` (ISO) / `furtka-welcome.timer` (installed) re-run the script every 5 s; fallback line is always present ("no IP address yet — check the cable / DHCP" until one arrives); ISO no longer auto-logs root in on tty1 | 26.20-alpha (`0e05bef`) | Installed side: pending reboot or next update. ISO side: pending reinstall from the 26.20 ISO |
|
||
| 2 | Install failed at partitioning on a disk with a previous OS | Live ISO ships lvm2/mdadm/dmraid/cryptsetup; udev auto-activates whatever the disk carries; parted cannot re-read the partition table while dm nodes hold the partitions | `webinstaller/diskprep.py`: `release_disk()` runs before archinstall — swapoff/umount leaves-first, `dmsetup remove` / `mdadm --stop`, `wipefs -a`, `blockdev --rereadpt`; steps logged at the top of the install log; progress page names this failure explicitly | 26.20-alpha (`5ddda53`) | Pending reinstall from the 26.20 ISO (the disk now carries Furtka itself, which exercises the same path minus LVM) |
|
||
| 3 | "Remove the USB stick" was easy to miss — the box booted the installer again | Hint lived on the *rebooting* page and in a `confirm()` popup, i.e. after the click, with the machine restarting 3 s later | Done state shows a two-step list (pull the stick → restart) above the button; button reads "USB stick is out — restart now" | 26.20-alpha (`60404ae`) | Pending reinstall |
|
||
| 4 | No apps after first boot | Boot-time catalog sync raced DHCPv4 (IPv6 made `network-online` fire early), no retry, timer jitter pushed the next attempt 2 h out, state file stuck on `checking` | `furtka-catalog-sync.service` retries on failure (20 s, up to 8× in 15 min); failures recorded as stage `error`; apps page shows "last sync failed: …" | 26.20-alpha (`32ba522`) | Retry config confirmed live on the box after the update; a cold boot with the new unit is still pending |
|
||
| 5 | New timer dead after self-update | `_link_new_units()` linked + enabled but never started the timer; also linked from the versioned dir instead of `current` | Link via `current`, enable, start | Unreleased (this branch) | — |
|
||
|
||
Not a bug, but worth knowing: on this board the BIOS boots the USB stick
|
||
before the SSD, so every reinstall needs the stick pulled at the reboot
|
||
prompt. The stock Arch splash on the BIOS/syslinux boot path is unbranded
|
||
(only menu labels are) — cosmetic.
|
||
|
||
## What worked first time
|
||
|
||
- Fedora Media Writer → hybrid ISO boots on BIOS firmware.
|
||
- mDNS: `proksi.local` (live ISO) and `furtka.local` (installed) resolved
|
||
from a Fedora client once spelled correctly.
|
||
- BIOS detection → `Grub` + MBR layout; the installed system booted from
|
||
the SSD without intervention.
|
||
- Login, `/settings`, catalog sync via the UI button, `furtka update --check`,
|
||
self-update 26.19 → 26.20 with symlink flip and clean unit state.
|
||
- Time sync, Docker 29.7.2, Caddy on :80, sshd with key auth for the wizard
|
||
user (`wheel` + `docker` groups as configured).
|
||
|
||
## Still open from this run
|
||
|
||
- Reinstall from the 26.20-alpha ISO on the bench to validate findings 1–3
|
||
on the ISO side (the installed side of #1 validates on the next reboot).
|
||
- Cold boot with the retrying catalog-sync unit (#4) on a dual-stack LAN.
|
||
- Rebrand the syslinux boot splash.
|
||
- `smoke-latest.yml` is `workflow_dispatch` only and has not run since
|
||
April; the smoke VM would not have caught any of the above anyway, but it
|
||
is currently not running at all.
|
||
|
||
## Reproducing the bench conditions in a VM
|
||
|
||
Most of this can be provoked on Proxmox if you want a regression check
|
||
without the hardware:
|
||
|
||
- Finding 2: attach a disk that already carries an LVM install (e.g. a
|
||
spare Proxmox VE or Ubuntu LVM disk image) before booting the ISO.
|
||
- Finding 3: leave the ISO attached as CD-ROM with boot order CD-first.
|
||
- Finding 4: give the VM's bridge IPv6 RA before DHCPv4 — harder; the
|
||
retry logic is unit-tested instead.
|
||
- Finding 1: use SeaBIOS instead of OVMF (the README notes SeaBIOS fails on
|
||
`ldlinux.c32` from this ISO, so this one really needs hardware).
|