# Hardware test report — "junk bench", 2026-08-24/26

First end-to-end run of Furtka on real hardware that was not a Proxmox VM
and not the Medion box: a bare old-laptop mainboard lying on the desk, a
1 TB Crucial BX SATA SSD that had previously lived in a Proxmox VE host, a
USB hub for keyboard + boot stick, LAN cable, HDMI monitor. Tester: Daniel.
Release under test: **26.19-alpha**, then self-updated to **26.20-alpha**.

Why it mattered: every bug below is invisible on the smoke VM. A VM has a
fresh empty disk, a fast virtual NIC, UEFI firmware and no USB stick to pull.

## Timeline

| When | What happened | Outcome |
|---|---|---|
| 08-24 | ISO flashed with Fedora Media Writer, bench booted, browser opened `proski.local:5000` | "Server Not Found" — **typo** (`proski` vs `proksi`). Bench powered off. |
| 08-25 | `proksi.local:5000` — wizard came up (BIOS/syslinux boot path, stock Arch splash) | OK |
| 08-25 13:07 | Install run 1 | **Failed at 12 %**: `Partition(s) 2, 3 on /dev/sda have been written, but we have been unable to inform the kernel of the change … in use` |
| 08-25 | Root shell over SSH (`passwd` + `ssh-copy-id` on the live ISO): SSD still carried a Proxmox VE `pve` volume group (swap, root, thin pool); udev had activated it, six device-mapper nodes sat on `sda3` | Cleared by hand: `dmsetup remove` ×6, `wipefs -a /dev/sda`, `blockdev --rereadpt` |
| 08-25 13:16 | Install run 2 | `Installation completed without any errors`; BIOS detected → `Grub`, MBR table, `sda1` 1 G vfat `/boot`, `sda2` ext4 `/` |
| 08-25 | "Reboot now" clicked | Box **booted the USB stick again** (BIOS order). Stick pulled, rebooted → `furtka.local` = 192.168.178.142, Caddy answering |
| 08-25 16:22 | First boot of the installed system | `furtka-catalog-sync.service` failed: `Network is unreachable`. `network-online.target` had fired on the IPv6 SLAAC address ~1 s before DHCPv4 landed. State file stuck on `checking`, apps page empty; next timer run would have been 18:26 |
| 08-25 ~17:00 | Catalog synced via the "Sync apps catalog" button | Catalog 26.13-alpha, 7 apps listed |
| 08-25 17:47 | 26.20-alpha released with the fixes below | Box reports `Update available: 26.19-alpha → 26.20-alpha` |
| 08-26 09:34 | Self-update to 26.20-alpha from `/settings` | `current → 26.20-alpha`, no failed units. **New finding:** `furtka-welcome.timer` linked + enabled by the updater but `inactive (dead)` — `enable` alone only arms it for the next boot |

## Findings and fixes

| # | Finding | Root cause | Fix | Shipped in | Validated on hardware |
|---|---|---|---|---|---|
| 1 | Console banner showed only `proksi.local` / `furtka.local`, never the IP fallback | Banner script was a oneshot after `network-online.target` and dropped the IP line when no address existed yet. On the live ISO releng's root autologin on tty1 also made `agetty --reload` a no-op | `furtka-issue.timer` (ISO) / `furtka-welcome.timer` (installed) re-run the script every 5 s; fallback line is always present ("no IP address yet — check the cable / DHCP" until one arrives); ISO no longer auto-logs root in on tty1 | 26.20-alpha (`0e05bef`) | Installed side: pending reboot or next update. ISO side: pending reinstall from the 26.20 ISO |
| 2 | Install failed at partitioning on a disk with a previous OS | Live ISO ships lvm2/mdadm/dmraid/cryptsetup; udev auto-activates whatever the disk carries; parted cannot re-read the partition table while dm nodes hold the partitions | `webinstaller/diskprep.py`: `release_disk()` runs before archinstall — swapoff/umount leaves-first, `dmsetup remove` / `mdadm --stop`, `wipefs -a`, `blockdev --rereadpt`; steps logged at the top of the install log; progress page names this failure explicitly | 26.20-alpha (`5ddda53`) | Pending reinstall from the 26.20 ISO (the disk now carries Furtka itself, which exercises the same path minus LVM) |
| 3 | "Remove the USB stick" was easy to miss — the box booted the installer again | Hint lived on the *rebooting* page and in a `confirm()` popup, i.e. after the click, with the machine restarting 3 s later | Done state shows a two-step list (pull the stick → restart) above the button; button reads "USB stick is out — restart now" | 26.20-alpha (`60404ae`) | Pending reinstall |
| 4 | No apps after first boot | Boot-time catalog sync raced DHCPv4 (IPv6 made `network-online` fire early), no retry, timer jitter pushed the next attempt 2 h out, state file stuck on `checking` | `furtka-catalog-sync.service` retries on failure (20 s, up to 8× in 15 min); failures recorded as stage `error`; apps page shows "last sync failed: …" | 26.20-alpha (`32ba522`) | Retry config confirmed live on the box after the update; a cold boot with the new unit is still pending |
| 5 | New timer dead after self-update | `_link_new_units()` linked + enabled but never started the timer; also linked from the versioned dir instead of `current` | Link via `current`, enable, start | Unreleased (this branch) | — |

Not a bug, but worth knowing: on this board the BIOS boots the USB stick
before the SSD, so every reinstall needs the stick pulled at the reboot
prompt. The stock Arch splash on the BIOS/syslinux boot path is unbranded
(only menu labels are) — cosmetic.

## What worked first time

- Fedora Media Writer → hybrid ISO boots on BIOS firmware.
- mDNS: `proksi.local` (live ISO) and `furtka.local` (installed) resolved
  from a Fedora client once spelled correctly.
- BIOS detection → `Grub` + MBR layout; the installed system booted from
  the SSD without intervention.
- Login, `/settings`, catalog sync via the UI button, `furtka update --check`,
  self-update 26.19 → 26.20 with symlink flip and clean unit state.
- Time sync, Docker 29.7.2, Caddy on :80, sshd with key auth for the wizard
  user (`wheel` + `docker` groups as configured).

## Still open from this run

- Reinstall from the 26.20-alpha ISO on the bench to validate findings 1–3
  on the ISO side (the installed side of #1 validates on the next reboot).
- Cold boot with the retrying catalog-sync unit (#4) on a dual-stack LAN.
- Rebrand the syslinux boot splash.
- `smoke-latest.yml` is `workflow_dispatch` only and has not run since
  April; the smoke VM would not have caught any of the above anyway, but it
  is currently not running at all.

## Reproducing the bench conditions in a VM

Most of this can be provoked on Proxmox if you want a regression check
without the hardware:

- Finding 2: attach a disk that already carries an LVM install (e.g. a
  spare Proxmox VE or Ubuntu LVM disk image) before booting the ISO.
- Finding 3: leave the ISO attached as CD-ROM with boot order CD-first.
- Finding 4: give the VM's bridge IPv6 RA before DHCPv4 — harder; the
  retry logic is unit-tested instead.
- Finding 1: use SeaBIOS instead of OVMF (the README notes SeaBIOS fails on
  `ldlinux.c32` from this ISO, so this one really needs hardware).