feat: add mosquitto + zigbee2mqtt dependency pair #3

Open
daniel wants to merge 1 commit from feat/mqtt-dependency-pair into main
15 changed files with 427 additions and 1 deletions

View file

@ -31,4 +31,12 @@ jobs:
sudo apt-get update sudo apt-get update
sudo apt-get install -y --no-install-recommends shellcheck sudo apt-get install -y --no-install-recommends shellcheck
- name: Run shellcheck - name: Run shellcheck
run: shellcheck scripts/*.sh run: |
set -e
shellcheck scripts/*.sh
# App dependency hooks (apps/*/scripts/*.sh) run inside provider
# containers — lint them too. The glob may match nothing, so guard it.
hooks=$(find apps -path 'apps/*/scripts/*.sh' 2>/dev/null || true)
if [ -n "$hooks" ]; then
echo "$hooks" | xargs shellcheck
fi

View file

@ -6,6 +6,46 @@ Versioning: CalVer (`YY.N`) — same scheme as the core Furtka repo.
## [Unreleased] ## [Unreleased]
### Added
- **Mosquitto** (v1.0.0, image `eclipse-mosquitto:2.0`). MQTT broker, the
first *dependency provider* in the catalog. Auth is mandatory
(`allow_anonymous false`); the broker ships no hand-managed accounts.
Instead it carries two hook scripts (`scripts/provision-client.sh`,
`scripts/ensure-client.sh`) that any consumer app references from its
`requires` block — they run inside the mosquitto container, create a
per-consumer `mosquitto_passwd` account, and hand the credentials back to
the consumer. One Docker volume (`data`) for persistence + the password
file + a provider-side password stash so `on_start` can restore an account
after a volume wipe. The `ensure-client.sh` hook prefers the consumer
password that post-26.17 core injects as `FURTKA_CONSUMER_ENV_MQTT_PASS` and
falls back to the stash on 26.17, so it works on both. Publishes 1883 on the
host so consumers in separate compose projects can reach it via
`host.docker.internal`.
- **Zigbee2MQTT** (v1.0.0, image `koenkk/zigbee2mqtt:1.42.0`). First
*dependency consumer*: `requires` mosquitto, so installing it pulls the
broker in automatically and wires up MQTT credentials with no manual config.
MQTT settings are injected via `ZIGBEE2MQTT_CONFIG_*` env from the hook
output; the Zigbee USB coordinator path is a `text` setting
(`ZIGBEE_SERIAL_PORT`) mapped through `devices:`. Frontend on host port 8084
(8080 is taken by it-tools). Needs a physical Zigbee coordinator to fully
start; the MQTT-credential handshake is observable without one.
- These two are the catalog's first real exercise of core 26.17's app-to-app
dependency feature — until now every catalog app was standalone. Building
them surfaced three framework limitations now documented in
[apps/README.md](apps/README.md#gotchas-learned-building-mosquitto--zigbee2mqtt):
`on_start` can't talk back to the consumer, there is no shared app network,
and there is no device/serial setting type.
### Changed
- **Bumped the vendored `scripts/vendor/furtka_manifest.py`** from the
26.10-era copy to core 26.17, so the catalog validator actually understands
and validates the `requires` / `on_install` / `on_start` schema instead of
silently ignoring it.
- **CI now shellchecks app hook scripts** (`apps/*/scripts/*.sh`), not just
the repo-root `scripts/`.
## [26.12-alpha] - 2026-04-28 ## [26.12-alpha] - 2026-04-28
### Added ### Added

View file

@ -118,6 +118,70 @@ Do not use `changeme` (or any value listed in `furtka.installer.PLACEHOLDER_SECR
For non-secret values (usernames, paths), sensible defaults are fine and go straight into `.env` on first install. For non-secret values (usernames, paths), sensible defaults are fine and go straight into `.env` on first install.
## App-to-app dependencies (`requires`)
An app that needs another app at runtime (e.g. `zigbee2mqtt` needs an MQTT
broker) declares it in `manifest.json`. Installing the consumer pulls in the
provider automatically; the UI shows a confirm dialog first.
```json
{
"name": "zigbee2mqtt",
"requires": [
{
"app": "mosquitto",
"on_install": "scripts/provision-client.sh",
"on_start": "scripts/ensure-client.sh"
}
]
}
```
- `app` — the provider's name. Must resolve in installed/catalog/bundled apps.
- `on_install` / `on_start` — optional hook scripts, **paths relative to the
_provider's_ folder** (not the consumer's). See `apps/mosquitto/scripts/` for
the reference pair.
### How hooks run
Both hooks execute **inside the provider's container** via
`docker compose exec -T <provider> sh -s` — so they use whatever tools the
*provider* image ships (mosquitto's hooks call `mosquitto_passwd`). The
reconciler exports `FURTKA_CONSUMER_APP` and `FURTKA_CONSUMER_VERSION` into the
hook's environment so it knows who it's provisioning for.
- **`on_install`** runs once, while the consumer is being installed, after the
provider is up. Its **stdout is merged into the consumer's `.env`**: print
`KEY=VALUE` lines (keys must match `^[A-Z_][A-Z0-9_]*$`) and they become env
vars the consumer's compose file can `${SUBSTITUTE}`. The placeholder-secret
check re-runs over the merged `.env`, so a hook returning `MQTT_PASS=changeme`
is refused just like an unedited `.env.example`.
- **`on_start`** runs on every boot, before the consumer starts. Must be
**idempotent**.
### Gotchas (learned building mosquitto + zigbee2mqtt)
- **`on_start` is read-only toward the consumer.** Its stdout is *discarded*
(unlike `on_install`, which merges its stdout into the consumer's `.env`).
It reads consumer state to reconcile provider state; it doesn't mutate the
consumer. Core that post-dates 26.17 injects the consumer's stored `.env`
into the hook as `FURTKA_CONSUMER_ENV_<KEY>`, so `on_start` can re-establish
provider-side state (e.g. re-create an account after a volume wipe) with the
**same** credentials the consumer already holds. On 26.17 itself that
injection isn't there, so a provider that must survive that gap stashes what
it needs at `on_install` time on its own volume. mosquitto's
`ensure-client.sh` uses `FURTKA_CONSUMER_ENV_MQTT_PASS` when present and
falls back to a stash (`/mosquitto/data/furtka-clients/`) otherwise.
- **No shared app network (yet).** Each app is its own compose project on its
own network, so a consumer can't reach a provider by service name. Publish
the provider's port on the host and hand the consumer
`host.docker.internal:<port>` (the consumer adds
`extra_hosts: ["host.docker.internal:host-gateway"]`).
- **No device/serial setting type.** A `path`-type setting can't point at a
`/dev` node (the validator rejects non-directories and `/dev` is on the
deny-list). Use a plain `text` setting for the device path plus a `devices:`
mapping in compose, and document the hardware requirement.
## `icon.svg` ## `icon.svg`
- 64×64 viewBox, no width/height attributes so the UI can scale it. - 64×64 viewBox, no width/height attributes so the UI can scale it.

View file

@ -0,0 +1,2 @@
# Mosquitto has no user-facing settings. Accounts are created per consumer
# app by the install/start hooks in ./scripts/, not via this file.

View file

@ -0,0 +1,39 @@
# Furtka Mosquitto — MQTT broker (dependency provider).
#
# Provider for the app-to-app dependency feature: consumer apps declare
# `requires: [{app: mosquitto, ...}]` and their hooks (which live in this
# folder under ./scripts/) run INSIDE this container via
# `docker compose exec sh -s` to provision a per-consumer MQTT account.
#
# Networking note: Furtka runs each app as its own compose project on its
# own default network, so consumers can't reach this broker by the
# `mosquitto` service name. We publish 1883 on the host instead, and the
# provisioning hook hands the consumer `mqtt://host.docker.internal:1883`
# (the consumer maps host.docker.internal to the docker host-gateway). A
# shared furtka app network would be the cleaner long-term fix; until then
# the host-port bridge is what works across separate compose projects.
#
# The password_file in mosquitto.conf must exist before the broker starts
# or mosquitto refuses to boot, so the command touches an empty one first
# (zero accounts = nobody can connect, which is the correct secure default
# until a consumer is provisioned). mosquitto then reloads the file on
# SIGHUP, which is how the hooks make a freshly-added account live without
# bouncing the broker.
#
# TODO(image-pin): pin to a digest once verified against the upstream
# registry. `2.0` tracks the latest 2.0.x patch — acceptable for the MVP.
services:
mosquitto:
image: eclipse-mosquitto:2.0
restart: unless-stopped
command: sh -c "touch /mosquitto/data/passwd && exec /usr/sbin/mosquitto -c /mosquitto/config/mosquitto.conf"
ports:
- "1883:1883"
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
- furtka_mosquitto_data:/mosquitto/data
volumes:
furtka_mosquitto_data:
external: true

11
apps/mosquitto/icon.svg Normal file
View file

@ -0,0 +1,11 @@
<svg viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round">
<circle cx="32" cy="32" r="6" fill="currentColor" stroke="none"/>
<circle cx="12" cy="14" r="5"/>
<circle cx="52" cy="14" r="5"/>
<circle cx="12" cy="50" r="5"/>
<circle cx="52" cy="50" r="5"/>
<line x1="16" y1="17" x2="28" y2="29"/>
<line x1="48" y1="17" x2="36" y2="29"/>
<line x1="16" y1="47" x2="28" y2="35"/>
<line x1="48" y1="47" x2="36" y2="35"/>
</svg>

After

Width:  |  Height:  |  Size: 538 B

View file

@ -0,0 +1,10 @@
{
"name": "mosquitto",
"display_name": "MQTT-Broker (Mosquitto)",
"version": "1.0.0",
"description": "Lightweight MQTT message broker — the backbone other smart-home apps talk through.",
"description_long": "Eclipse Mosquitto ist ein schlanker MQTT-Broker: die Vermittlungsstelle, über die Smart-Home-Apps (z.B. Zigbee2MQTT) lokal Nachrichten austauschen. Läuft komplett auf dieser Maschine, ohne Cloud. Anonyme Zugriffe sind aus — jede App, die den Broker voraussetzt, bekommt beim Installieren automatisch ein eigenes Konto. Der Broker lauscht auf Port 1883 im LAN.",
"volumes": ["data"],
"ports": [1883],
"icon": "icon.svg"
}

View file

@ -0,0 +1,15 @@
# Furtka Mosquitto broker config.
#
# Auth is mandatory: anonymous publish/subscribe is off and every client
# must present credentials. Accounts are NOT managed by hand here — each
# consumer app (e.g. zigbee2mqtt) provisions its own account through the
# install/start hooks in ./scripts/, which call mosquitto_passwd against the
# password_file below. The file lives on the persistent data volume so
# accounts survive reboots.
listener 1883
allow_anonymous false
password_file /mosquitto/data/passwd
persistence true
persistence_location /mosquitto/data/

View file

@ -0,0 +1,47 @@
#!/bin/sh
# on_start hook (provider: mosquitto).
#
# Runs INSIDE the mosquitto container on EVERY boot, before the consumer's
# container starts (reconcile visits providers first). Must be idempotent.
#
# Job: make sure the consumer's MQTT account still exists. The common path is
# a no-op — the passwd file lives on a persistent volume and survives reboots.
# This only does work if the account went missing (e.g. the data volume was
# wiped), in which case it restores the SAME password the consumer holds. It
# never rotates a live password, so the consumer's stored MQTT_PASS keeps
# working.
#
# Where the password comes from:
# - Core that injects consumer .env into on_start hooks hands it to us as
# $FURTKA_CONSUMER_ENV_MQTT_PASS — the clean path.
# - Older core gives on_start no consumer context, so we fall back to the
# copy provision-client.sh stashed on our own volume at install time.
# This hook's stdout is discarded by the reconciler (unlike on_install), so
# restoring the password is the only way to keep the consumer connectable.
# Errors go to stderr and fail the hook, which makes reconcile skip the
# consumer's `compose up` rather than start it with a broken account.
set -eu
user="${FURTKA_CONSUMER_APP:?ensure-client: FURTKA_CONSUMER_APP not set}"
passwd_file=/mosquitto/data/passwd
stash="/mosquitto/data/furtka-clients/${user}.pw"
# Account already present → nothing to do.
if [ -f "$passwd_file" ] && grep -q "^${user}:" "$passwd_file"; then
exit 0
fi
pass="${FURTKA_CONSUMER_ENV_MQTT_PASS:-}"
if [ -z "$pass" ] && [ -f "$stash" ]; then
pass="$(cat "$stash")"
fi
if [ -z "$pass" ]; then
echo "ensure-client: account '${user}' is missing and no password is" \
"available to restore it (no FURTKA_CONSUMER_ENV_MQTT_PASS, no stash);" \
"reinstall the app to re-provision." >&2
exit 1
fi
mosquitto_passwd -b "$passwd_file" "$user" "$pass"
kill -HUP 1 2>/dev/null || true

View file

@ -0,0 +1,40 @@
#!/bin/sh
# on_install hook (provider: mosquitto).
#
# Runs ONCE, INSIDE the mosquitto container, via `docker compose exec sh -s`
# while a consumer app that `requires` mosquitto is being installed. The
# reconciler/install-runner exports FURTKA_CONSUMER_APP (the app being
# installed) into our environment. stdout KEY=VALUE lines are merged into the
# consumer's .env (hook wins on conflict), so this is how the consumer learns
# its broker address and credentials.
#
# Why we stash the password provider-side: the on_start hook (ensure-client.sh)
# gets NO access to the consumer's stored password and its stdout is discarded,
# so it cannot re-create the same account after a passwd wipe unless we keep a
# copy here, on mosquitto's own persistent data volume.
set -eu
user="${FURTKA_CONSUMER_APP:?provision-client: FURTKA_CONSUMER_APP not set}"
passwd_file=/mosquitto/data/passwd
stash_dir=/mosquitto/data/furtka-clients
stash="${stash_dir}/${user}.pw"
umask 077
mkdir -p "$stash_dir"
# One random 32-char password per consumer, generated once and stashed.
pass="$(tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 32)"
printf '%s' "$pass" > "$stash"
# -b: batch mode (user + password as args). Creates the account, or updates
# it if this consumer is being reinstalled.
mosquitto_passwd -b "$passwd_file" "$user" "$pass"
# Reload the password file so the new account is usable without bouncing the
# broker. PID 1 in this container is mosquitto (see compose `exec mosquitto`).
kill -HUP 1 2>/dev/null || true
# Handed back to the install runner and merged into the consumer's .env.
echo "MQTT_SERVER=mqtt://host.docker.internal:1883"
echo "MQTT_USER=${user}"
echo "MQTT_PASS=${pass}"

View file

@ -0,0 +1,9 @@
# MQTT_* are filled in automatically at install time by mosquitto's
# provision-client.sh hook (the app requires mosquitto). Leave them blank.
MQTT_SERVER=
MQTT_USER=
MQTT_PASS=
# Path to your Zigbee USB coordinator. Asked for in the install form;
# default shown there. Change to match your stick (e.g. /dev/ttyUSB0).
ZIGBEE_SERIAL_PORT=/dev/ttyACM0

View file

@ -0,0 +1,50 @@
# Furtka Zigbee2MQTT — Zigbee-to-MQTT bridge (dependency consumer).
#
# Declares `requires: [{app: mosquitto, ...}]` in manifest.json. Installing
# this app pulls in mosquitto first; mosquitto's provision-client.sh hook then
# creates a dedicated MQTT account and writes MQTT_SERVER / MQTT_USER /
# MQTT_PASS into THIS app's .env before the container below starts. We feed
# those into zigbee2mqtt via its ZIGBEE2MQTT_CONFIG_* env overrides, so no
# configuration.yaml has to be templated by hand.
#
# host.docker.internal: the broker runs in a separate compose project on a
# separate network, so we reach its host-published 1883 via the docker
# host-gateway. The hook hands us mqtt://host.docker.internal:1883 to match.
#
# Hardware: zigbee2mqtt needs a physical Zigbee USB coordinator. ${ZIGBEE_SERIAL_PORT}
# is a required text setting (a `path`-type setting can't express a /dev node:
# the validator rejects non-directories and the /dev deny-list). The `devices`
# entry maps that host device into the container. The `:-/dev/null` fallback is
# ONLY so `docker compose config` (catalog validation, run with no .env) parses
# cleanly; a real install always has the device set. On a box with no stick
# attached the container won't fully come up — it still connects to MQTT first,
# which is enough to confirm the dependency/credential handshake.
#
# Port 8084 on the host -> 8080 in the container (8080 is taken by it-tools).
#
# TODO(image-pin): pin to a digest once verified against the upstream registry.
services:
zigbee2mqtt:
image: koenkk/zigbee2mqtt:1.42.0
restart: unless-stopped
ports:
- "8084:8080"
environment:
- TZ=Europe/Berlin
- ZIGBEE2MQTT_CONFIG_MQTT_SERVER=${MQTT_SERVER}
- ZIGBEE2MQTT_CONFIG_MQTT_USER=${MQTT_USER}
- ZIGBEE2MQTT_CONFIG_MQTT_PASSWORD=${MQTT_PASS}
- ZIGBEE2MQTT_CONFIG_SERIAL_PORT=${ZIGBEE_SERIAL_PORT}
- ZIGBEE2MQTT_CONFIG_FRONTEND_ENABLED=true
- ZIGBEE2MQTT_CONFIG_FRONTEND_PORT=8080
devices:
- ${ZIGBEE_SERIAL_PORT:-/dev/null}:${ZIGBEE_SERIAL_PORT:-/dev/null}
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- furtka_zigbee2mqtt_data:/app/data
volumes:
furtka_zigbee2mqtt_data:
external: true

View file

@ -0,0 +1,4 @@
<svg viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round">
<path d="M32 4 56 18 56 46 32 60 8 46 8 18 Z"/>
<polyline points="22,24 42,24 22,40 42,40"/>
</svg>

After

Width:  |  Height:  |  Size: 263 B

View file

@ -0,0 +1,28 @@
{
"name": "zigbee2mqtt",
"display_name": "Zigbee2MQTT",
"version": "1.0.0",
"description": "Control Zigbee sensors, lights and switches over MQTT — no vendor cloud, no vendor hub.",
"description_long": "Bindet Zigbee-Geräte (Sensoren, Lampen, Schalter, Steckdosen vieler Hersteller) über einen USB-Koordinator lokal ein und stellt sie per MQTT bereit — ohne Hersteller-Cloud und ohne Hersteller-Bridge. Benötigt einen unterstützten Zigbee-USB-Stick (z.B. ConBee II, Sonoff ZBDongle-E) an dieser Maschine. Setzt den MQTT-Broker (Mosquitto) voraus; dieser wird bei Bedarf automatisch mitinstalliert und ein eigenes Broker-Konto angelegt. Die Weboberfläche zum Koppeln und Steuern der Geräte läuft auf Port 8084.",
"volumes": ["data"],
"ports": [8084],
"icon": "icon.svg",
"open_url": "http://{host}:8084/",
"requires": [
{
"app": "mosquitto",
"on_install": "scripts/provision-client.sh",
"on_start": "scripts/ensure-client.sh"
}
],
"settings": [
{
"name": "ZIGBEE_SERIAL_PORT",
"label": "Zigbee-Stick (serieller Port)",
"description": "Gerätepfad deines Zigbee-USB-Koordinators, z.B. /dev/ttyACM0 oder /dev/ttyUSB0. Den stabilen Pfad findest du per 'ls /dev/serial/by-id/' auf der Konsole.",
"type": "text",
"required": true,
"default": "/dev/ttyACM0"
}
]
}

View file

@ -15,6 +15,7 @@ REQUIRED_FIELDS = (
VALID_SETTING_TYPES = frozenset({"text", "password", "number", "path"}) VALID_SETTING_TYPES = frozenset({"text", "password", "number", "path"})
SETTING_NAME_RE = re.compile(r"^[A-Z_][A-Z0-9_]*$") SETTING_NAME_RE = re.compile(r"^[A-Z_][A-Z0-9_]*$")
APP_NAME_RE = re.compile(r"^[a-z][a-z0-9_-]*$")
class ManifestError(Exception): class ManifestError(Exception):
@ -31,6 +32,18 @@ class Setting:
default: str | None default: str | None
@dataclass(frozen=True)
class Requirement:
app: str # name of the required app — must resolve in installed/catalog/bundled
# Hook paths are relative to the PROVIDER's app folder (not the consumer's).
# Resolved at hook-fire time, not manifest-load time — the provider may not
# be installed yet when this manifest is parsed.
# on_install: script run via `docker compose exec` on the provider during install.
on_install: str | None
# on_start: script run on every boot before the consumer starts (must be idempotent).
on_start: str | None
@dataclass(frozen=True) @dataclass(frozen=True)
class Manifest: class Manifest:
name: str name: str
@ -48,6 +61,7 @@ class Manifest:
# furtka.local, a raw IP, a future reverse-proxy hostname. Apps with # furtka.local, a raw IP, a future reverse-proxy hostname. Apps with
# no frontend (CLI-only, background workers) leave this empty. # no frontend (CLI-only, background workers) leave this empty.
open_url: str = "" open_url: str = ""
requires: tuple[Requirement, ...] = field(default_factory=tuple)
def volume_name(self, short: str) -> str: def volume_name(self, short: str) -> str:
# Namespace volume names so two apps can each declare e.g. "data" # Namespace volume names so two apps can each declare e.g. "data"
@ -98,6 +112,49 @@ def _parse_settings(raw: object, manifest_path: Path) -> tuple[Setting, ...]:
return tuple(out) return tuple(out)
def _validate_hook_path(value: object, manifest_path: Path, where: str) -> str | None:
if value is None:
return None
if not isinstance(value, str) or not value:
raise ManifestError(f"{manifest_path}: {where} must be a non-empty string if set")
if value.startswith("/"):
raise ManifestError(f"{manifest_path}: {where} must be relative (no leading /)")
parts = value.replace("\\", "/").split("/")
if any(p == ".." for p in parts):
raise ManifestError(f"{manifest_path}: {where} must not contain '..'")
return value
def _parse_requires(raw: object, manifest_path: Path, self_name: str) -> tuple[Requirement, ...]:
if raw is None:
return ()
if not isinstance(raw, list):
raise ManifestError(f"{manifest_path}: requires must be a list")
out: list[Requirement] = []
seen: set[str] = set()
for i, item in enumerate(raw):
if not isinstance(item, dict):
raise ManifestError(f"{manifest_path}: requires[{i}] must be an object")
app = item.get("app")
if not isinstance(app, str) or not app or not APP_NAME_RE.match(app):
raise ManifestError(
f"{manifest_path}: requires[{i}].app must be a non-empty lowercase app name"
)
if app == self_name:
raise ManifestError(f"{manifest_path}: requires[{i}].app {app!r} is a self-reference")
if app in seen:
raise ManifestError(f"{manifest_path}: requires has duplicate app {app!r}")
seen.add(app)
on_install = _validate_hook_path(
item.get("on_install"), manifest_path, f"requires[{app}].on_install"
)
on_start = _validate_hook_path(
item.get("on_start"), manifest_path, f"requires[{app}].on_start"
)
out.append(Requirement(app=app, on_install=on_install, on_start=on_start))
return tuple(out)
def load_manifest(path: Path, expected_name: str | None = None) -> Manifest: def load_manifest(path: Path, expected_name: str | None = None) -> Manifest:
"""Parse and validate a manifest.json. """Parse and validate a manifest.json.
@ -132,6 +189,7 @@ def load_manifest(path: Path, expected_name: str | None = None) -> Manifest:
raise ManifestError(f"{path}: ports must be a list of integers") raise ManifestError(f"{path}: ports must be a list of integers")
settings = _parse_settings(raw.get("settings"), path) settings = _parse_settings(raw.get("settings"), path)
requires = _parse_requires(raw.get("requires"), path, name)
open_url_raw = raw.get("open_url", "") open_url_raw = raw.get("open_url", "")
if not isinstance(open_url_raw, str): if not isinstance(open_url_raw, str):
@ -148,4 +206,5 @@ def load_manifest(path: Path, expected_name: str | None = None) -> Manifest:
description_long=str(raw.get("description_long", "")), description_long=str(raw.get("description_long", "")),
settings=settings, settings=settings,
open_url=open_url_raw, open_url=open_url_raw,
requires=requires,
) )