docs: add SOURCES, DESIGN, DATA-MODEL; record v1 decisions (Kijiji source, rank-all deck, badge-only feed); vendor hunt reference modules

docs/DESIGN.md

@@ -0,0 +1,101 @@
+# DESIGN — Bike-Tinder
+
+Architecture for v1 (single source, single user, tailnet-only PWA). Written 2026-11-10 by Alex; engineering input from the Torres consultation (same date). Items marked 【decided】 are locked for v1.
+
+## Components
+
+| piece | what | where |
+|---|---|---|
+| Crawler | existing `motorcycle-hunt-scan.py` (**unchanged**) — sweeps Kijiji NS+NB nightly at 08:00 | Alex's pipeline (`~/.hermes/scripts/`) |
+| Importer | reads the crawler's state JSONs, gates them, upserts into the app DB, scores, dedupes, queues images | `app/importer.py` |
+| API | FastAPI + uvicorn — deck, swipes, list, stats, images | `app/api.py` |
+| PWA | no-build single-page frontend (deck, list, matches, trends) | `app/static/` |
+| DB | SQLite (WAL mode) | `data/bike-tinder.sqlite3` (stack volume) |
+
+## Data flow
+
+```
+08:00  motorcycle-hunt crawl (existing, unchanged) → state JSONs: kijiji-ns.json, kijiji-nb.json
+08:15  importer (script-only, zero LLM):
+         gates → upsert → dedupe → score → gone-marking → image queue
+       → SQLite (WAL)
+Deck   phone ⇄ API ⇄ SQLite
+             ↘ images: local mirror first, hotlink (302) fallback
+```
+
+## Import design 【decided】
+
+- **Source of truth = the hunt's state files, not a re-crawl.** Rationale: zero duplicate load on Kijiji; inherits the crawler's retry / partial / last-known-good / absence logic instead of re-implementing it; the format is stable and self-describing (`ok`, `partial`, `fetched_at`, per-page counts).
+- **Escape hatch:** the app vendors the Kijiji parser from `reference/motorcycle-hunt-scan.py` behind a `--direct` flag — if the hunt pipeline ever moves or dies, the importer can sweep Kijiji itself.
+- **Gates (fail loud, keep last good):** each state file must have `ok == true`, `partial == false`, `fetched_at` within 26 h, and non-empty `items`. Any failure → abort that region, log to `import_runs`, leave the DB untouched.
+- **Gone marking:** a listing absent from ≥2 consecutive *good* imports flips to `gone` (never deleted; reappearance flips it back). House discipline: a single miss — or any blocked page — never counts as "sold."
+- **Dedupe:** cross-province twins (same normalized title + price) collapse to one canonical row (`dup` status), keeping the more complete row (img > km > older). Merges are logged and reversible.
+- **Scoring:** rubric tables ported from `reference/motorcycle-hunt-rank.py` (family tiers + extras + veto) — the score is Glenn's accumulated taste, computed at import, stored on each row. See `RANKING.md`.
+- **Idempotent:** safe to re-run any time; also exposed as `POST /api/import` for manual triggers.
+
+## Scheduling 【decided】
+
+An in-container daily task (asyncio, 08:15 America/Halifax) inside the API process, with on-boot catch-up for missed runs. No host crontab is involved (the host user crontab is empty; the 08:00 crawl lives in Hermes cron — the app depends on neither). Every run writes an `import_runs` row; `GET /api/stats` surfaces the last run.
+
+## API surface (v1)
+
+| route | purpose |
+|---|---|
+| `GET /api/health` | liveness + last-import summary |
+| `GET /api/stats` | counts: active / gone / dup / swiped; last import time |
+| `GET /api/deck?limit=` | next cards (deck query below) |
+| `POST /api/swipe` `{id, action}` | record or edit like / nope / maybe (upsert) |
+| `DELETE /api/swipe/{id}` | clear a call (returns the card to the deck) |
+| `GET /api/list?filter=&q=&offset=` | full list, filters + search |
+| `POST /api/import` | manual import trigger |
+| `GET /img/{id}` | cached image file, else 302 to source CDN |
+
+**Deck query:** `status='active'`, not noped, and either no swipe or a `maybe` older than 7 days — ordered `score DESC, first_seen DESC`. Unscored rows sort below scored ones but still appear — nothing is hidden.
+
+## Images 【decided: hybrid mirror + hotlink】
+
+- The importer pulls each new listing's card image once into `data/img/` (~100 KB avg; ≈64 MB the first day, then ~20–30 MB/month). Cards reference the local mirror; a missing file falls back to a 302 hotlink of the source CDN (verified token-free and size-rewritable).
+- Why: ad removal eventually kills CDN URLs (unverifiable until it happens — the mirror makes it a non-issue), phone-side speed, and "gone" cards stay viewable.
+- Bootstrap: today's state carries `img` on only ~3% of rows — the first import does a one-time headless re-sweep (8 pages/province via the vendored parser — 16 fetches, not 640 detail pages) to harvest card images. A placeholder renders for any listing still without a photo; it never holds up the deck. Mirrored files for rows gone >30 days are pruned.
+
+## Frontend
+
+No-build SPA — vanilla JS + CSS, pointer-events drag, keyboard arrows on desktop (~300 lines). A small service worker caches the shell; the API is always live. Screens, gestures, and visual anatomy: see `UI-SPEC.md`.
+
+## Deployment (house constraints)
+
+- **Stack dir:** `/var/mnt/DATA/podman-stacks/bike-tinder/` — `docker-compose.yml` + `Dockerfile` (`python:3.14-slim` + fastapi/uvicorn/requests), volume `./data:/data`, port `127.0.0.1:9453:8000`, `restart: unless-stopped`.
+- After adding files: `chcon -Rt container_file_t /var/mnt/DATA/podman-stacks/bike-tinder`.
+- **Systemd user unit** `podman-stack-bike-tinder.service`, cloned from the machine's `podman-stack-searxng.service` pattern: `Type=oneshot`, `RemainAfterExit=yes`, **`Delegate=yes`** (keeps the stack out of the hermes-gateway cgroup so gateway restarts can't SIGKILL it); ExecStart/ExecStop = the venv `podman-compose up -d` / `down`.
+- **Tailnet:** `tailscale serve --bg 9453` → reachable at `http://rotarran.tailf35a34.ts.net:9453`, tailnet-only. The tailnet name can't be curled from this host — first deploy verification happens from Glenn's phone.
+- **Backups:** nightly `sqlite3 .backup` into `data/backup/` right after import (the DB is the only mutable state; house rule). Restore = stop stack, swap the file, start.
+
+## Security
+
+Single user; **no auth in v1** — localhost bind + tailnet-only reachability. An optional `AUTH_TOKEN` env var is noted as a one-line later hardening. No secrets are required in the stack.
+
+## Risks & mitigations (from consultation)
+
+| risk | mitigation |
+|---|---|
+| Kijiji markup drift | parser keys on `data-testid`; failures → gates abort, deck stays last-known-good; crawler's fail/partial tracking monitors |
+| Anti-bot on deep sweeps | ship at the proven 8-page window; grow only after two clean weeks |
+| Image persistence after ad removal | local mirror (built-in) |
+| Cross-province dedupe false merges | logged, reversible, low blast radius; canonical keeps the more complete row |
+| Hunt pipeline dependency | `fetched_at` gate fails loud; vendored `--direct` parser escape hatch |
+| Dealer blast dominance in score ties | tie-break `first_seen DESC`; "hide this seller" planned as a v1.1 prefs row |
+| Rubric blindness (no family match) | rows still appear, sorted below scored ones — nothing hidden |
+
+## Repo layout (target)
+
+```
+app/            importer.py · api.py · static/{index.html,app.js,app.css,manifest.json,sw.js,icons}
+deploy/         docker-compose.yml · Dockerfile · podman-stack-bike-tinder.service
+reference/      copies of the hunt modules the app vendors from
+data/           runtime state (gitignored)
+docs/           this design set
+```
+
+## Runbook (completed at Phase 4)
+
+Deploy / update / rollback steps, log locations, import troubleshooting (gates), backup-restore drill, phone verification checklist.