# DESIGN — Bike-Tinder Architecture for v1 (single source, single user, tailnet-only PWA). Written 2026-11-10 by Alex; engineering input from the Torres consultation (same date). Items marked 【decided】 are locked for v1. ## Components | piece | what | where | |---|---|---| | Crawler | existing `motorcycle-hunt-scan.py` (**unchanged**) — sweeps Kijiji NS+NB nightly at 08:00 | Alex's pipeline (`~/.hermes/scripts/`) | | Importer | reads the crawler's state JSONs, gates them, upserts into the app DB, scores, dedupes, queues images | `app/importer.py` | | API | FastAPI + uvicorn — deck, swipes, list, stats, images | `app/api.py` | | PWA | no-build single-page frontend (deck, list, matches, trends) | `app/static/` | | DB | SQLite (WAL mode) | `data/bike-tinder.sqlite3` (stack volume) | ## Data flow ``` 08:00 motorcycle-hunt crawl (existing, unchanged) → state JSONs: kijiji-ns.json, kijiji-nb.json 08:15 importer (script-only, zero LLM): gates → upsert → dedupe → score → gone-marking → image queue → SQLite (WAL) Deck phone ⇄ API ⇄ SQLite ↘ images: local mirror first, hotlink (302) fallback ``` ## Import design 【decided】 - **Source of truth = the hunt's state files, not a re-crawl.** Rationale: zero duplicate load on Kijiji; inherits the crawler's retry / partial / last-known-good / absence logic instead of re-implementing it; the format is stable and self-describing (`ok`, `partial`, `fetched_at`, per-page counts). - **Escape hatch:** the app vendors the Kijiji parser from `reference/motorcycle-hunt-scan.py` behind a `--direct` flag — if the hunt pipeline ever moves or dies, the importer can sweep Kijiji itself. - **Gates (fail loud, keep last good):** each state file must have `ok == true`, `partial == false`, `fetched_at` within 26 h, and non-empty `items`. Any failure → abort that region, log to `import_runs`, leave the DB untouched. - **Gone marking:** a listing absent from ≥2 consecutive *good* imports flips to `gone` (never deleted; reappearance flips it back). House discipline: a single miss — or any blocked page — never counts as "sold." - **Dedupe:** cross-province twins (same normalized title + price) collapse to one canonical row (`dup` status), keeping the more complete row (img > km > older). Merges are logged and reversible. - **Scoring:** rubric tables ported from `reference/motorcycle-hunt-rank.py` (family tiers + extras + veto) — the score is Glenn's accumulated taste, computed at import, stored on each row. See `RANKING.md`. - **Idempotent:** safe to re-run any time; also exposed as `POST /api/import` for manual triggers. ## Scheduling 【decided】 An in-container daily task (asyncio, 08:15 America/Halifax) inside the API process, with on-boot catch-up for missed runs. No host crontab is involved (the host user crontab is empty; the 08:00 crawl lives in Hermes cron — the app depends on neither). Every run writes an `import_runs` row; `GET /api/stats` surfaces the last run. ## API surface (v1) | route | purpose | |---|---| | `GET /api/health` | liveness + last-import summary | | `GET /api/stats` | counts: active / gone / dup / swiped; last import time | | `GET /api/deck?limit=` | next cards (deck query below) | | `POST /api/swipe` `{id, action}` | record or edit like / nope / maybe (upsert) | | `DELETE /api/swipe/{id}` | clear a call (returns the card to the deck) | | `GET /api/list?filter=&q=&offset=` | full list, filters + search | | `POST /api/import` | manual import trigger | | `GET /img/{id}` | cached image file, else 302 to source CDN | **Deck query:** `status='active'`, not noped, and either no swipe or a `maybe` older than 7 days — ordered `score DESC, first_seen DESC`. Unscored rows sort below scored ones but still appear — nothing is hidden. ## Images 【decided: hybrid mirror + hotlink】 - The importer pulls each new listing's card image once into `data/img/` (~100 KB avg; ≈64 MB the first day, then ~20–30 MB/month). Cards reference the local mirror; a missing file falls back to a 302 hotlink of the source CDN (verified token-free and size-rewritable). - Why: ad removal eventually kills CDN URLs (unverifiable until it happens — the mirror makes it a non-issue), phone-side speed, and "gone" cards stay viewable. - Bootstrap: today's state carries `img` on only ~3% of rows — the first import does a one-time headless re-sweep (8 pages/province via the vendored parser — 16 fetches, not 640 detail pages) to harvest card images. A placeholder renders for any listing still without a photo; it never holds up the deck. Mirrored files for rows gone >30 days are pruned. ## Frontend No-build SPA — vanilla JS + CSS, pointer-events drag, keyboard arrows on desktop (~300 lines). A small service worker caches the shell; the API is always live. Screens, gestures, and visual anatomy: see `UI-SPEC.md`. ## Deployment (house constraints) - **Stack dir:** `/var/mnt/DATA/podman-stacks/bike-tinder/` — `docker-compose.yml` + `Dockerfile` (`python:3.14-slim` + fastapi/uvicorn/requests), volume `./data:/data`, port `127.0.0.1:9453:8000`, `restart: unless-stopped`. - After adding files: `chcon -Rt container_file_t /var/mnt/DATA/podman-stacks/bike-tinder`. - **Systemd user unit** `podman-stack-bike-tinder.service`, cloned from the machine's `podman-stack-searxng.service` pattern: `Type=oneshot`, `RemainAfterExit=yes`, **`Delegate=yes`** (keeps the stack out of the hermes-gateway cgroup so gateway restarts can't SIGKILL it); ExecStart/ExecStop = the venv `podman-compose up -d` / `down`. - **Tailnet:** `tailscale serve --bg 9453` → reachable at `http://rotarran.tailf35a34.ts.net:9453`, tailnet-only. The tailnet name can't be curled from this host — first deploy verification happens from Glenn's phone. - **Backups:** nightly `sqlite3 .backup` into `data/backup/` right after import (the DB is the only mutable state; house rule). Restore = stop stack, swap the file, start. ## Security Single user; **no auth in v1** — localhost bind + tailnet-only reachability. An optional `AUTH_TOKEN` env var is noted as a one-line later hardening. No secrets are required in the stack. ## Risks & mitigations (from consultation) | risk | mitigation | |---|---| | Kijiji markup drift | parser keys on `data-testid`; failures → gates abort, deck stays last-known-good; crawler's fail/partial tracking monitors | | Anti-bot on deep sweeps | ship at the proven 8-page window; grow only after two clean weeks | | Image persistence after ad removal | local mirror (built-in) | | Cross-province dedupe false merges | logged, reversible, low blast radius; canonical keeps the more complete row | | Hunt pipeline dependency | `fetched_at` gate fails loud; vendored `--direct` parser escape hatch | | Dealer blast dominance in score ties | tie-break `first_seen DESC`; "hide this seller" planned as a v1.1 prefs row | | Rubric blindness (no family match) | rows still appear, sorted below scored ones — nothing hidden | ## Repo layout (target) ``` app/ importer.py · api.py · static/{index.html,app.js,app.css,manifest.json,sw.js,icons} deploy/ docker-compose.yml · Dockerfile · podman-stack-bike-tinder.service reference/ copies of the hunt modules the app vendors from data/ runtime state (gitignored) docs/ this design set ``` ## Runbook (completed at Phase 4) Deploy / update / rollback steps, log locations, import troubleshooting (gates), backup-restore drill, phone verification checklist.