DATA-MODEL — Bike-Tinder

SQLite, WAL mode, busy_timeout. One file: data/bike-tinder.sqlite3. Schema as consulted (Torres, 2026-11-10), locked for v1.

Tables

listings (
  id             INTEGER PRIMARY KEY AUTOINCREMENT,
  region         TEXT NOT NULL CHECK (region IN ('ns','nb')),
  src_id         TEXT NOT NULL,                    -- source ad id (Kijiji id in v1)
  title          TEXT NOT NULL,
  price_display  TEXT,                             -- "$5,000"; "Please Contact"
  price_num      INTEGER,                          -- NULL when unpackable
  km             INTEGER,                          -- NULL when not stated (~2/3 of cards)
  location       TEXT,
  url            TEXT NOT NULL,                    -- source ad page
  img_url        TEXT,                             -- source CDN card image
  desc           TEXT,                             -- card description snippet
  family         TEXT,                             -- top rubric family label (taste signal)
  score          INTEGER NOT NULL DEFAULT 0,       -- rubric score, recomputed each import
  status         TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','gone','dup')),
  dup_of         INTEGER,                          -- canonical row id when status='dup'
  first_seen     TEXT,  last_seen TEXT,
  created_at     TEXT DEFAULT (datetime('now')),  updated_at TEXT,
  UNIQUE (region, src_id)
);
CREATE INDEX idx_listings_deck ON listings (status, score DESC);
CREATE INDEX idx_listings_dup  ON listings (dup_of) WHERE status = 'dup';

swipes (
  listing_id  INTEGER PRIMARY KEY REFERENCES listings(id) ON DELETE CASCADE,
  action      TEXT NOT NULL CHECK (action IN ('like','nope','maybe')),
  created_at  TEXT DEFAULT (datetime('now')),
  updated_at  TEXT
);

import_runs (
  id, started_at, ok, partial,
  src_fetched_ns, src_fetched_nb,
  total, new, gone, dup, error
);                                                 -- one observability row per nightly run

prefs (
  key TEXT PRIMARY KEY, value TEXT
);                                                 -- deck_size, depth, etc. — tiny

Notes:

One row per swipe; a new swipe upserts (a re-swiped card overwrites its action). The latest action is the truth; edits from the list view are upserts. Append-history is deliberately not kept in v1.
import_runs is the dashboard's evidence trail — every run writes one row, success or failure.

State semantics

new — active with no swipes row → the deck.
seen — has any swipe row.
gone — was active, then absent from ≥2 consecutive good imports → status='gone' (kept forever, swipes preserved; reappearance flips it back to active). Single misses and failed/partial imports never mark gone.
dup — the second province's twin of the same ad (dedupe below) → hidden from the deck, visible under its canonical row in the list.
nope — never re-enters the deck unless cleared from the list (DELETE swipe → back to unswiped).
maybe — soft skip: the deck query re-admits maybe rows after 7 days (computed at query time — no cron needed). Editing a maybe to like/nope is just an upsert.
like — pins to the Liked ("matches") view.

Deck query (one statement)

SELECT * FROM listings
WHERE status = 'active'
  AND id NOT IN (SELECT listing_id FROM swipes WHERE action = 'nope')
  AND id NOT IN (SELECT listing_id FROM swipes WHERE action IN ('like','maybe')
                 AND (action = 'like' OR updated_at > datetime('now','-7 days')))
ORDER BY score DESC, first_seen DESC
LIMIT :deck_size;

(Implementers: a LEFT JOIN swipes variant is equivalent and cheaper — the intent above is the contract. Unscored rows sort below scored ones; nothing is hidden.)

Dedupe (Kijiji cross-province)

Group imports by (normalized_title, price_num); when a group spans both provinces, keep the more-complete row canonical (img present → km present → older row) and mark the twin(s) status='dup', dup_of=canonical. The twin's URL is preserved on the canonical row (extra note field / desc suffix).
Today's snapshot: 58/320 NS rows are twins of NB rows (dealers cross-post one ad to both provinces).
False merges are possible (same title + price, different bike) but rare; every merge is logged in import_runs and reversible in the DB. Honest v1 tradeoff.

Import gates (must pass or the run aborts)

State file ok == true and partial == false.
fetched_at within 26 hours.
Non-empty items.
On failure: log, leave the DB untouched (deck stays last-known-good).

Maintenance

Backup: nightly sqlite3 .backup to data/backup/ immediately after import (house rule; the DB is the app's only mutable state).
Prune: mirrored image files for rows gone >30 days; rows themselves are never auto-deleted in v1.
VACUUM: optional; the DB is KV-sized (a few thousand rows).

P2 additions — frozen 2026-11-10 (production build)

New columns (guarded, idempotent migrations in db.init_db: ALTER TABLE ... ADD COLUMN when missing):
- listings.year INTEGER — parsed from title at import (rubric.year_from_title); NULL when absent.
- listings.seller_type TEXT — 'dealer' | 'private' | NULL. Filled ONLY from a real page signal (the hunt image backfill / a later enrichment pass); never inferred from titles.
- listings.extras TEXT — JSON array of rubric-matched extra labels (MVI/cert, whitewalls, spokes, windshield, bags, ABS); UI displays at most 3.
Importer: fills the three new fields; all P1 behavior unchanged (gates, upsert, dedupe, gone-marking, idempotence, counts 640/560/80).
Deck feed + cursor pagination (production): semantics unchanged (no swipe, or maybe older than 7 days; active only); order score DESC, first_seen DESC, id DESC. Cursor = opaque "<score>|<first_seen>|<id>" of the last returned row; next page = strictly-after in that tuple order (OR-chain on the three keys). GET /api/deck?limit&cursor → {items, next_cursor, remaining}. Cursor stays correct while swipes delete rows underneath it.
Badge: prefs['badge_seen_at'] (ISO). new = active rows with first_seen > COALESCE(badge_seen_at,'0'); POST /api/badge/seen stamps now.
Media mirror: files at data/img/{listing_id}.webp; GET /img/{id} → file (200) → img_url (302) → 404 (UI placeholder). Downloads via app/imgfetch.py (polite: ~1 req/s, skip-existing, resumable). img_url enrichment happens upstream in the hunt state (motorcycle-hunt-img.py; atomic writes; rescans keep curated imgs) and lands via import.
Dedupe refinement (stretch, targeted): for cross-province groups with >2 members, replace collapse-to-one with min(ns,nb) completeness pairing; surplus siblings stay active. Expected dup ≈ 77 (vs 80). Verify counts and report.

6.7 KiB Raw Blame History