DATA-MODEL — Bike-Tinder

SQLite, WAL mode, busy_timeout. One file: data/bike-tinder.sqlite3. Schema as consulted (Torres, 2026-11-10), locked for v1.

Tables

listings (
  id             INTEGER PRIMARY KEY AUTOINCREMENT,
  region         TEXT NOT NULL CHECK (region IN ('ns','nb')),
  src_id         TEXT NOT NULL,                    -- source ad id (Kijiji id in v1)
  title          TEXT NOT NULL,
  price_display  TEXT,                             -- "$5,000"; "Please Contact"
  price_num      INTEGER,                          -- NULL when unpackable
  km             INTEGER,                          -- NULL when not stated (~2/3 of cards)
  location       TEXT,
  url            TEXT NOT NULL,                    -- source ad page
  img_url        TEXT,                             -- source CDN card image
  desc           TEXT,                             -- card description snippet
  family         TEXT,                             -- top rubric family label (taste signal)
  score          INTEGER NOT NULL DEFAULT 0,       -- rubric score, recomputed each import
  status         TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','gone','dup')),
  dup_of         INTEGER,                          -- canonical row id when status='dup'
  first_seen     TEXT,  last_seen TEXT,
  created_at     TEXT DEFAULT (datetime('now')),  updated_at TEXT,
  UNIQUE (region, src_id)
);
CREATE INDEX idx_listings_deck ON listings (status, score DESC);
CREATE INDEX idx_listings_dup  ON listings (dup_of) WHERE status = 'dup';

swipes (
  listing_id  INTEGER PRIMARY KEY REFERENCES listings(id) ON DELETE CASCADE,
  action      TEXT NOT NULL CHECK (action IN ('like','nope','maybe')),
  created_at  TEXT DEFAULT (datetime('now')),
  updated_at  TEXT
);

import_runs (
  id, started_at, ok, partial,
  src_fetched_ns, src_fetched_nb,
  total, new, gone, dup, error
);                                                 -- one observability row per nightly run

prefs (
  key TEXT PRIMARY KEY, value TEXT
);                                                 -- deck_size, depth, etc. — tiny

Notes:

One row per swipe; a new swipe upserts (a re-swiped card overwrites its action). The latest action is the truth; edits from the list view are upserts. Append-history is deliberately not kept in v1.
import_runs is the dashboard's evidence trail — every run writes one row, success or failure.

State semantics

new — active with no swipes row → the deck.
seen — has any swipe row.
gone — was active, then absent from ≥2 consecutive good imports → status='gone' (kept forever, swipes preserved; reappearance flips it back to active). Single misses and failed/partial imports never mark gone.
dup — the second province's twin of the same ad (dedupe below) → hidden from the deck, visible under its canonical row in the list.
nope — never re-enters the deck unless cleared from the list (DELETE swipe → back to unswiped).
maybe — soft skip: the deck query re-admits maybe rows after 7 days (computed at query time — no cron needed). Editing a maybe to like/nope is just an upsert.
like — pins to the Liked ("matches") view.

Deck query (one statement)

SELECT * FROM listings
WHERE status = 'active'
  AND id NOT IN (SELECT listing_id FROM swipes WHERE action = 'nope')
  AND id NOT IN (SELECT listing_id FROM swipes WHERE action IN ('like','maybe')
                 AND (action = 'like' OR updated_at > datetime('now','-7 days')))
ORDER BY score DESC, first_seen DESC
LIMIT :deck_size;

(Implementers: a LEFT JOIN swipes variant is equivalent and cheaper — the intent above is the contract. Unscored rows sort below scored ones; nothing is hidden.)

Dedupe (Kijiji cross-province)

Group imports by (normalized_title, price_num); when a group spans both provinces, keep the more-complete row canonical (img present → km present → older row) and mark the twin(s) status='dup', dup_of=canonical. The twin's URL is preserved on the canonical row (extra note field / desc suffix).
Today's snapshot: 58/320 NS rows are twins of NB rows (dealers cross-post one ad to both provinces).
False merges are possible (same title + price, different bike) but rare; every merge is logged in import_runs and reversible in the DB. Honest v1 tradeoff.

Import gates (must pass or the run aborts)

State file ok == true and partial == false.
fetched_at within 26 hours.
Non-empty items.
On failure: log, leave the DB untouched (deck stays last-known-good).

Maintenance

Backup: nightly sqlite3 .backup to data/backup/ immediately after import (house rule; the DB is the app's only mutable state).
Prune: mirrored image files for rows gone >30 days; rows themselves are never auto-deleted in v1.
VACUUM: optional; the DB is KV-sized (a few thousand rows).

P2 additions — frozen 2026-11-10 (production build)

New columns (guarded, idempotent migrations in db.init_db: ALTER TABLE ... ADD COLUMN when missing):
- listings.year INTEGER — parsed from title at import (rubric.year_from_title); NULL when absent.
- listings.seller_type TEXT — 'dealer' | 'private' | NULL. Filled ONLY from a real page signal (the hunt image backfill / a later enrichment pass); never inferred from titles.
- listings.extras TEXT — JSON array of rubric-matched extra labels (MVI/cert, whitewalls, spokes, windshield, bags, ABS); UI displays at most 3.
Importer: fills the three new fields; all P1 behavior unchanged (gates, upsert, dedupe, gone-marking, idempotence, counts 640/560/80).
Deck feed + cursor pagination (production): semantics unchanged (no swipe, or maybe older than 7 days; active only); order score DESC, first_seen DESC, id DESC. Cursor = opaque "<score>|<first_seen>|<id>" of the last returned row; next page = strictly-after in that tuple order (OR-chain on the three keys). GET /api/deck?limit&cursor → {items, next_cursor, remaining}. Cursor stays correct while swipes delete rows underneath it.
Badge: prefs['badge_seen_at'] (ISO). new = active rows with first_seen > COALESCE(badge_seen_at,'0'); POST /api/badge/seen stamps now.
Media mirror: files at data/img/{listing_id}.webp; GET /img/{id} → file (200) → img_url (302) → 404 (UI placeholder). Downloads via app/imgfetch.py (polite: ~1 req/s, skip-existing, resumable). img_url enrichment happens upstream in the hunt state (motorcycle-hunt-img.py; atomic writes; rescans keep curated imgs) and lands via import.
Dedupe refinement (stretch, targeted): for cross-province groups with >2 members, replace collapse-to-one with min(ns,nb) completeness pairing; surplus siblings stay active. Expected dup ≈ 77 (vs 80). Verify counts and report.

P3 additions — frozen 2026-11-10 (personalization)

New tables (guarded, idempotent creation; fresh AND existing DBs migrate in place):

swipe_events (                       -- append-only strike log; the learning source of truth
  id          INTEGER PRIMARY KEY AUTOINCREMENT,
  listing_id  INTEGER NOT NULL REFERENCES listings(id) ON DELETE CASCADE,
  action      TEXT NOT NULL CHECK (action IN ('like','nope','maybe')),
  round       INTEGER NOT NULL DEFAULT 1,   -- 1 = first verdict; 2+ = second look (engine caps influence at round 2)
  section_id  INTEGER,                      -- section it happened in (NULL for list edits / legacy)
  created_at  TEXT DEFAULT (datetime('now'))
);
CREATE INDEX idx_events_listing ON swipe_events (listing_id, round);
CREATE INDEX idx_events_time    ON swipe_events (created_at);

sections (
  id              INTEGER PRIMARY KEY AUTOINCREMENT,
  status          TEXT NOT NULL DEFAULT 'open' CHECK (status IN ('open','report','closed')),
  size            INTEGER NOT NULL,
  profile_version INTEGER,                  -- taste_profile.version used at compose
  recipe          TEXT,                     -- JSON summary {new, return, wildcard, caps_applied, note}
  created_at      TEXT DEFAULT (datetime('now')),
  closed_at       TEXT
);

section_items (
  section_id  INTEGER NOT NULL REFERENCES sections(id) ON DELETE CASCADE,
  listing_id  INTEGER NOT NULL REFERENCES listings(id) ON DELETE CASCADE,
  slot        TEXT NOT NULL CHECK (slot IN ('new','return','wildcard')),
  position    INTEGER NOT NULL,
  PRIMARY KEY (section_id, listing_id)
);

taste_profile (
  version    INTEGER PRIMARY KEY AUTOINCREMENT,
  created_at TEXT DEFAULT (datetime('now')),
  source     TEXT NOT NULL CHECK (source IN ('agent','mock','fallback')),
  payload    TEXT NOT NULL                  -- JSON: lead, leanings, changes, chips, adjustments, wildcards
);

agent_runs (
  id         INTEGER PRIMARY KEY AUTOINCREMENT,
  started_at TEXT, finished_at TEXT,
  ok         INTEGER NOT NULL DEFAULT 0,
  source     TEXT, model TEXT, cost_note TEXT, error TEXT,
  profile_version INTEGER
);

Semantics:

Learning = swipe_events only (append-only). A cleared swipe keeps its history — clearing removes the UI verdict, not the record. UI truth = swipes (latest verdict / cleared), unchanged from P1–P2.
Round = prior events for that listing + 1, assigned at write time. Rounds cap at 2 for engine influence (any later events weigh as round 2; v1 returns never produce rounds >2 by design).
Migration seeding: when swipe_events is empty and swipes has rows → insert one round-1 event per swipe row (created_at copied). Only-when-empty, hence idempotent.
Sections: at most ONE open section at a time. The deck = that section's section_items in position order, filtered to items undecided at read. Exhausted = no item undecided → status='report'. Forge = close current + open new. First-ever read composes a cold section (rubric-weighted, w=0).
Returns eligibility (v1 contract): a verdict exists (like/maybe/nope in swipes); listing active; NOT in either of the previous two sections; no round≥2 nope ever (rested); not already returned once (round cap). Selection prefers maybe > like > nope; nope candidates must be "surprising" (points ≥55 or net-positive family). Max 5 returns per section, ≤2 of them nopes.
points is computed, not stored (deterministic from events + profile; a cache is allowed only if it never changes semantics).
prefs: forge_mode ('ondemand' default | 'strict'). Existing keys unchanged.
Legacy deck semantics: /api/deck (cursor paging; 7-day maybe re-admit) stays as a read-only fallback for debugging; sections are the production flow.

10 KiB Raw Blame History Unescape Escape