bgg.gvoth.dev

BoardGameGeek Corpus · CZ/SK E-Commerce Entity Resolution · Client-Side Recommender · Remote MCP

BoardGameGeek Analytics & CZ/SK E-Commerce Deal Intelligence

An analytical stack that enriches 43,647 geek-rated games across 20 historical BoardGameGeek rank snapshots, resolves multilingual product listings from 15 Czech & Slovak e-commerce retailers to canonical BGG IDs, and compiles the entire query engine and 6-factor recommender to run client-side on static hosting—or inside Claude via a remote Cloudflare Workers MCP connector.

Architecture & Pipeline

One Analytical Spine, Three Execution Targets: DuckDB, Browser & Cloudflare Workers

Rather than running a live database server in production, the Python + DuckDB backend serves as the canonical reference engine and static-shard compiler. Before every deploy, npm run parity proves that the browser TypeScript ports (and the Cloudflare Workers MCP connector that wraps them) return byte-identical answers to the Python DuckDB API across filter, deal, basket, and similarity queries.

Parquet & DuckDB Reference Pipeline → Parity-Checked Static & MCP Runtime BGG Ranks + XML API2 182.7k rows → 43,647 rated games per-ID XML cache · 20 snapshots 15 CZ/SK E-Commerce Shops 7 feed, HTML & JSON-LD adapters 50% shrink floor · ECB FX rates Embeddings & Collection 384-d MiniLM description vectors hash-diffed · owned-games sync Canonical Parquet Store + 7-Tier Entity & Language Resolver (DuckDB / FastAPI) enriched · history · offers · prices · matches (86.8k name keys + 3.2k edition keys + GTIN-14 barcode bridge) 33 reference API routes · 90-day-to-weekly native-currency price rollups · outlier & RRP demotion Static Shard Compiler & Exact Parity Gate — export_static.py + npm run parity columnar JSON shards + int8 embeddings.bin · demands exact equality between TS browser engine & Python API AES-GCM encryption (seal.py) with key gated in Firestore to authorized Google readers Next.js 16 Static App (Firebase Hosting) 10 views (/browse, /deals, /baskets, /trees, /movers…) 100% in-browser filtering, basket math & 6-factor scoring Claude MCP Connector (Cloudflare Workers) 2 Durable Objects running web/src/lib/api.ts over sealed data 14 read-only tools + 2 interactive in-chat MCP Apps

Entity Resolution & Data Hygiene

Matching Multilingual CZ/SK E-Commerce Listings Without False Bargains

Joining Czech, Slovak, and English retail listings to BoardGameGeek ratings is an adversarial entity-resolution problem: localized titles, unannounced editions, promo packs sharing a base game's barcode, and standing RRP markdowns all conspire to manufacture fake "deals." The shops/ pipeline enforces strict invariants across all 15 CZ/SK e-commerce sources:

1. 7-Tier Matcher & Canonical Edition Keys

shops/match.py resolves titles through seven tiers (manual → exact → alias → ean → prefix → fuzzy → miss) against ~86,800 primary/alternate BGG names and ~3,200 canonical edition keys (2nd Edition, 2. edice, druhá edícia, Deluxe, Collector's), ensuring a 2nd Edition box lands on its own BGG entry rather than the out-of-print original.

2. GTIN-14 Barcode Bridge & 5-Rung Language Ladder

Zero-padded GTIN-14 barcodes bridge resolved BGG IDs across retailers for localized titles no normalizer could parse—guarded by GS1 check-digit validation, family-barcode detection, and a 0.2×–5× seed-median price veto. Box language resolves down a 5-rung ladder (declared → EAN sibling → BGG version → exact-tier English → title marker, plus exclusion sets).

3. Price-Outlier, RRP & FX-Drift Defenses

Two demotion passes strip expansion-to-base mismatches and underpriced prefix matches (such as €10 promo-card packs wearing a €150 base game's ID). Standing list-price gaps are separated from genuine retailer markdowns (discount_source = list_price vs declared), sentinel placeholder prices (99,999 CZK) are dropped, and price history stores native CZK/EUR so exchange-rate movement never looks like a price cut.

4. 50% Shrink Floor & Snapshot Carry-Forward

Every adapter verifies feed XML structure or pagination completion before replacing cache files, and build.check_plausible rejects any build under 50% of a retailer's previous offer count. When an e-shop serves a maintenance page or truncates a walk, the pipeline raises ShopFetchError and carries forward the retailer's last verified snapshot.

Algorithms & Interactive Exploration

6-Factor Hybrid Recommender, De-Trended Movers & 10 Views

Subsystem / View Mathematical & Engineering Design
6-Factor Recommender + MMR (/browse, /shortlist, /compare) Multiplies six per-pair factors: TF-IDF cosine over mechanics (1.3× weight), categories, and curated families; Gaussian penalties on complexity (Δweight) and Δlog₂(playtime); IDF-weighted shared-designer boost (capped at 1.5×); 384-d MiniLM description embedding cosine (dequantized from int8 in-browser); and player-count overlap. Results are diversified with Maximal Marginal Relevance (MMR) and filtered via transitive closure over boardgameimplementation edition links.
Where to Order & Basket Optimizer (/baskets, /deals) Groups standout below-market titles by retailer against each shop's hand-verified CZ/SK shipping tariff and free-shipping threshold, surfacing optimal filler games from your shortlist or top value-per-euro picks that cost less than the shipping fee they save.
Percentile-De-Trended Rank Movers (/movers) Ranks risers and fallers across the 20 historical BGG snapshots by vs_field (percentile-rank delta rank / total_ranked re-expressed as places) rather than raw rank delta—eliminating the uniform downward drift (~350 places at rank 2,400) caused by new releases entering the rankings.
Interactive Similarity Trees (/trees) Renders sparse [3, 2, 1, 1] similarity graphs in @xyflow/react with both hierarchical dagre and custom polar radial layouts (allocating angular sectors proportional to subtree leaf count), rooted either at any of the 8 BGG category #1s or any individual game.

Zero-Server Security & AI Integration

AES-GCM Sealed Static Hosting & Remote Claude MCP Apps

Zero-Backend Sealed Static Deployment

publish.py compiles the corpus into content-hashed static shards, runs the 1,189-test Python suite and the TypeScript parity gate, and encrypts every file under /data/* with AES-GCM (seal.py). The decryption key lives in a Firestore document readable only by allowlisted Google accounts, so Firebase Hosting's free static plan serves an authenticated, zero-server analytical app.

Cloudflare Workers MCP Server + In-Chat UI Cards

connector/ deploys the exact same TypeScript query facade (web/src/lib/api.ts) into two Cloudflare Workers Durable Objects behind the same Google OAuth allowlist. It exposes 14 read-only tools plus 2 interactive MCP Apps that render directly inside Claude (web, desktop, and mobile): a cover-and-price game carousel (show_games) and an interactive retailer basket with live free-shipping progress (show_basket).