One application that extracts and reviews show calendars for multiple
theaters. A shared pipeline in common/ produces a common JSON format
for every venue; each theater is a thin adapter on top of it.
The registry in common/venues.py is the source of truth — this table is a
summary and can lag behind it (a new venue lands as one registry entry; see
"Adding a new theater" below).
| Dir | Venue | Source | Route | Notes |
|---|---|---|---|---|
independent/ |
The Independent, SF | theindependentsf.com | :8050/independent/ |
music; Apple Music matching (confidence tiers) |
castro/ |
The Castro Theatre, SF | thecastro.com | :8050/castro/ |
music + film + comedy + other (derived); Apple Music matching on music + comedy (film artists are movie/director names, not catalog artists) |
fillmore/ |
The Fillmore, SF | thefillmore.com | :8050/fillmore/ |
music (site "Arts & Theatre" segment = comedy/variety → other); Apple Music matching on all |
gamh/ |
Great American Music Hall, SF | greatmusichall.com | :8050/gamh/ |
music (festival/awards/poetry/Christmas bills → other); support acts from each show's detail page; Apple Music matching on all |
regency/ |
The Regency Ballroom, SF | theregencyballroom.com (AEG/AXS blob feed) | :8050/regency/ |
music (all bills); one unauthenticated AEG JSON GET = the whole calendar with headliners + support pre-structured; Apple Music matching on all |
augusthall/ |
August Hall, SF | augusthallsf.com (TicketWeb WordPress) | :8050/augusthall/ |
music (all bills); listing pages + per-show detail pages (plain HTML, server-side); tour titles, "with" support, and censored names handled; Apple Music matching on all |
warfield/ |
The Warfield Theatre, SF | thewarfieldtheatre.com (AEG/AXS template) | :8050/warfield/ |
music (all bills); the /events/all listing is fully server-rendered with presenter / tour / headliner / support as separate lines (no per-detail title parsing); detail pages add doors + age; Apple Music matching on all |
cafedunord/ |
Cafe du Nord, SF | cafedunord.com (WordPress, two rooms) | :8050/cafedunord/ |
music (all bills); the /shows listing is fully server-rendered with presenter / headliner / comma-list support as separate fields; detail pages add the description (MusicEvent JSON-LD); the upstairs Swedish American Hall room is surfaced via the room field; Apple Music matching on all |
foxoakland/ |
Fox Theater, Oakland | thefoxoakland.com (Another Planet / TicketWeb WordPress) | :8050/foxoakland/ |
music and comedy; the /listing/ listing is fully server-rendered (one GET = the whole schedule, no pagination) with tour / headliner / <br>-separated support as separate fields; detail pages add the age rating + a per-artist bio (description); event_type is derived from the bio/topline (no venue taxonomy); Apple Music matching on all |
greek/ |
Greek Theatre, Berkeley | thegreekberkeley.com (UC Berkeley, Another Planet / TicketWeb WordPress) | :8050/greek/ |
music (APE's exclusive-promoter concerts — the full season incl. theater/film is on thegreek.com, which is Cloudflare-walled); the /event-listing/ page is fully server-rendered with tour / headliner / <br>-separated support as separate fields; detail pages add age + per-artist bio; single room; Apple Music matching on all |
brickandmortar/ |
Brick & Mortar Music Hall, SF | brickandmortarmusic.com (TicketWeb WordPress) | :8050/brickandmortar/ |
music; the /shows listing is fully server-rendered and NOT paginated (one GET = the whole schedule; /page/N/ returns the same shows) — a 250-cap venue, so bills are + multi-headline / Name — Tour / comma- or w/-support; the button class (tw_ticketssold/tw_soldout) encodes sale state, detail pages add the full date (the listing's date is a year-less MM.DD) + the per-artist bio; Apple Music matching on all |
frost/ |
Frost Amphitheater, Stanford | live.stanford.edu (Stanford Live, public JSON API) | :8050/frost/ |
music (the summer concert season, ticketed via AXS); one API GET = the whole season, filtered by the Frost flag; server-rendered detail pages add the tour, per-date "Featured Artists" support, age, doors, bio; the off-season is legitimately EMPTY (a summer venue — ~April–October) |
chapel/ |
The Chapel, SF | thechapelsf.com | :8050/chapel/ |
music (a single-room live-music calendar); the /calendar page is a plain-GET WordPress page of 12 server-rendered month tables — the date is reconstructed from the table's month header + the card's day number (2026-10-25: bound to the card's own cell); the Seetickets ticket link is kept but bot-walled, never scraped; Apple Music matching on all |
rickshawstop/ |
Rickshaw Stop, SF (Civic Center) | rickshawstop.com (calendar via WP REST API) | :8050/rickshawstop/ |
eclectic room (indie/rock/dance + the occasional stand-up); the HTML routes are Cloudflare-walled, so /calendar is read via the WordPress REST API, merging 12 month tables with the nearest list-view cards (clean name, support, genre, ages); stand-up genre → comedy, everything else music; Apple Music matching on all |
bgc/ |
Bill Graham Civic Auditorium, SF | billgrahamcivic.com (Another Planet / wpengine WordPress) | :8050/bgc/ |
music (all bills); the /calendar listing is fully server-rendered and venue-filtered (one GET = the whole season, no pagination) with clean headliner / support / date / times / button fields; detail pages add the age rating + per-artist Spotify links (venue-published) + a headliner bio; the Theater at BGC is a separate room with its own category; Apple Music matching on all |
bellyup/ |
Belly Up, Solana Beach | bellyup.com (shared 6-room calendar) | :8050/bellyup/ |
the Belly Up promoter runs all six San Diego rooms from one TicketWeb calendar — one GET returns the multi-venue season and each of the six adapters keeps its own rows; music + occasional comedy; bellyup carries promoter placeholder rows with null event_page (by contract, see schema 5); Apple Music matching on all |
thesound/ |
The Sound, Del Mar | bellyup.com (shared 6-room calendar) | :8050/thesound/ |
see the Belly Up row — same shared calendar, this venue's rows only |
thecasbah/ |
The Casbah, San Diego | bellyup.com (shared 6-room calendar) | :8050/thecasbah/ |
see the Belly Up row — same shared calendar, this venue's rows only |
theobservatory/ |
The Observatory, San Diego (North Park) | bellyup.com (shared 6-room calendar) | :8050/theobservatory/ |
see the Belly Up row — same shared calendar, this venue's rows only |
musicbox/ |
Music Box, San Diego (Little Italy) | bellyup.com (shared 6-room calendar) | :8050/musicbox/ |
see the Belly Up row — same shared calendar, this venue's rows only |
humphreys/ |
Humphreys Concerts by the Bay, San Diego | bellyup.com (shared 6-room calendar) | :8050/humphreys/ |
see the Belly Up row — currently an empty file (no rows on the shared calendar at build time); the file still ships with zeroed stats |
pappy/ |
Pappy & Harriet's, Pioneertown | pappyandharriets.com (calendar via WP REST API) | :8050/pappy/ |
country/rock/roots desert saloon, grouped with the LA market in the app; the HTML routes are bot-walled, so /calendar is read via the WordPress REST API (12 month tables + nearest list-view cards); headliner/support derived from bill grammar (ft:/w:/with → support, // = co-headline, tour tail → show title); every published genre (Rock/Country/Jazz/Hip Hop) is music; Apple Music matching on all |
crystalball/ |
Crystal Ballroom, Portland | crystalballroompdx.com (WordPress + The Events Calendar) | :8050/crystalball/ |
first Portland-metro venue; two joined open sources: the TEC REST API (ISO show-start dates + description) and the tour-schedule CARD grid (homepage + /events/page/N — promoter, billed h1/h2, etix buy buttons) keyed by event URL; the card grid supplies billing (REST rows are the universe; recurring non-show programming — museum tour, market — is refused by pattern; a real show can be card-less and then bills from its detail page); detail pages add the "6:30pm doors, 8pm show" line + age note; h2 "Presents …/… Tour" tails are the headliner's tour, never a co-bill; Apple Music matching on all; Spotify via API lookup |
wonderball/ |
Wonder Ballroom, Portland | wonderballroom.com/events/ (WordPress + rhp-events) | :8050/wonderball/ |
one GET, fully server-rendered — head, support h4, promoter, doors/show, age, CTA status class and etix link all on the card (no detail crawl); TEC REST 404s here; the venue writes status INTO titles (*CANCELLED* kept as cancelled, *Postponed* kept with note+needs_review, *MOVED TO ROSELAND* dropped — Roseland's feed carries it); "X PRESENTS:" glued promoters split; en-dash tour tails mapped to the shared grammar; Apple Music matching on all; Spotify via API lookup |
holocene/ |
Holocene, Portland | holocene.org/events (rebuilt site — old holocenepdx.com is NXDOMAIN) | :8050/holocene/ |
SSR calendar in one GET (etix-backed; ISO date from the href, badge has no year) + per-show pages that bill pre-parsed (ev-title/ev-sub/ev-with, Doors/Ages dl, schema.org MusicEvent JSON-LD: performer[], doorTime, eventStatus, offers) — calendar titles never go through parse_bill ("presented by" butchery); DJ/rave nights are music; sold-out/at-the-door button classes + LD availability map to status; Apple Music matching on all; Spotify via API lookup |
roseland/ |
Roseland Theater, Portland | roselandpdx.com/events/ (WordPress + rhp-events, Wonder Ballroom's plugin but drifted) | :8050/roseland/ |
server-rendered cards like wonderball, drifted template: wrapper class suffixes, eventDate has no year (month separators "November 2026" supply it; mismatches warn), age from the notes box (age_rating + verbatim note), page/2 is a client mirror; receives Wonder's moved shows (Frost Children); wonderball build grammar (title status prefixes, colon-presenters, en-dash tails); Apple Music matching on all; Spotify via API lookup |
keller/ |
Keller Auditorium, Portland | portland5.com/events (the City venue's calendar only exists in the Portland'5 five-theater feed; old portlandoregon.gov DNS dead) | :8050/keller/ |
one row PER PERFORMANCE — Broadway runs explode from event pages (event-performance-list, per-performance evue links; update_all.show_key learned date+time suffixes); Keller is the multiplex so the event_type enum does real work (comedy regex, theater nouns gated to runs/titles, music default); multi-season alias collisions (two Clue cards, one /event/clue) dedup by URL+card-date with card-only needs_review fallback when the detail redirects; yearless perf dates inferred cross-year-safe; doors phrasing "1 hour prior to start" passes verbatim; Apple Music matching on all; Spotify via API lookup |
polarishall/ |
Polaris Hall, Portland | polarishall.com (custom-CMS one-pager, not WordPress) | :8050/polarishall/ |
whole calendar in ONE GET — SSR #shows grid (~58 cards, 6-month horizon): title/support/doors/show/etix per card; card dates carry no year (month-vs-today inference + monotonic assertion); etix CTA text is the status vocab (Get Tickets/Sold Out); constant "Polaris Hall Presents" presenter skipped; colon-presenter split caught a true positive day one; real comedy nights classified; age_rating null by design (etix-only age, bot-walled 202); Apple Music matching on all; Spotify via API lookup |
paramount/ |
The Paramount, Seattle | stgpresents.org/calendar (Seattle Theatre Group — WordPress + Modern Events Calendar) | :8050/paramount/ |
one row PER PERFORMANCE — the public admin-ajax.php action=mec_list_load_more endpoint (no nonce) returns per-occurrence cards, walked month-by-month (start_date = previous page's end_date, offset++); MEC JSON-LD startDate clock is timezone-corrupt (7:30pm curtain surfaced as 11:30-08:00) — only its DATE half is trusted, the true time is the card's mec-start-time span; categories 21,18,19 = the three STG rooms (ONE source, THREE feeds — moore/neptune reuse it); theater/comedy titles never enter parse_bill ("Oh, Mary!" → comma-slaughter); recurring "Free Paramount Tour" classes dropped; status from LD eventStatus + SOLD OUT card class; Apple Music matching on all; Spotify via API lookup |
moore/ |
The Moore, Seattle | stgpresents.org/calendar (shared STG source) | :8050/moore/ |
same common/stg.py machinery as paramount, venue label "The Moore Theatre"; 55 occurrences incl. heavy comedy programming (classified, titles kept whole); "Free Moore Tour" dropped; Apple Music matching on all; Spotify via API lookup |
neptune/ |
The Neptune, Seattle | stgpresents.org/calendar (shared STG source) | :8050/neptune/ |
same source, venue label "The Neptune Theatre" — 105 occurrences, music+comedy club nights, age/doors parsed from event posts; "Free Neptune Tour" dropped; Apple Music matching on all; Spotify via API lookup |
showbox/ |
The Showbox, Seattle | showboxpresents.com/events (AEG Live Carbonhouse, AXS checkout) | :8050/showbox/ |
ONE aggregate calendar for both Showbox rooms and every venue they present at (WAMU/Moda/Roseland/…): each card carries a @ venue label — the label splits feeds and foreign rooms are dropped (their venues own those bills); events_ajax/<offset> pages are JSON-encoded HTML strings (raw grep sees an empty calendar — json.loads first); button vocab enumerated (status_1 Buy/Coming Soon, 2 Cancelled, 6 Get Tickets); doors/curtain/age from per-show detail pages (incrementally cached); Apple Music matching on all; Spotify via API lookup |
showboxsodo/ |
Showbox SoDo, Seattle | showboxpresents.com/events (shared aggregate) | :8050/showboxsodo/ |
same parser, venue label "Showbox SoDo" (37 on-sales at discovery incl. cancelled-row kept with status); Apple Music matching on all; Spotify via API lookup |
neumos/ |
Neumos, Seattle | neumos.com/events (own Carbonhouse skin, AXS) | :8050/neumos/ |
flavor-B cards (aria-label full dates, tagline support, .time is DOORS — curtain genuinely absent site-wide → show_time null, honest); full Chrome User-Agent 406s while plain curl passes (TLS-fingerprint strict wall — the inverse of the BGC lesson; send no custom UA); on-sale-only calendar (~12-13 shows; small but the venue's real horizon); Apple Music matching on all; Spotify via API lookup |
elcorazon/ |
El Corazón, Seattle | elcorazonseattle.com/calendar (Webflow CMS, shared site with Funhouse) | :8050/elcorazon/ |
one org, one calendar, two feeds — every billing field server-rendered (calendar-headliners/calendar-supports/start-date/doors/show-time/venue), venue field splits El Corazón from Funhouse; third-party "Mammoth NW Presents" nights kept with note; no ticket link or status vocabulary exists site-wide (null by design, polarishall precedent; tickets: opendate.io); Apple Music matching on all; Spotify via API lookup |
funhouse/ |
Funhouse, Seattle | elcorazonseattle.com/calendar (shared) | :8050/funhouse/ |
same source, venue "The Funhouse" (30 punk/metal nights at discovery, dense co-bills → many multi_artist); shares the 109 E Eastlake building with El Corazón; Apple Music matching on all; Spotify via API lookup |
crocodile/ |
The Crocodile, Seattle | ticketweb.com/venue/…/10352 (primary-ticket platform; the venue's own Squarespace calendar is DEAD — placeholder "Event Five" markup) | :8050/crocodile/ |
edge bot-manager 502s bare curl AND a headless browser from the flagged IP — a full browser HEADER SET on plain curl passes (send common/ticketweb.py HEADERS verbatim); the venue page server-renders an inline JSON-LD window of ~10 newest on-sales with a CORRECT local clock; the longer list is JS-hydrated (endpoint unpinned 2026-11-07 — honest short horizon, refreshed nightly, Casbah-tier); comma bills = flat bills; Apple Music matching on all; Spotify via API lookup |
common/ THE shared pipeline (venue-agnostic)
venues.py venue registry — ADD A NEW THEATER HERE (one dict entry)
build.py shared show record shape + run_build() (sort, stats, writer)
apple_music.py Apple Music confidence engine + artist records (one copy)
spotify.py per-artist Spotify links, scraped from each venue's own
pages (no API key) + the shared merge/stats/join helpers
lookup.py Apple Music API driver (run per venue that wants links)
viewer/index.html ONE reviewer template; adapts to each venue's fields
independent/ venue adapter: fetch + title parser + build.py
castro/ venue adapter: fetch + category classifier + build.py
fillmore/ venue adapter: fetch (RSC payload) + bill reconciler + build.py
gamh/ venue adapter: fetch (JSON-LD + detail pages) + support parser + build.py
regency/ venue adapter: fetch (AEG/AXS public JSON feed) + build.py
augusthall/ venue adapter: fetch (TicketWeb listing + detail pages) + title parser + build.py
warfield/ venue adapter: fetch (AEG/AXS server-rendered listing + detail pages) + build.py
cafedunord/ venue adapter: fetch (server-rendered /shows listing + detail pages for descriptions) + build.py
foxoakland/ venue adapter: fetch (server-rendered /listing/ + detail pages for age + bio) + build.py
greek/ venue adapter: fetch (server-rendered /event-listing/ + detail pages for age + bio) + build.py
brickandmortar/ venue adapter: fetch (server-rendered /shows listing + detail pages for date + bio) + build.py
frost/ venue adapter: fetch (Stanford Live JSON API + detail pages for tour/age/bio) + build.py
serve.py single entry point: picker at /, venues at /<key>/
update_all.py one command to refresh ALL venues (fetch/build/lookup/build
+ per-venue diff; --full = full Apple Music re-match)
Each venue directory holds only what's venue-specific:
<venue>/
fetch*.py # pull the venue's schedule into cache/
build.py # map the cache into the shared show shape, then run_build()
cache/ # raw captures + (if Apple Music) lookup.json
data/shows.json # the shared-format dataset
Top level (every venue, same 13 keys):
{
"venue": "The Castro Theatre", "venue_id": "castro",
"venue_url": "...", "source_url": "...", "city": "San Francisco",
"metro": "San Francisco", "neighborhood": null,
"schema": 5, "field_hints": { ... }, "scraped_at": "2026-10-16",
"count": 78,
"shows": [ ... ],
"stats": { "event_types": {"music": 58, ...}, "apple_music": {...}, "spotify": {...} }
}Each show (18 keys — superset, null where a venue doesn't have the field):
event_type, needs_review, headline_artists, support_artists, multi_artist, show_title, presenters, set_format, date, doors, show_time, ticket_link, ticket_status, event_page, room, age_rating, note, description.
event_page is null for calendar placeholder rows (a promoter listing with
no detail page — bellyup "Private Event"). Upcoming-only retention
(schema 5): run_build drops shows whose date is strictly before the
build date and recomputes all stats from the surviving shows, so stats
always describe exactly what the file contains.
Artist records are always the same object (8 keys):
{name, apple_music, spotify, confidence, matched, apple_label, spotify_label, spotify_source} — venues without Apple Music matching leave the match fields
null; spotify is null unless the venue's own page published that artist's
link. An artist has an apple_music link iff a confident match was found (the
found/not-found signal).
stats carries any number of named blocks (event_types, apple_music, …);
the viewer and the picker render whichever are present.
Every venue links Apple Music artists (a catalog search, confidence-scored —
see the mandate below). In addition, each artist record carries a spotify
field: a direct link to that artist's Spotify page, scraped from the venue's
own event page (no API key, no Spotify dependency). common/spotify.py owns
the extraction (a small family of venue markup patterns) plus the shared
merge_shows / stats / from_captures / fetch_spotify helpers each build
and fetch calls.
Coverage is honest: a show whose venue page publishes no Spotify link gets
spotify: null, exactly like an Apple Music not-found — we never guess a link.
| Venue | Spotify links on its pages? |
|---|---|
| Independent, Brick & Mortar, Cafe du Nord, Greek, Fox | yes — per-artist social buttons on the detail page |
| Castro | yes — per-attraction social row on the detail page |
| Regency | yes — the AEG feed's association records carry them (no extra fetch) |
| Fillmore, GAMH, August Hall, Warfield, Frost | no — the sites don't publish them; the registry turns on the Spotify API lookup (common/spotify_api) instead, so links still appear (spotify_source: api) |
| Chapel, Rickshaw Stop, the Belly Up group (six rooms), Pappy & Harriet's | no — same deal: pages publish none, Spotify API lookup covers them (honest nulls where the API finds nothing) |
mkdir <key>and writefetch*.pyto capture the schedule into<key>/cache/(Cloudflare-gated sites: capture detail pages via a real browser, as the Castro does).- Write
<key>/build.py: map each raw event intocommon.build.show(...)records, then callcommon.build.run_build(venue, key, venue_url, source_url, shows, stats). Shape the artists withcommon.apple_music.artist_obj(name, lookup)so every show carries Apple Music links — see the mandate below. - Register it in
common/venues.pywithhas_apple_music: True(one dict entry: name, city, dir, has_apple_music). - Apple Music links — REQUIRED for every new theater:
cd <key> && python3 ../common/lookup.py --full && python3 build.py. Matching is best-effort: the shared engine resolves whatever it can resolve and honestly marks the rest not-found. We do NOT try to code every possible billed name — a small number of unresolvable names (compound bills, film credits, odd spellings) is expected and acceptable. The goal is to get most, and thestats.apple_musicblock (exact / close / needs_review / not_found) makes the coverage visible and reviewable.
That's it — the picker card, the reviewer page, and the JSON envelope come
from common/ automatically.
python3 serve.py # ONE entry point: http://127.0.0.1:8050
python3 update_all.py # refresh every venue's data/shows.json
python3 update_all.py --full
update_all.py runs, per venue (registry order): fetch → build → Apple Music
lookup → build, then prints a diff vs the previous shows.json
(added / removed / schedule-changed / link-only). By default the site is
re-crawled only when the venue's cache is > 24h old (.fetched_at stamp;
--refresh forces it), and Apple Music is incremental — common/lookup.py
skips artists that already have a terminal status in the venue's
cache/lookup.json (and retries error entries). --full forces a complete
Apple Music re-match (and a re-crawl). --venues a,b restricts the run. A
venue whose fetch fails is skipped with a warning instead of blocking the rest.
/ is the theater picker; /independent/ and /castro/ are the reviewers
(shared template, per-venue data injected). Data is re-read per request, so a
fresh build.py shows up immediately.
Every theater gets Apple Music links on every show where possible. When
adding a new theater, run the shared lookup (common/lookup.py) over its
artists — matching is on for all venues, not opt-in.
Matching is best-effort by design: we cannot code every possible billed
name, and the engine is honest rather than clever. A name it cannot
confidently resolve is marked not-found (or flagged needs_review for a close
near-miss) rather than force-linked. A handful of unresolvable names per
venue is expected and fine — the success bar is "get most," not 100%. The
stats.apple_music block (unique / found / exact / close / needs_review /
not_found) reports the real coverage so the reviewer can see what didn't
resolve.
Two safety rules guard the "get most" goal from becoming "get wrong":
LINK_FLOOR0.80 (incommon/apple_music.py): a fuzzy (non-exact) match below confidence 0.80 is never auto-linked — the artist is marked not-found but the near-miss name + score are retained for human review.- Compound bills: a billed "A and B" / "A x B" that matches only one act is linked to that act but flagged "compound bill — links to … only", never silently presented as a full match.
Both rules live in exactly one place and apply to every venue that matches.
Each venue may carry one representative building photo as <venue_id>.jpg
(>= 750 px on both sides, any aspect ratio) in its folder. publish-data.sh
copies the jpg + its CREDITS.json attribution sidecar into the data repo,
where the nightly push ships them to HeadlinerListings. No manifest/schema
change: the images are plain files beside shows.json.
The whole workflow lives in common/venue_images.py (Pillow from
.venv-images/):
.venv-images/bin/python common/venue_images.py find # proposals + report
.venv-images/bin/python common/venue_images.py apply # save approved images
.venv-images/bin/python common/venue_images.py audit # verify stored jpgs
- Source of truth: Wikimedia Commons only. The feed repo is public and
the app displays these, so every image must be free-licensed; venue-site
and blog photos are never scraped.
applyre-verifies the license at download time and records author + license inCREDITS.jsonnext to the jpg (CC-BY-SA requires attribution wherever the image is displayed). - Identity before aesthetics. Auto-propose fires only on venue-certain
candidates — files under a Commons category that names the venue, or
geotagged within ~300 m of the building AND file-named for it. Namesake
traps ("David Warfield", "Belly Up" Chicago, Fillmore Street) are the
reason bare filename matches never auto-pass. Concert/stage shots are
fine candidates — the venue's identity must be certain; the subject may
be the room mid-show. Everything else lands in
common/venue_images_report.mdfor a human pick: setfile(andapproved) incommon/venue_images.json, then runapply. - No qualifying image => no image. A venue with nothing >= 750 px saves nothing and stays on the report list — never an upscale, never a smaller-file compromise.