Gilmore Girls Breakdown: v3 (with TVTropes layer)

v3 inherits everything in v2 and adds a TVTropes harvest layer. Same puller as Community v3; one shared process handles both shows so the request pattern reads as cross-interest browsing rather than a per-series scrape. Cloudflare managed challenge is cleared by WARP egress plus a full Chrome header set.

What v3 adds

The shared self-discovering puller seeds at Series/GilmoreGirls, discovers in-show namespaces (Recap / Characters / Heartwarming / Awesome / Funny / Tearjerker / NightmareFuel / YMMV / WMG / Trivia / Fridge / Headscratchers / Quotes / Analysis / FanWorks), randomizes the queue across both shows, sleeps 25-75 seconds between fetches with occasional 5-15 minute long pauses.

• Puller: tvtropes_pull.py
• Output root: Projects/tvtropes-pull/
• Gilmore Girls raw HTML lands in: Projects/tvtropes-pull/raw/ (filenames start with Series__GilmoreGirls, Recap__GilmoreGirls, Characters__GilmoreGirls, etc.)
• Extracted text: Projects/tvtropes-pull/text/
• Resume state: state.json
• Per-fetch log: pull.log.jsonl
• Stdout: pull.stdout.log
• Running PID: puller.pid

Operating model

Same single process as Community v3. The interleave is intentional: it raises the realism of the traffic pattern and amortizes the inter-request delay across both harvests.

• Cloudflare bypass = WARP egress + realistic Chrome headers. No JS solver needed at this pace.
• On 403 / 429 / 503 the puller backs off 60s, 180s, 600s before giving up on a URL.
• Files are atomic-saved; resume after a kill picks the queue back up.

Stop / resume / inspect

• Inspect progress: tail -f ~/Projects/tvtropes-pull/pull.stdout.log
• Stop: kill $(cat ~/Projects/tvtropes-pull/puller.pid)
• Resume: re-run the launch command (see Community v3 for the exact invocation)

Integration with v2

The pulled TVTropes text becomes a third evidence source for the Episode Breakdown Bible / Gilmore-Girls.md voice. The voice already cites the pilot script PDF and the Forever Dreaming transcript; the TVTropes Recap and Characters subpages add a tropes-vocabulary layer for cross-referencing recurring patterns by name.

• Episode Breakdown Bible (target for merge): Gilmore-Girls.md
• Mirror viewer (target for merge): gilmore-girls-mirrors-v2.html

Inherited layers from v2

• v1 conversation: Gilmore v1
• v2 integrated index: Gilmore v2
• v2 covers: Episode Breakdown Bible Gilmore slice (51 messages), voice-building setup, both viewer HTMLs.

Operational notes

• First successful Gilmore fetch: Series/GilmoreGirls 480 KB at 19:38, 21 in-show sub-pages discovered on the spot.
• If raw files start showing "Just a moment" in their first 600 bytes, WARP rotated to a flagged egress IP. Fall back to playwright-stealth.
• Pacing: at 25-75s per page with occasional long pauses, ~50 pages per hour combined across both shows. Roughly half land per show. Full Gilmore harvest (Series + 7 seasons × ~22 Recap pages + Characters + Heartwarming + Funny + Tearjerker + NightmareFuel + YMMV + WMG + Trivia + Fridge + Headscratchers + Quotes + Analysis + FanWorks, roughly 200 pages) lands in 4-6 hours.
• Disk: each raw HTML averages ~250-500 KB; extracted text ~50-100 KB.

Provenance

• Shared puller built: 2026-06-02
• Egress: Cloudflare WARP, confirmed connected
• First successful Gilmore fetch: 480 KB at 19:38