Refresh, Storage, and Snapshots
The current coordinated refresh pipeline, table roles, and snapshot lifecycle behind the public scanner.
This page is intentionally explicit and implementation-first. If the product or upstream APIs change, the current behavior described here can change with them.
Coordinated refresh cycle
The project runs a coordinated refresh job:
- scrape market and event data
- write a fresh market snapshot
- optionally run smart-money enrichment if the 6-hour interval has elapsed
The smart-money step only runs after a successful scrape.
Cadence
| Job | Current cadence | Purpose |
|---|---|---|
| Market scrape | Every 60 minutes | Refresh active market snapshot and tags |
| Smart-money enrichment | Every 6 hours | Refresh holders and wallet-PnL context |
Snapshot replacement rules
The scraper drops and rebuilds:
active_market_outcomesmarket_tags
This means each scrape creates a clean active snapshot instead of incrementally updating old rows.
Storage tables
| Table | Role |
|---|---|
active_market_outcomes | Main outcome-level market snapshot |
market_tags | Tag rows attached to market ids |
holders | Sampled holder rows keyed by condition id |
wallets_stats | Wallet-level PnL and alias data |
market_smart_money_stats | Background smart_money_win_rate storage and timestamp |
request_logs | API request timing logs in the metrics database |
How holders are refreshed
For each distinct condition_id in the active snapshot, PolyLab fetches holders, deletes prior holder rows for that condition, and inserts the new sampled rows with a fresh snapshot_at.
How wallet stats are refreshed
PolyLab extracts unique wallet addresses from sampled holder rows, fetches wallet PnL, and upserts into wallets_stats.
What survives between scrapes
The smart-money-related tables are not dropped on every market scrape. They are updated by the smart-money job and can therefore be newer or older than the market snapshot depending on job timing.
Why this architecture was chosen
The current design keeps the read path straightforward:
- the app reads from a compact snapshot table
- tag logic stays simple
- holder and wallet enrichment can be refreshed on a slower schedule
The cost is that PolyLab is a snapshot system, not a live state mirror.