Skip to main content
Looking for the system reference? See ARCHITECTURE.md for deployment topology, directory layout, caching layers, CI/CD, and contributor how-to guides.

​
Design Principles

​
Intelligence Analysis Tradecraft

The dashboard’s design draws from established intelligence analysis methodology, adapted for automated open-source intelligence: Structured Analytic Techniques (SATs) — rather than presenting raw data, the system applies structured frameworks to reduce cognitive bias. The Country Instability Index decomposes “instability” into four weighted components (unrest, conflict, security, information velocity) — forcing analysts to consider each dimension independently rather than anchoring on the most salient headline. The Strategic Risk Score similarly decomposes geopolitical risk into convergence, CII, infrastructure, theater, and breaking news components. Analysis of Competing Hypotheses (ACH) — composite intelligence products reduce single-source bias by corroborating independent inputs before they move a score: the Country Instability Index blends unrest, conflict, security, and information-velocity signals; protest data merges ACLED and GDELT; disaster data fuses USGS, GDACS, and NASA EONET. The breaking-news banner is deliberately different: the current code wires two independent producers, rss_alert and oref_siren, without a corroboration-count or cross-origin convergence requirement. A qualifying RSS item may raise a high or critical alert after recency, source-tier, story-phase, importance, deduplication, cooldown, startup, and user-sensitivity gates. A non-empty OREF siren batch escalates directly to critical when alerts are enabled and the event is not a duplicate; it bypasses the RSS-only gates and global cooldown. Intelligence gap awareness — professional intelligence assessments always note what they don’t know. The data freshness tracker explicitly reports “what can’t be seen” — 35 sources with status categorization (fresh, stale, very_stale, no_data, error, disabled). Two browser-tracker sources (GDELT, RSS) are flagged as requiredForRisk, meaning their absence can force the Strategic Risk panel into its hard insufficient-data state. That flag is narrower than the full CII scoring dependency graph: UCDP events, ACLED conflict, news summaries, cyber threats, and other score inputs are monitored through source-specific health plus the riskScores health/cache signal-density coverage. When a critical data source goes down, the system displays the gap prominently rather than silently omitting it, preventing false confidence from incomplete data. Source credibility weighting — the 4-tier source hierarchy (wire services → major outlets → specialty → aggregators) mirrors intelligence community source evaluation (A–F reliability, 1–6 confidence). State-affiliated sources are included for completeness but tagged with propaganda risk indicators, enabling analysts to factor in editorial bias. Higher-tier sources carry more weight in focal point detection and alert generation. Temporal context — Welford’s online baseline computation provides the temporal context that raw counts lack. “50 military flights” is meaningless without knowing that the average for this day of week and month is 15 — making the observation 3.3σ above normal. The system automatically provides this context for every signal type. Alert-channel separation — keyword spikes, hotspot escalation, and military surges contribute to correlation and anomaly products, but they are not wired as breaking-news banner producers. The banner presents qualifying RSS items and OREF sirens as separate alerts; it does not imply that those channels were fused into a corroborated assessment.

​
Algorithmic Design Decisions

Several non-obvious algorithmic choices are worth explaining: Logarithmic vs. linear protest scoring — Democracies experience routine protests that don’t indicate instability (France’s yellow vest movement, US campus protests). Authoritarian states rarely see public protest, so each event is significant. The CII uses log(protestCount) for democracies and linear scaling for authoritarian states, preventing democratic noise from drowning genuine authoritarian unrest signals. Welford’s online algorithm for baselines — Traditional mean/variance computation requires storing all historical data points. Welford’s method maintains a running mean and M2 (sum of squared deviations) that can be updated with each new observation in O(1) time and O(1) space. This makes it feasible to track baselines for hundreds of event-type × region × weekday × month combinations in Redis without storing raw observations. H3 hexagonal grid for GPS jamming — Hexagonal grids (H3 resolution 4, ~22km edge length) are used instead of rectangular lat/lon cells because hexagons have uniform adjacency (6 neighbors vs. 4/8 for squares), equal area at any latitude, and no meridian convergence distortion. This matters for interference zone detection where spatial uniformity affects clustering accuracy. Cosine-latitude-corrected distance — Cable health matching and several proximity calculations use equirectangular approximation with cos(lat) longitude correction instead of full Haversine. At the distances involved (50–600km), the error is <0.5% while being ~10x faster — important when computing distances against 500+ infrastructure assets per event. Negative caching — When an upstream API returns an error, the system caches the failure state for a defined period (5 minutes for UCDP, 30 seconds for Polymarket queue rejections) rather than retrying immediately. This prevents thundering-herd effects where hundreds of concurrent users all hammer a downed API, and it provides clear signal to the intelligence gap tracker that a source is unavailable. O(1) inflection suffix matching — The keyword-matching pipeline checks every word in every ingested headline against a set of English inflection suffixes (-ing, -ed, -tion, -ment, etc.) for morphological normalization. The suffix list was converted from an Array (O(n) .some() scan per word) to a Set (O(1) .has() lookup), eliminating a linear scan executed on every word of every headline — a meaningful hot-path optimization given the system processes thousands of headlines per refresh cycle. Stack-safe array operations — The Math.min(...array) and Math.max(...array) spread patterns are limited by V8’s argument stack (~65,535 entries). With large news clusters (common during breaking events), the spread silently overflows and returns Infinity / -Infinity, corrupting firstSeen and lastUpdated timestamps. These are replaced with Array.prototype.reduce loops that operate in O(1) stack space regardless of array size.

​
TypeScript Architecture

​
Vanilla TypeScript Architecture

World Monitor is written in vanilla TypeScript — no frontend framework (React, Vue, Svelte, Angular) is used. This is a deliberate architectural decision, not an oversight. Why no framework:
  • Bundle size — the dashboard loads dozens of data layers, map renderers, ML models, and live video streams. Every kilobyte of framework overhead competes with actual intelligence data. The entire application shell (panel system, routing, state management) compiles to less JavaScript than React’s runtime alone
  • DOM control — the panel system manipulates innerHTML directly with debounced content replacement (setContent()) and event delegation on stable container elements. Framework virtual DOM diffing would fight this pattern, adding overhead without benefit — the dashboard doesn’t have the fine-grained reactive state updates that frameworks optimize for
  • WebView compatibility — the Tauri desktop app runs in WKWebView (macOS) and WebKitGTK (Linux), which have idiosyncratic behavior around drag-and-drop, clipboard, autoplay, and memory management. Direct DOM manipulation makes it possible to work around these platform quirks without fighting framework abstractions
  • Long-term simplicity — no framework version upgrades, no breaking API migrations, no adapter libraries. The codebase depends on browser standards (DOM, Web Workers, IndexedDB, Intersection Observer, ResizeObserver) that are stable across engine updates
What fills the framework gap:

​
Discriminated Union Marker System

All map markers — across both the globe.gl and deck.gl engines — carry a _kind discriminant field that identifies their type at runtime. Rather than using class inheritance (which requires instanceof checks and prevents marker data from being plain serializable objects), each marker is a plain TypeScript object with a literal _kind string:
This enables exhaustive switch matching in the rendering pipeline — the TypeScript compiler verifies that every marker kind is handled, and adding a new kind produces compile errors at every unhandled site. Marker data can be serialized to/from JSON (for IndexedDB persistence and Web Worker transfer) without custom serialization logic. The same marker objects flow through clustering, tooltip generation, and layer filtering without type casting.

​
Panel Event Delegation Pattern

The Panel base class uses a debounced setContent(html) method (150ms delay) to batch rapid DOM updates. This creates a subtle but critical problem: any event listeners attached to elements inside the panel’s innerHTML are destroyed when the debounce fires and replaces the content. The solution is event delegation — all click, change, and input handlers are bound to the stable outer this.content container element (which is never replaced, only its innerHTML changes), using event.target.closest('.selector') to match the intended element:
This pattern is enforced project-wide across all panel subclasses. In E2E tests, element references also go stale after the debounced render — test code must re-query the DOM after each render cycle rather than holding onto cached element references.

​
API & Data Pipeline

CORS — all API endpoints enforce an origin allowlist. See CORS.md for the allowed origins, implementation details, and how to add CORS to new endpoints.

​
Proto-First API Contracts

The entire API surface is defined in Protocol Buffer (.proto) files using sebuf HTTP annotations. Code generation produces TypeScript clients, server handler stubs, and OpenAPI 3.1.0 documentation from a single source of truth — eliminating request/response schema drift between frontend and backend. 35 service domains cover every data vertical: Code generation pipeline — a Makefile drives buf generate with three custom sebuf protoc plugins:
  1. protoc-gen-ts-client → typed fetch-based client classes (src/generated/client/)
  2. protoc-gen-ts-server → handler interfaces and route descriptors (src/generated/server/)
  3. protoc-gen-openapiv3 → OpenAPI 3.1.0 specs in YAML and JSON (docs/api/)
Proto definitions include buf.validate field constraints (e.g., latitude ∈ [−90, 90]), so request validation is generated automatically — handlers receive pre-validated data. Breaking changes are caught at CI time via buf breaking against the main branch. Edge gateway — proto-backed domain entry points (api/[domain]/v1/[rpc].ts) register their generated createServiceRoutes() descriptors with a flat path router. Each RPC uses the HTTP method declared in its proto/OpenAPI contract: most read endpoints are GET (for example, GET /api/aviation/v1/list-airport-delays), while mutating and batch operations use POST. The shared gateway adds CORS enforcement, a top-level error boundary that hides internal details on 5xx responses, and rate-limit support (retryAfter on 429). The same router runs locally via a Vite dev-server plugin (sebufApiPlugin in vite.config.ts) with HMR invalidation on handler changes.

​
Bootstrap Hydration

The dashboard eliminates cold-start latency by pre-fetching 38 commonly needed datasets in a single Redis pipeline call before any panel renders. On page load, the client fires two parallel requests — a fast tier and a slow tier — to the /api/bootstrap edge function, both with an 800ms abort timeout to avoid blocking first paint.
The edge function reads all keys in a single Upstash Redis pipeline — one HTTP round-trip for up to 38 keys. Results are stored in an in-memory hydrationCache Map. When panels initialize, they call getHydratedData(key) which returns the pre-fetched data and evicts it from the cache (one-time read) to free memory. Panels that find hydrated data skip their initial API call entirely, rendering instantly with pre-loaded content. Panels that mount after the hydration data has been consumed fall back to their normal fetch cycle. Negative sentinel caching — when a Redis key contains no data, the bootstrap endpoint stores a __WM_NEG__ sentinel in the response rather than omitting the key. This allows consumers to distinguish between “data not yet loaded” (key absent from hydration) and “data source has no content” (negative sentinel), preventing unnecessary RPC fallback calls for empty data sources. Per-tier CDN caching — the fast tier uses s-maxage=1200 (20 min) with stale-while-revalidate=300 for near-real-time data like earthquakes and market quotes. The slow tier uses s-maxage=7200 (2 hours) with stale-while-revalidate=1800 for infrequently changing data like BIS policy rates and climate anomalies. Both tiers include stale-if-error directives to serve cached responses when the origin is temporarily unreachable. Selective fetching — clients can request a custom subset of keys via ?keys=earthquakes,flightDelays,insights for targeted hydration, enabling partial bootstrap recovery when a specific panel needs re-initialization. This converts 38 independent API calls (each with its own DNS lookup, TLS handshake, and Redis round-trip) into exactly 2, cutting first-meaningful-paint time by 2–4 seconds on typical connections.

​
SmartPollLoop — Adaptive Data Refresh

The SmartPollLoop is the core refresh orchestration primitive used by all data-fetching panels. Rather than fixed-interval polling, it adapts to network conditions, tab visibility, panel visibility, and failure history: Adaptive behaviors:
  • Exponential backoff — consecutive failures multiply the poll interval by a configurable backoffMultiplier (default 2×), up to 4× the base interval. A single successful fetch resets the multiplier to 1×
  • Hidden-tab throttle — when document.visibilityState is hidden, the poll interval is multiplied by a hiddenMultiplier (default 5×). A panel polling every 60s in the foreground slows to every 5 minutes when the tab is backgrounded
  • Manual trigger — handle.triggerNow() forces an immediate poll regardless of the current interval, used when users explicitly request a refresh or when a related panel’s data changes
  • Attempt tracking — a consecutive failure counter feeds into circuit breaker integration. After maxAttempts failures, the poll loop stops entirely and the circuit breaker serves cached data
  • Reason tagging — each poll carries a SmartPollReason ('interval', 'resume', 'manual', 'startup') so handlers can adjust behavior (e.g., startup polls may fetch larger datasets)
Panel integration — panels create a SmartPollLoop in their constructor with their base interval and callback, call handle.start() on mount, and handle.stop() on destroy. The loop is paused automatically when the panel is collapsed or scrolled out of view (via Intersection Observer), and resumed when it reappears.

​
Railway Seed Data Pipeline

21 Railway cron jobs continuously refresh the Redis cache with pre-computed data from external APIs. Seeds run on configurable schedules from minutes to daily cadence and write both a canonical domain key (for RPC handler lookups) and a bootstrap key (for page-load hydration). This dual-key strategy ensures that bootstrap hydration and RPC handlers always agree on data format and freshness. Seeds use cachedFetchJson with in-flight promise coalescing — if a seed run overlaps with a previous run still writing, the concurrent write is deduplicated. Each seed script is self-contained (single .mjs file, no build step), runs on Node.js 20+, and connects to Upstash Redis via REST API. Failed seed runs log errors but never corrupt existing cached data — the previous cache entry persists until a successful run replaces it.

​
Edge Functions & Deployment

​
Edge Function Architecture

World Monitor uses 60+ Vercel Edge Functions as a lightweight API layer, split into two generations. Legacy endpoints in api/*.js each handle a single data source concern — proxying, caching, or transforming external APIs. The newer proto-first endpoints use per-domain thin entry points — 34 separate edge functions, each importing only its own handler module. This replaced the original monolithic gateway that loaded all 34 domains on every cold start. Each domain’s function tree-shakes to include only its dependencies, reducing cold-start time by ~85% (sub-100ms for most endpoints vs. 500ms+ with the monolithic handler). A shared server/gateway.ts provides common routing logic. Both generations coexist, with new features built proto-first. This architecture avoids a monolithic backend while keeping API keys server-side:
  • RSS Proxy — domain-allowlisted proxy for 500+ feeds, preventing CORS issues and hiding origin servers. The Edge proxy and Railway relay both keep redirects manual and re-check the RSS domain allowlist before every hop. Feeds from domains that block Vercel IPs are automatically routed through the Railway relay.
  • AI Pipeline — Groq and OpenRouter edge functions with Redis deduplication, so identical headlines across concurrent users only trigger one LLM call. The classify-event endpoint pauses its queue on 500 errors to avoid wasting API quota.
  • Data Adapters — GDELT, ACLED, OpenSky, USGS, NASA FIRMS, FRED, Yahoo Finance, CoinGecko, mempool.space, BIS, WTO, and others each have dedicated edge functions that normalize responses into consistent schemas
  • Market Intelligence — macro signals, ETF flows, and stablecoin monitors compute derived analytics server-side (VWAP, SMA, peg deviation, flow estimates) and cache results in Redis
  • Temporal Baseline — Welford’s algorithm state is persisted in Redis across requests, building statistical baselines without a traditional database
  • Custom Scrapers — sources without RSS feeds (FwdStart, GitHub Trending, tech events) are scraped and transformed into RSS-compatible formats
  • Finance Geo Data — stock exchanges (29), financial centers (19), central-bank and supranational finance institutions (14), and commodity hubs (10) are served as static typed datasets with market caps, GFCI rankings, trading hours, and commodity specializations
  • BIS Integration — policy rates, real effective exchange rates, and credit-to-GDP ratios from the Bank for International Settlements, cached with 30-minute TTL
  • WTO Trade Policy — trade restrictions, tariff trends, bilateral trade flows, and SPS/TBT barriers from the World Trade Organization
  • Supply Chain Intelligence — maritime chokepoint disruption scores (cross-referencing NGA warnings + AIS data), FRED shipping freight indices with spike detection, and critical mineral supply concentration via Herfindahl-Hirschman Index analysis
  • Company Enrichment — IntelligenceService.GetCompanyEnrichment aggregates GitHub organization data, inferred tech stack (derived from repository language distributions weighted by star count), SEC EDGAR public filings (10-K, 10-Q, 8-K), and Hacker News mentions into a single response. IntelligenceService.ListCompanySignals surfaces real-time company activity signals — funding events, hiring surges, executive changes, and expansion announcements — sourced from Hacker News and GitHub, each classified by signal type and scored for strength based on engagement, comment volume, and recency
All edge functions include circuit breaker logic and return cached stale data when upstream APIs are unavailable, ensuring the dashboard never shows blank panels.

​
Cold-Start Optimization — Per-Domain Edge Function Split

The original monolithic edge gateway (api/[domain]/v1/[rpc].ts) imported every service domain handler into a single function. When any RPC was called, the edge runtime loaded the entire handler graph — initializing Redis clients, parsing configuration, and importing utility modules for every domain even though only 1 was needed. This was split into per-domain thin entry points — one per service domain — each importing only its own handler module. The shared gateway (server/gateway.ts) provides common routing logic, but each domain’s edge function tree-shakes to include only its dependencies. Impact: Cold-start time dropped by ~85% — a market quote request no longer loads the cyber threat intelligence parser, the OREF alert handler, or the climate anomaly detector. On Vercel’s edge runtime, this translates to sub-100ms cold starts for most endpoints, compared to 500ms+ with the monolithic handler.

​
Single-Deployment Variant Consolidation

All six dashboard variants (World Monitor, Tech Monitor, Finance Monitor, Commodity Monitor, Happy Monitor, Energy Monitor) serve from a single Vercel deployment. The variant is determined at runtime by hostname detection: On the desktop app, the variant is stored in localStorage['worldmonitor-variant'] and can be switched without rebuilding. The variant selector in the header bar navigates between deployed domains on the web or toggles the localStorage value on desktop. This architecture replaced the original multi-deployment approach (separate Vercel projects per variant) and provides several advantages:
  • Instant switching — users toggle variants in the header bar without a full page navigation or DNS lookup
  • Shared CDN cache — the static SPA assets are identical across variants; only runtime configuration differs. CDN cache hit rates are 4× higher than with separate deployments
  • Single CI pipeline — one build, one deployment, one set of edge functions. No cross-deployment configuration drift
  • Social bot routing — the OG image endpoint generates variant-specific preview cards based on the requesting hostname, so sharing a Tech Monitor link produces tech-branded social previews

​
Real-Time Systems

​
AIS Relay Backpressure Architecture

The AIS vessel tracking relay maintains a persistent WebSocket connection to AISStream.io that can deliver hundreds of position reports per second during peak maritime traffic. Without flow control, a slow consumer (e.g., a client on a poor network) would cause unbounded memory growth in the relay’s message queue. The relay implements a three-watermark backpressure system: Additionally, the relay caps the total tracked vessel count at 20,000 positions (the most recent position per MMSI). A secondary density cell system aggregates positions into 2°×2° geographic grid cells (max 5,000 cells) for overview visualization when the full vessel list exceeds rendering capacity. Vessel history trails are capped at 30 position points per vessel. When a new position arrives, the oldest trail point is evicted. This creates a “comet tail” visualization showing recent movement direction without unbounded memory growth. The relay also implements HMAC authentication between the frontend and relay server, preventing unauthorized clients from consuming the expensive AIS data feed.

​
ONNX Runtime Capability Detection

The browser-side ML pipeline (embeddings, NER, sentiment, summarization) uses ONNX Runtime Web for inference. Model execution speed varies dramatically across browsers and devices depending on available hardware acceleration. The system uses a cascading capability detection strategy at initialization:
  1. WebGPU — checked via navigator.gpu presence. Provides GPU-accelerated inference with the lowest latency. Available in Chrome 113+ and Edge 113+
  2. WebGL — fallback when WebGPU is unavailable. Uses the existing GPU via WebGL compute shaders. Available in all modern browsers
  3. WASM + SIMD — CPU-only fallback. SharedArrayBuffer and WASM SIMD availability are probed. SIMD provides ~2–4x speedup over plain WASM for vector operations
A deviceMemory API guard excludes the ML pipeline entirely on low-memory devices (mobile phones with <4GB RAM), preventing out-of-memory crashes from loading 384-dimensional float32 embedding models alongside the map renderer and live video streams.

​
Map & Visualization

​
Geopolitical Boundary Overlays

The map supports typed geopolitical boundary polygons with associated metadata. Each boundary carries a boundaryType discriminant (demilitarized, ceasefire, disputed, armistice) that controls rendering style and popup content. Korean DMZ — the first boundary implemented is the Korean Demilitarized Zone, defined as a 43-point closed-ring polygon derived from OpenStreetMap Way 369265305 and the Korean Armistice Agreement Article I demarcation line. On the flat map, it renders as a GeoJsonLayer with a translucent blue fill and labeled tooltip. On the 3D globe, it renders as polygonsData under the conflicts layer. The boundary has a dedicated help entry and layer toggle, and is enabled by default on the full variant only. The boundary system is designed to be extensible — additional geopolitical boundaries (Line of Control in Kashmir, Golan Heights, Northern Cyprus Green Line) can be added to the GEOPOLITICAL_BOUNDARIES constant with appropriate typing and will render automatically on both map engines.

​
CII Choropleth Heatmap

The Country Instability Index can be projected as a full-coverage choropleth layer on both map engines, painting every country’s polygon in a five-stop color gradient based on its live CII score (0–100): On the flat map (deck.gl), a GeoJsonLayer maps ISO 3166-1 alpha-2 country codes to fixed RGBA values via the getLevel() threshold function. Updates are triggered by a monotonic version counter (ciiScoresVersion) — the layer compares the counter on each render pass and only recomputes fill colors when it increments, avoiding O(n) data spreads. On the 3D globe (globe.gl), CII country polygons merge into the same polygonsData array as geopolitical boundaries. A _kind discriminant ('boundary' | 'cii') in each polygon object lets a single .polygonCapColor() callback dispatch rendering logic for both types. CII polygons render at polygonAltitude: 0.002 (below the 0.006 altitude used by conflict-zone outlines), preventing visual Z-fighting. Countries GeoJSON is lazy-loaded from a shared getCountriesGeoJson() function, cached after first fetch, and shared between the CII layer and the country-detection ray-casting service.

​
Unified Layer Toggle Catalog

All 56 map layer toggle definitions — icon, localization key, fallback display label, and supported renderer types — are consolidated in a single shared registry (src/config/map-layer-definitions.ts). Each entry declares which map renderers support it via a renderers: MapRenderer[] field (e.g., dayNight is flat-only, ciiChoropleth is both flat and globe), preventing the two map components from showing inconsistent layer options. A def() factory function reduces per-entry boilerplate. Variant-specific layer ordering (VARIANT_LAYER_ORDER) defines the display sequence for each of the six dashboard variants without duplicating the definitions themselves. Adding a new map layer requires a single registry entry — both the flat map and 3D globe derive their toggle panels from this catalog automatically. The same registry also carries the v1 layer-explanation catalog for high-adoption layers. Each curated explanation records purpose, provider/source, freshness expectations, confidence notes, limitations, related panels/actions, and grounding paths; unsupported layers intentionally fall back to a generic card so the UI never implies a fabricated freshness or confidence value.

​
Bandwidth & Caching

​
Vercel CDN Headers

Every API edge function includes Cache-Control headers that enable Vercel’s CDN to serve cached responses without hitting the origin: Static assets use content-hash filenames with 1-year immutable cache headers. The service worker file (sw.js) is never cached (max-age=0, must-revalidate) to ensure update detection.

​
Client-Side Circuit Breakers

Every data-fetching panel uses a circuit breaker that prevents cascading failures from bringing down the entire dashboard. The pattern works at two levels: Per-feed circuit breakers (RSS) — each RSS feed URL has an independent failure counter. After 2 consecutive failures, the feed enters a 5-minute cooldown during which no fetch attempts are made. The feed automatically re-enters the pool after the cooldown expires. This prevents a single misconfigured or downed feed from consuming fetch budget and slowing the entire news refresh cycle. Per-panel circuit breakers (data panels) — panels that fetch from API endpoints use IndexedDB-backed persistent caches (worldmonitor_persistent_cache store) with TTL envelopes. When a fetch succeeds, the result is stored with an expiration timestamp. On subsequent loads, the circuit breaker serves the cached result immediately and attempts a background refresh. If the background refresh fails, the stale cached data continues to display — panels never go blank due to transient API failures. Cache entries survive page reloads and browser restarts. The circuit breaker degrades gracefully across storage tiers: IndexedDB (primary, up to device quota) → localStorage fallback (5MB limit) → in-memory Map (session-only). When device storage quota is exhausted (common on mobile Safari), a global _storageQuotaExceeded flag disables all further writes while reads continue normally.

​
Brotli Pre-Compression (Build-Time)

vite build now emits pre-compressed Brotli artifacts (*.br) for static assets larger than 1KB (JS, CSS, HTML, SVG, JSON, XML, TXT, WASM). This reduces transfer size by roughly 20–30% vs gzip-only delivery when the edge can serve Brotli directly. For the Hetzner Nginx origin, enable static compressed file serving so dist/*.br files are returned without runtime recompression:
Cloudflare will negotiate Brotli automatically for compatible clients when the origin/edge has Brotli assets available.

​
Railway Relay Compression

All relay server responses pass through gzipSync when the client accepts gzip and the payload exceeds 1KB. Sidecar API responses prefer Brotli and use gzip fallback with proper Content-Encoding/Vary headers for the same threshold. This applies to OpenSky aircraft JSON, RSS XML feeds, UCDP event data, AIS snapshots, and health checks — reducing wire size by approximately 50–80%.

​
In-Flight Request Deduplication

When multiple connected clients poll simultaneously (common with the relay’s multi-tenant WebSocket architecture), identical upstream requests are deduplicated at the relay level. The first request for a given resource key (e.g., an RSS feed URL or OpenSky bounding box) creates a Promise stored in an in-flight Map. All concurrent requests for the same key await that single Promise rather than stampeding the upstream API. Subsequent requests are served from cache with an X-Cache: DEDUP header. This prevents scenarios like 53 concurrent RSS cache misses or 5 simultaneous OpenSky requests for the same geographic region — all resolved by a single upstream fetch.

​
Adaptive Refresh Scheduling

Rather than polling at fixed intervals, the dashboard uses an adaptive refresh scheduler that responds to network conditions, tab visibility, and data freshness:
  • Exponential backoff on failure — when a refresh fails or returns no new data, the next poll interval doubles, up to a maximum of 4× the base interval. A successful fetch with new data resets the multiplier to 1×
  • Hidden-tab throttle — when document.visibilityState is hidden, all poll intervals are multiplied by 10×. A tab polling every 60 seconds in the foreground slows to every 10 minutes in the background, dramatically reducing wasted requests from inactive tabs
  • Jitter — each computed interval is randomized by ±10% to prevent synchronized API storms when multiple tabs or users share the same server. Without jitter, two tabs opened at the same time would poll in lockstep indefinitely
  • Stale flush on visibility restore — when a hidden tab becomes visible, the scheduler identifies all refresh tasks whose data is older than their base interval and re-runs them immediately, staggered 150ms apart to avoid a request burst. This ensures users returning to a background tab see fresh data within seconds
  • In-flight deduplication — concurrent calls to the same named refresh are collapsed; only one is allowed in-flight at a time
  • Conditional registration — refresh tasks can include a condition function that is evaluated before each poll; tasks whose conditions are no longer met (e.g., a panel that has been collapsed) skip their fetch cycle entirely

​
Frontend Polling Intervals

Panels refresh at staggered intervals to avoid synchronized API storms: All animations and polling pause when the tab is hidden or after 2 minutes of inactivity, preventing wasted requests from background tabs.

​
Caching Architecture

Every external API call passes through a three-tier cache with stale-on-error fallback:
Cache keys are versioned (opensky:v2:lamin=..., macro-signals:v2:default) so schema changes don’t serve stale formats. Every response includes an X-Cache header (HIT, REDIS-HIT, MISS, REDIS-STALE, REDIS-ERROR-FALLBACK) for debugging. Shared caching layer — all sebuf handler implementations share a unified Upstash Redis caching module (_upstash-cache.js) with a consistent API: getCachedOrFetch(cacheKey, ttlSeconds, fetchFn). This eliminates per-handler caching boilerplate and ensures every RPC endpoint benefits from the three-tier strategy. Cache keys include request-varying parameters (e.g., requested symbols, country codes, bounding boxes) to prevent cache contamination across callers with different inputs. On desktop, the same module runs in the sidecar with an in-memory + persistent file backend when Redis is unavailable. In-flight promise deduplication — the cachedFetchJson function in server/_shared/redis.ts maintains an in-memory Map<string, Promise> of active upstream requests. When a cache miss occurs, the first caller’s fetch creates and registers a Promise in the map. All concurrent callers for the same cache key await that single Promise rather than independently hitting the upstream API. This eliminates the “thundering herd” problem where multiple edge function instances simultaneously race to refill an expired cache entry — a scenario that previously caused 50+ concurrent upstream requests during the ~15-second refill window for popular endpoints. Negative caching — when an upstream API returns an error, the system caches a sentinel value (__WM_NEG__) for 120 seconds rather than leaving the cache empty. This prevents a failure cascade where hundreds of concurrent requests all independently discover the cache is empty and simultaneously hammer the downed API. The negative sentinel is transparent to consumers — cachedFetchJson returns null for negative-cached keys, and panels fall back to stale data or show an appropriate empty state. Longer negative TTLs are used for specific APIs: UCDP uses 5-minute backoff, Polymarket queue rejections use 30-second backoff. The AI summarization pipeline adds content-based deduplication: headlines are hashed and checked against Redis before calling Groq, so the same breaking news viewed by 1,000 concurrent users triggers exactly one LLM call.

​
Error Tracking

​
Sentry Error Noise Filtering

The Sentry SDK initialization includes a beforeSend hook and ignoreErrors list that suppress known unactionable error sources — Three.js WebGL traversal crashes occurring entirely in minified code with no source-mapped frames, cross-origin Web Worker construction failures from browser extensions, iOS media element crashes, and jQuery $ injection by extensions. The Three.js filter specifically avoids blanket suppression: it only drops events where all stack frames are anonymous or from the minified bundle. If even one frame has a source-mapped .ts filename, the event is kept for investigation.

​
Error Tracking & Production Hardening

Sentry captures unhandled exceptions and promise rejections in production, with environment-aware routing (production on worldmonitor.app, preview on *.vercel.app, disabled on localhost and Tauri desktop). The configuration includes 30+ ignoreErrors patterns that suppress noise from:
  • Third-party WebView injections — Twitter, Facebook, and Instagram in-app browsers inject scripts that reference undefined variables (CONFIG, currentInset)
  • Browser extensions — Chrome/Firefox extensions that fail importScripts or violate CSP policies
  • WebGL context loss — transient GPU crashes in MapLibre/deck.gl that self-recover
  • iOS Safari quirks — IndexedDB connection drops on background tab kills, NotAllowedError from autoplay policies
  • Network transients — TypeError: Failed to fetch, TypeError: Load failed, TypeError: cancelled
  • MapLibre internal crashes — null-access in style layers, light, and placement that originate from the map chunk
A custom beforeSend hook provides second-stage filtering: it suppresses single-character error messages (minification artifacts), Importing a module script failed errors from browser extensions (identified by chrome-extension: or moz-extension: in the stack trace), and MapLibre internal null-access crashes when the stack trace originates from map chunk files. Chunk reload guard — after deployments, users with stale browser tabs may encounter vite:preloadError events when dynamically imported chunks have new content-hash filenames. The guard listens for this event and performs a one-shot page reload, using sessionStorage to prevent infinite reload loops. If the reload succeeds (app initializes fully), the guard flag is cleared. This recovers gracefully from stale-asset 404s without requiring users to manually refresh. Storage quota management — when a device’s localStorage or IndexedDB quota is exhausted (common on mobile Safari with its 5MB limit), a global _storageQuotaExceeded flag disables all further write attempts across both the persistent cache (IndexedDB + localStorage fallback) and the utility saveToStorage() function. The flag is set on the first DOMException with name === 'QuotaExceededError' or code === 22, and prevents cascading errors from repeated failed writes. Read operations continue normally — cached data remains accessible, only new writes are suppressed. Transactions are sampled at 10% to balance observability with cost. Release tracking (worldmonitor@{version}) enables regression detection across deployments.

​
Fault Tolerance

External APIs are unreliable. Rate limits, outages, and network errors are inevitable. The system implements circuit breaker patterns to maintain availability.

​
Circuit Breaker Pattern

Each external service is wrapped in a circuit breaker that tracks failures:
Behavior during cooldown:
  • New requests return cached data (if available)
  • UI shows “temporarily unavailable” status
  • No API calls are made (prevents hammering)

​
Protected Services

RSS feeds use per-feed circuit breakers—one failing feed doesn’t affect others.

​
Graceful Degradation

When a service enters cooldown:
  1. Cached data continues to display (stale but available)
  2. Status panel shows service health
  3. Automatic recovery when cooldown expires
  4. No user intervention required

​
System Health Monitoring

The status panel (accessed via the health indicator in the header) provides real-time visibility into data source status and system health.

​
Health Indicator

The header displays a system health badge: Click the indicator to expand the full status panel.

​
Data Source Status

The status panel lists all data feeds with their current state:

​
Per-Feed Information

Each feed entry shows:
  • Source name - The data provider
  • Last update - Time since last successful fetch
  • Next refresh - Countdown to next scheduled fetch
  • Cooldown remaining - Time until circuit breaker resets (if in cooldown)

​
Why This Matters

External APIs are unreliable. The status panel helps you understand:
  • Data freshness - Is the news feed current or stale?
  • Coverage gaps - Which sources are currently unavailable?
  • Recovery timeline - When will failed sources retry?
This transparency enables informed interpretation of the dashboard data.

​
Data Freshness Tracking

Beyond simple “online/offline” status, the system tracks fine-grained freshness for each data source to indicate data reliability and staleness.

​
Freshness Levels

​
Source-Specific Thresholds

Each data source has calibrated freshness expectations:

​
Visual Indicators

The status panel displays freshness for each source:
  • Colored dot indicates freshness level
  • Time since update shows exact staleness
  • Next refresh countdown shows when data will update

​
Why This Matters

Understanding data freshness is critical for decision-making:
  • A “fresh” earthquake feed means recent events are displayed
  • A “stale” news feed means you may be missing breaking stories
  • A “critical” AIS stream means vessel positions are unreliable
This visibility enables appropriate confidence calibration when interpreting the dashboard.

​
Core vs. Optional Sources

Data sources are classified by their importance to risk assessment: The Strategic Risk Overview panel adapts its display based on core source availability:

​
Freshness-Aware Risk Assessment

The composite risk score is adjusted based on data freshness:
This prevents false “all clear” signals when the system actually lacks data to make that determination.

​
Conditional Data Loading

API calls are expensive. The system only fetches data for enabled layers, reducing unnecessary network traffic and rate limit consumption.

​
Layer-Aware Loading

When a layer is toggled OFF:
  • No API calls for that data source
  • No refresh interval scheduled
  • WebSocket connections closed (for AIS)
When a layer is toggled ON:
  • Data is fetched immediately
  • Refresh interval begins
  • Loading indicator shown on toggle button

​
Unconfigured Services

Some data sources require API keys (AIS relay, Cloudflare Radar). If credentials are not configured:
  • The layer toggle is hidden entirely
  • No failed requests pollute the console
  • Users see only functional layers
This prevents confusion when deploying without full API access.

​
Performance Optimizations

The dashboard processes thousands of data points in real-time. Several techniques keep the UI responsive even with heavy data loads.

​
Web Worker for Analysis

CPU-intensive operations run in a dedicated Web Worker to avoid blocking the main thread: The worker manager implements:
  • Lazy initialization: Worker spawns on first use
  • 10-second ready timeout: Rejects if worker fails to initialize
  • 30-second request timeout: Prevents hanging on stuck operations
  • Automatic cleanup: Terminates worker on fatal errors

​
Virtual Scrolling

Large lists (100+ news items) use virtualized rendering: Fixed-Height Mode (VirtualList):
  • Only renders items visible in viewport + 3-item overscan buffer
  • Element pooling—reuses DOM nodes rather than creating new ones
  • Invisible spacers maintain scroll position without rendering all items
Variable-Height Mode (WindowedList):
  • Chunk-based rendering (10 items per chunk)
  • Renders chunks on-scroll with 1-chunk buffer
  • CSS containment for performance isolation
This reduces DOM node count from thousands to ~30, dramatically improving scroll performance.

​
Request Deduplication

Identical requests within a short window are deduplicated:
  • Market quotes batch multiple symbols into single API call
  • Concurrent layer toggles don’t spawn duplicate fetches
  • Promise.allSettled ensures one failing request doesn’t block others

​
Efficient Data Updates

When refreshing data:
  • Incremental updates: Only changed items trigger re-renders
  • Stale-while-revalidate: Old data displays while fetch completes
  • Delta compression: Baselines store 7-day/30-day deltas, not raw history

​
Cross-Module Integration

Intelligence modules don’t operate in isolation. Data flows between systems to enable composite analysis.

​
Data Flow Architecture

​
Module Dependencies

​
Alert Propagation

When a threshold is crossed:
  1. Source module generates alert (e.g., CII spike)
  2. Alert merges with related alerts (same country/region)
  3. Strategic Risk receives composite alert
  4. UI updates header badge and panel indicators
This ensures a single escalation (e.g., Ukraine military flights + protests + news spike) surfaces as one coherent signal rather than three separate alerts.

​
Service Status Monitoring

The Service Status panel tracks the operational health of external services that WorldMonitor users may depend on.

​
Monitored Services

​
Status Levels

​
Why This Matters

External service outages can affect:
  • AI summarization (Groq, OpenRouter outages)
  • Deployment pipelines (Vercel, GitHub outages)
  • API availability (Cloudflare, AWS outages)
Monitoring these services provides context when dashboard features behave unexpectedly.

​
Refresh Intervals

Different data sources update at different frequencies based on volatility and API constraints.

​
Polling Schedule

​
Real-Time Streams

AIS vessel tracking uses WebSocket for true real-time:
  • Connection: Persistent WebSocket to Railway relay
  • Messages: Position updates as vessels transmit
  • Reconnection: Automatic with exponential backoff (5s → 10s → 20s)

​
User Control

Time range selector affects displayed data, not fetch frequency: Historical filtering is client-side—all data is fetched but filtered for display.

​
Design Philosophy

Information density over aesthetics. Every pixel should convey signal. The dark interface minimizes eye strain during extended monitoring sessions. Panels are collapsible, draggable, and hideable—customize to show only what matters. Authority matters. Not all sources are equal. Wire services and official government channels are prioritized over aggregators and blogs. When multiple sources report the same story, the most authoritative source is displayed as primary. Correlation over accumulation. Raw news feeds are noise. The value is in clustering related stories, detecting velocity changes, and identifying cross-source patterns. A single “Broadcom +2.5% explained by AI chip news” signal is more valuable than showing both data points separately. Signal, not noise. Deduplication is aggressive. The same market move doesn’t generate repeated alerts. Signals include confidence scores so you can prioritize attention. Alert fatigue is the enemy of situational awareness. Knowledge-first matching. Simple keyword matching produces false positives. The entity knowledge base understands that AVGO is Broadcom, that Broadcom competes with Nvidia, and that both are in semiconductors. This semantic layer transforms naive string matching into intelligent correlation. Fail gracefully. External APIs are unreliable. Circuit breakers prevent cascading failures. Cached data displays during outages. The status panel shows exactly what’s working and what isn’t—no silent failures. Local-first. No accounts, no cloud sync. All preferences and history stored locally. The only network traffic is fetching public data. Your monitoring configuration is yours alone. Compute where it matters. CPU-intensive operations (clustering, correlation) run in Web Workers to keep the UI responsive. The main thread handles only rendering and user interaction.

​
News Grounding Pipeline

Every LLM-adjacent surface that consumes news (brief magazine description card, brief whyMatters analyst blurb, dashboard News summary, email digest, notification relay, MCP world-brief tool) grounds on the article’s cleaned RSS description so the model paraphrases what the article actually says instead of filling headline role-labels from its parametric priors.
Fallback rule everywhere: empty description → consumer behavior is byte-identical to headline-only. No regression path. The notification-relay audit (tests/notification-relay-payload-audit.test.mjs) enforces that RSS-origin producers set payload.description and domain-origin producers do not.