Charting Platform Architecture & Improvement Plan

Architecture review — TradingView-style charting platform (broker hist + live WS) We’re building an Indian-markets charting platform (TradingView Advanced Charts on the front end) powered by a broker market-data API. I’d like your help as a senior engineer to review and propose a clean architecture for historical + live candle delivery, with a clear plan to fix the gaps we’re seeing in production. Product context (high level) Web charts for NSE equities / indices (and related F&O instruments). Front end: TradingView Charting Library (multi-timeframe candles, watchlist, symbol switch). Backend: our own API layer in front of the broker (we do not want the browser talking to the broker directly for hist/live). Sessions are IST market hours (e.g. 09:15–15:30), with overnight / weekend / holiday gaps that must be handled correctly on the chart. Current data model (two paths) Historical / REST Load past OHLC for a symbol + resolution (1m / 5m / higher TFs). Used for first paint, scroll-left (“load more history”), and cold symbols. Often a short “tip” window for fast load, plus deeper history behind it. Intraday live / WebSocket Broker (or our relay) streams ticks / LTP / partial bars for the current session. Front end updates the forming candle and commits bars as time buckets close. Problems we need solved These show up as bad UX on TradingView and are our main pain: Candle gaps — missing bars inside a session (holes in the series), or duplicate / overlapping bars when hist and live are stitched. Date / session gaps — wrong handling of day breaks, weekends, holidays; empty space vs collapsed sessions; bars landing outside session; “teleport” jumps when switching symbols or resolutions. Stitching bugs — hist edge vs live edge disagree (different last bar time/OHLC); race where live updates arrive before hist finishes; zoom/scroll reload fighting the live stream. TF consistency — 1m → 5m/15m aggregation must match what users expect; live forming bar must align to the same bucket rules as hist. Reliability — broker rate limits, reconnects, late ticks, and how we backfill after a WS drop without creating gaps or spikes. What I’m asking you to deliver Please treat this as an architecture + implementation plan engagement (not just opinions). Specifically: Target architecture diagram Client (TV datafeed) → our API/WS → cache/store → broker hist + broker live. Call out components (gateway, candle builder, session calendar, cache, backfill job, relay). Canonical candle model Bar time convention (open vs close of bucket, timezone, session calendar). How hist REST and live WS both produce the same bar identity (symbol + resolution + bucketStart). Rules for merge / upsert / replace when both sources touch the same bar. End-to-end flows First load (tip + deeper hist). Live subscribe + forming bar. Scroll-left history. Reconnect / gap fill after WS disconnect. Symbol / resolution change. Concrete fix plan for candle gaps & date gaps Detection (how we know a gap is missing vs intentional session break). Healing (backfill from broker vs derive from 1m warehouse). What the front-end datafeed must guarantee so TradingView doesn’t show holes or wrong day separators. Storage recommendation What should be durable vs hot cache (e.g. Redis for live tip, DB/files for deep hist), retention, and rebuild strategy. Risks & acceptance tests A short checklist we can run (e.g. “no missing 5m bar between 10:00–10:30 on a trading day”; “after kill WS for 2 min, chart heals without spike”; “weekend doesn’t invent Saturday candles”). Constraints Broker is the source of truth for prices; we can add our own aggregation/cache, but we shouldn’t invent OHLC. Prefer a design that stays broker-agnostic at the edges (swap provider later). Keep the TradingView datafeed contract clean (onReady / resolveSymbol / getBars / subscribeBars). Production must stay online; propose a phased migration if a rewrite is needed. Please reply with Your proposed architecture (diagram + 1–2 pages of prose). Any clarifying questions before you lock the design. Happy to share anonymized sequence diagrams, sample hist payloads, and WS message shapes on a call once you’ve read this. PLEASE NOTE WE NEED ONLY THE ARCHITECTURE PLAN OR CONSULTATION and not PROJECT DEVELOPMENT Thanks