Real-Time Market Data Infrastructure for Quant Desks: Feed Handlers, Time-Series Databases, and Point-in-Time Correctness in 2026

Every quant stack has a data layer, and every data layer has a ceiling. Signal quality cannot exceed data quality. Latency cannot beat the feed. A backtest trained on retroactively adjusted history is backtesting on future information. The market data infrastructure layer is not a commodity input — it is the constraint that determines which strategies are physically executable and which are not. A 10ms feed versus a 1ms feed is not a latency preference in cross-asset spread trading; it is the difference between a viable strategy and a guaranteed loser. What follows is a practitioner-level treatment of how production market data stacks are actually built at institutional desks.

Why Market Data Infrastructure Is the Alpha Foundation

Signal alpha is bounded by data alpha. A machine learning in quantitative finance pipeline that ingests a 50ms-stale consolidated feed will produce signals that are 50ms stale — regardless of model sophistication. For intraday mean-reversion strategies on correlated pairs, that staleness is catastrophic: by the time the signal fires, the arbitrage spread has already partially closed. The infrastructure layer determines which algorithmic trading strategies are even executable. Desks that treat market data as an afterthought — relying on a third-party aggregator, no gap detection, no normalization audit trail — are not competing with desks that own their data stack. They are competing in a different, slower market that institutional flow has already arbitraged down to zero.

The data layer also determines backtest validity. Every how to backtest discipline problem — lookahead bias, survivorship bias, split contamination — originates in data infrastructure failures, not model failures. A clean, point-in-time correct data layer is the prerequisite for every other quant function. It is not glamorous infrastructure. It is foundational infrastructure.

The Five Layers of a Production Market Data Stack

Feed Handlers

The feed handler is the first contact between your infrastructure and the exchange. The choice between direct exchange connections and consolidated tapes (SIP) is a latency decision with asymmetric consequences. Direct CME Globex connections deliver raw feed latency around 50μs. The SIP consolidated tape — which aggregates quotes from all 16 lit equity exchanges — adds 3–5ms of latency as the SIP processor normalizes and redistributes. Third-party vendors (Bloomberg, Refinitiv, ICE Data Services) sit another 5–50ms behind that, depending on delivery mechanism and co-location proximity.

The transport layer matters as much as the connection type. Direct exchange feeds for futures and options (CME Globex, ICE, CBOE) use UDP multicast, which is low-latency but lossy: packets are not retransmitted on loss. Feed handlers must implement sequence number gap detection — any discontinuity in the sequence number field triggers a retransmit request or snapshot recovery immediately. Gaps that go undetected for even a few seconds propagate stale quotes throughout the downstream pipeline. TCP feeds eliminate packet loss at the cost of retransmission latency; for strategies that can tolerate 1–5ms additional latency, TCP offers more operational simplicity. The production standard for sub-millisecond strategies is UDP multicast with kernel-bypass networking (DPDK or RDMA) to eliminate OS-level latency.

Normalization Layer

Raw exchange feeds are not interoperable. CME instruments are identified by contract codes and clearing codes. Nasdaq uses SIP symbology. Bloomberg tickers, CUSIPs, and ISINs are all different identification namespaces that must be mapped to a unified internal symbol scheme before any cross-venue logic can function. This mapping is not static: corporate actions — splits, dividends, spin-offs, ticker changes, exchange transfers — modify it continuously. A normalization layer that processes a 2-for-1 split incorrectly will double-count share quantities across the split date, producing a fake 2x return in every backtest that crosses that date. This is not an edge case; it is the single most common source of inflated backtest performance at shops that build their own data pipelines without rigorous corporate action handling.

Timezone normalization and exchange calendar management are equally critical for alternative data strategies and multi-asset trading. A signal that fires at 15:59:59 ET on the last trading day before a holiday needs to know whether the market closes early, whether the equity and futures close at different times, and whether settlement prices are available before or after the equity close. Exchange calendar management is not a lookup table — it is a stateful process that must handle ad hoc exchange closures, early closes, and session extensions in real time.

Time-Series Storage

Database selection for tick data is a genuine tradeoff with no universally correct answer. The relevant benchmarks across the major options:

• kdb+/q — fastest columnar storage for tick data in production; purpose-built for time-series with native temporal joins and the q query language. Benchmarks consistently show 5–10× query performance advantage over relational databases on tick queries. Drawbacks: expensive licensing ($15K–$40K/year per server), steep q language learning curve, limited open-source tooling. The institutional standard for execution-critical tick data at desks that can absorb the cost.
• ClickHouse — open-source columnar OLAP database with excellent OHLCV aggregation performance. Sub-second queries on billions of rows for bar construction and rolling statistics. Weaker on real-time streaming ingestion (designed for batch inserts), but excels for research and backtesting workloads. The practical choice for most institutional research stacks that cannot justify kdb+ licensing.
• TimescaleDB — Postgres extension for time-series. Strong operational familiarity for teams already on Postgres; reasonable performance on OHLCV queries with hypertable partitioning. Not competitive with kdb+ or ClickHouse on raw tick throughput, but the SQL interface reduces integration friction for research pipelines.
• Arctic — Python-native time-series store built on MongoDB. Excellent for research workflows (pandas DataFrames as first-class citizens), poor performance at institutional tick volumes. Use for strategy research; do not use for production tick storage.
• InfluxDB — easy operational setup, weak compression ratios on financial tick data compared to purpose-built columnar stores. Adequate for monitoring and infrastructure metrics; not the right choice for high-frequency market data.

The practical rule for institutional desks: kdb+ for execution-critical tick data where latency under 1ms matters; ClickHouse or TimescaleDB for research and backtesting workloads; S3 Parquet with columnar partitioning (by date and symbol) for historical archive. Each tier has a different cost profile and serves different query patterns. Collapsing everything into a single database is a false economy that produces either expensive kdb+ licenses for research queries or inadequate latency for execution.

Tick Data Schema Design

Schema decisions made on day one are architectural commitments that are expensive to reverse at scale. The critical design choices:

• Bid/ask vs. last trade separation — quote updates and trade prints must be stored in separate tables or streams. Conflating them produces nonsensical aggregations: volume calculations that include bid/ask updates, or spread calculations that include trade prices as quotes.
• Trade condition code filtering — not all reported trades belong in volume calculations. FINRA TRF (Trade Reporting Facility) trades — off-exchange prints from dark pools and internalizers — must be excluded from on-exchange volume and VWAP calculations. Odd-lot trades, late prints, and corrected trades each have condition codes that must be handled explicitly. A VWAP calculation that includes all reported trades is wrong.
• Quote stuffing detection — burst events where a single venue floods the tape with thousands of updates per second on a single symbol before canceling them within microseconds. These must be identified and flagged; they pollute latency metrics and can trigger false signals in order-book-based strategies.
• Post-close restatements — several exchanges publish corrected OHLCV data after the close (incorrect last sale prices, mis-stamped timestamps, auction corrections). The schema must support append-only writes with a restatement flag and an as-of timestamp, so downstream systems can choose whether to use the original or corrected data.

Derived Data Pipelines

Raw ticks are rarely the direct input to signal generation. Derived data pipelines transform raw ticks into the features that quantitative trading software consumes: real-time OHLCV bar construction (1-second, 1-minute, daily), rolling VWAP and VWAP-deviation (price location relative to the session's volume-weighted average), realized volatility estimators (Parkinson, Garman-Klass on OHLCV; bipower variation on tick data), and order book imbalance (bid depth minus ask depth as a fraction of total depth, a short-horizon execution signal). Each of these derived features must be recomputed on every new tick in real time and stored with the same point-in-time discipline as the underlying ticks.

Latency Tiers and Which Strategies Need Each

Latency requirements are strategy-specific, and over-engineering the data stack for a strategy that does not need microsecond feeds is a material waste of infrastructure budget and engineering time. The tiers:

• Sub-100μs — HFT, market making, stat arb on tightly correlated instruments (futures basis, ETF/underlying arbitrage). Requires FPGA-based feed handlers or kernel-bypass networking (DPDK, RDMA), co-location in the exchange's primary data center, and purpose-built hardware. The engineering investment is 3–5 engineer-years and $500K+/year in infrastructure costs. Justified only for strategies with execution frequency above 1,000 trades/day.
• 1–10ms — event-driven execution, fast mean-reversion, intraday momentum. Achievable with co-location and an optimized Linux network stack (kernel bypass is not required). Direct exchange feeds are necessary; SIP consolidated tape is insufficient.
• 10–100ms — intraday systematic strategies with holding periods above 5 minutes. Standard co-location with direct feeds is sufficient. SIP latency is borderline acceptable depending on the strategy's signal decay profile.
• >100ms — EOD systematic strategies, portfolio optimization and rebalancing, daily factor signals. Cloud-delivered vendor feeds are entirely adequate. The cost of direct feeds and co-location is not justified by any performance improvement at this holding period.

The common mistake is treating latency as a proxy for sophistication. A factor investing strategy that rebalances weekly has zero need for microsecond feeds. Every engineer-year spent on FPGA feed handlers for a daily rebalancing strategy is an engineer-year not spent on signal generation, risk management, or execution quality. Match the infrastructure to the strategy. The real cost of over-engineering is not the server bill — it is the opportunity cost of the engineering hours.

The Point-in-Time Correctness Problem

Point-in-time correctness is the single biggest source of backtest performance inflation at institutional desks, and it originates entirely in the data infrastructure layer. The problem: price data is not immutable. Splits retroactively halve historical prices. Dividend adjustments retroactively reduce historical close prices by the dividend amount. Exchange corrections restate intraday prints. Index composition changes restate which securities were in the index at any historical date. Every retroactive modification of historical data creates a lookahead bias opportunity: a backtest that uses today's adjusted price history was trained on information that was not available on the historical dates being simulated.

The canonical example: a stock that split 2-for-1 in March 2025 has its pre-split prices halved in every data vendor's adjusted history. A momentum strategy backtested on that data will see the split date as a 50% drawdown followed by a 100% recovery — or if the vendor adjusts correctly, will see artificially smooth price history that never existed. Either way, the backtest is not simulating what a live trader could have seen. For risk management software that computes drawdowns from adjusted price series, the corporate action handling discipline is equally critical — a missed adjustment inflates historical drawdowns; an overcorrected adjustment masks them.

The correct architecture: store raw unadjusted ticks with immutable append-only writes. Apply corporate action adjustments forward-only at query time, using a strict as-of-date parameter that limits which adjustments are visible to a given query. A research query with as-of-date 2024-01-01 should see only the adjustment factors that were publicly known on 2024-01-01 — not the 2025 split that had not yet been announced. Every query against the research database must specify a knowledge cutoff date. Systems that do not enforce this are not producing valid backtests; they are producing artifacts of retroactive data adjustment.

Vendor Evaluation Framework: Build vs. Buy

The build vs. buy decision for market data infrastructure is not primarily a cost question — it is a latency and coverage question with cost implications.

• Buy (Bloomberg B-PIPE, Refinitiv Elektron, Nasdaq Global Data Services) — the right choice when latency above 10ms is acceptable, compliance and legal teams require vendor-managed audit trails, or the quant team is fewer than 5 people and cannot absorb the infrastructure maintenance burden. Cost: $50K–$500K/year depending on asset class coverage and delivery mechanism. Bloomberg B-PIPE enterprise licenses for full equities, futures, and options coverage run $200K–$400K/year at typical hedge fund terms. Coverage is comprehensive, audit trails are vendor-managed, and the operational burden is low.
• Build (direct exchange feeds + custom normalization) — the right choice when latency below 1ms is required, asset class coverage is narrow and well-defined (e.g., CME equity futures only), and the team has dedicated infrastructure engineers who can own the stack. Cost: 2–3 engineer-years upfront for initial build, plus $20–100K/year in co-location fees (depending on exchange and proximity tier) and exchange connectivity fees ($5K–$50K/year per exchange depending on data depth). The ongoing operational burden is material: feed handler upgrades, corporate action pipeline maintenance, and exchange protocol changes require continuous engineering attention.
• Hybrid — most institutional desks land here. Vendor feeds for research and backtesting (where latency does not matter and coverage breadth does); direct exchange feeds for live execution (where latency and fill quality are primary). The hybrid architecture avoids paying for low-latency feeds for all research workflows while maintaining execution-quality data for live trading.

Questions to ask any vendor before signing: What is the feed latency SLA, and are there penalty clauses for SLA breaches? What is the historical data correction policy — when exchange corrections occur, how are they propagated and how far back? Is tick data, top-of-book, or full order book depth available for each asset class? What are the asset class gaps — specifically for crypto, OTC derivatives, and municipal bonds, which are frequently absent from standard equity data subscriptions? The answers define the ceiling on every strategy that uses that vendor's data.

Infrastructure Redundancy and Monitoring

Market data is business-critical infrastructure. A feed handler outage during an active trading session is not an operational inconvenience — it is a live risk management failure. Production data stacks require the same redundancy and monitoring discipline as any other critical system.

• Dual-feed primary/backup — primary and backup feed handlers running in parallel, with automatic failover triggered by gap detection or heartbeat failure. The backup feed can be a different delivery mechanism (e.g., TCP backup for a UDP primary) to ensure that a network-layer failure on the primary does not also affect the backup.
• Sequence number gap alerting — any gap greater than zero triggers an immediate investigation alert. Gaps that last more than 100ms trigger automatic failover to the backup feed. Tolerating gaps silently is not an option: a missed trade print or quote update propagates incorrect state through the entire downstream pipeline.
• Feed health dashboards — messages/second per feed and per symbol (to detect individual symbol staleness), end-to-end latency percentiles (p50/p95/p99), gap rate per session. Anomalies — a symbol going silent mid-session, latency spiking on a single feed — must surface immediately, not in end-of-day reconciliation.
• Daily OHLCV reconciliation — compare stored daily OHLCV against an independent reference (Bloomberg official close, exchange-published settlement prices). Discrepancies above a threshold (typically >0.05% on close price) trigger a data quality investigation. This catches normalization bugs, missed corporate actions, and feed handler defects before they contaminate backtests or live signals.
• Cold-start procedures — when a feed handler restarts mid-session, it must not replay the entire session history or start with a gap. The correct procedure depends on the exchange: some support snapshot requests that deliver current state; others require replaying from a specific sequence number. The cold-start procedure must be tested and documented for every exchange connection, because an untested cold-start procedure will fail at the worst possible moment.

The execution algorithms that route orders against live market data are entirely dependent on the integrity of the data they consume. A VWAP algorithm that ingests stale quotes will underperform its benchmark in a way that looks like poor signal quality but is actually a data infrastructure failure. Feed health monitoring is not separate from execution quality monitoring — it is part of the same operational stack.

How AlphaEdge AI Handles the Data Layer

The market data infrastructure stack described above — feed handlers, normalization pipelines, time-series storage, derived data pipelines, redundancy, and reconciliation — represents 2–4 engineer-years of build time and ongoing operational burden for every desk that owns it. For most institutional quant teams, that engineering time is better spent on signal generation, risk management, and strategy execution than on database partitioning schemes and feed handler gap recovery logic.

AlphaEdge AI abstracts the entire data infrastructure stack. Real-time ingestion across stocks, ETFs, forex, commodities, options, and crypto — all normalized, point-in-time correct, and directly accessible for signal generation, backtesting, and live execution. No feed handlers to manage. No normalization pipeline to maintain. No time-series database to tune. The data layer is handled; the signal layer is where your team spends its engineering resources.

For teams evaluating the build vs. buy decision, options volatility strategies — which require real-time option chain ingestion across thousands of strikes and expiries — illustrate the data infrastructure challenge most sharply: 500,000+ quote updates per second, SVI surface calibration on every tick, cross-venue Greeks aggregation in real time. AlphaEdge AI delivers this data layer without the desk building any of it. For cross-asset desks running both equity signals and vol overlays, having a unified data layer that covers both is a structural advantage that compound across every strategy the desk runs.

Market data infrastructure is invisible when it works and catastrophic when it fails. The desks that treat it as a strategic asset — owning the normalization logic, enforcing point-in-time correctness, monitoring feed health at tick resolution — are operating on a fundamentally different data foundation than those that treat it as a commodity subscription. Signal quality is capped by data quality. Backtest validity is determined by the data layer. Execution quality degrades with feed latency. The data layer is not peripheral to the alpha generation process. It is the alpha foundation.

AlphaEdge AI provides the production data infrastructure out of the box — so your team skips the 2-year build cycle and starts generating signals on day one. Real-time ingestion, full normalization, and point-in-time correctness across every asset class, delivered as a platform your quant team can query directly.

For the full data infrastructure architecture — point-in-time databases, corporate action handling, alternative data ingestion, and normalization pipelines — see our guide to quant fund data infrastructure and market data pipelines.

For the intraday signal layer built on top of real-time data infrastructure — how systematic funds exploit opening auction dynamics, gap strategies, and intraday momentum patterns — see our guide to quant fund intraday alpha.