Scaling European Equity and ETF Data Processing To Deliver the Consolidated Tape - big xyt

Laying the technological and operational foundation is essential for the European Consolidated Tape (CT). Our infrastructure is already operating at the scale and precision that the CT demands, demonstrating our readiness to serve as the independent provider of consolidated market data for the entire European capital markets ecosystem.

Fragmentation is still the biggest barrier to the hyper-competitive European financial markets. Delivering reliable, high-quality and real-time analytics at scale is no longer a nice-to-have – it’s a necessity. We process billions of records daily, ensuring nanosecond precision, delivering the most comprehensive analytics across the European equities and ETFs landscape.

Cloud-Native Architecture: The Foundation for the CT

By leveraging modern orchestration and storage technologies, our platform ensures elasticity, resilience and scalability, enabling us to:

• Orchestrate thousands of microservices across a distributed compute environment, with over 5,500 pods and 3,800 containers running concurrently.
• Design and manage failover and recovery processes, maintaining system stability even under extreme volume surges – essential for delivering the CT’s expected high-availability requirements.
• Handle over 4 petabytes of data, with infrastructure designed to scale seamlessly by 200–500 GB each day, designed to match the complexity of pan-European market data.

As with any firm wishing to deliver a CT, our architecture delivers geographic reach, low latency and redundancy needed to meet the CT’s cross-border data delivery goals. Multi-datacenters are deployed across Germany, Finland and the UK. 

CT-Grade Data

By ingesting and processing real-time data streams from over 120 global trading venues, we normalise diverse trading conditions into a harmonised, high-quality data set – a core requirement for the CT.

• Nanosecond and microsecond precision timestamping ensures trade and quote data is accurately synchronised across venues, essential for reliable consolidated reporting.
• Processing rates reach 200,000 messages per second, with up to 1.5 billion daily updates during peak volatility, demonstrating readiness to handle the scale of European market data for the CT.
• Support for 500+ status codes and 1,000+ condition codes monthly enables granular analytics of market depth and participant behaviour – capabilities that will power the CT’s comprehensive view of trading activity.

Powering the CT Vision

The European CT isn’t just about aggregating data; it’s about delivering it accurately, in real time, and with full market depth visibility. Key innovations include:

• Dynamic resource scaling, with Kubernetes-based orchestration, ensures seamless performance during surges and market events.
• Efficient aggregation of 2.5-3TB and approx. 70 billion rows of data monthly enables deep, Level 3 analytics – essential for the CT’s transparency and benchmarking goals.
• Harmonisation of off-book data, such as SI prints, with primary market quotes ensures fair, transparent reporting across all trade types.

A CT-Ready Solution

With real-time analytics already delivering pre-trade decision support and dynamic post-trade performance metrics, the solution integrates seamlessly into client workflows via APIs and web-based interfaces, mirroring the delivery model envisioned for the CT.

The capabilities are already in place to transition smoothly from internal data processing excellence to an industry-wide, independent CT that delivers reliable, transparent and regulatory-compliant data to all market participants.

The Backbone of the European CT

To date, our approach to data processing – built on a scalable, resilient and precision-driven technology stack – is transforming how market data is processed and establishing the blueprint for the European Consolidated Tape.

ESMA’s selection process continues, however, the support from industry participants underscores the requirement of an independent, technology-first solutions provider to meet the needs of Europe’s financial markets.