The days of the proprietary ticker plant are numbered. That’s the direction that Greg Maynard and the rest of the ISE believe the industry is moving. Speaking on Monday at the FIX Protocol Exchanges/ECN Working Group meeting, he was one of 6 exchange representatives speaking in glowing terms of FAST (FIX Adapted for STreaming). FAST is a high-performance wire protocol for encoding market data and other FIX traffic, which will replace proprietary wire protocols for market data entirely.
Historically, one of the primary complaints about the FIX protocol was the verbosity of the wire protocol, to which all manner of throughput and latency problems were ascribed. The FAST protocol addresses these complaints by achieving up to 90% compression ratios for typical FIX message traffic, while imposing almost zero CPU overhead. Rolf Andersson and the Market Data Optimization Working Group have designed a templating system that allows the omission of data that is redundant or easily calculated by the other side. In this way FAST is able to do much better than standard compression mechanisms like Lempel-Ziv or DEFLATE, which obviously know nothing of the high-level structure of the data. Using these techniques, Rolf asserts that the compression ratios and CPU usage of a good FAST implementation are as good as, if not better than, the proprietary algorithms used by the big consolidated feed vendors.
What constitutes a good FAST implementation is actually an interesting question. First, the particular implementation of FAST (specifically the codec) must be carefully designed, and implemented (and probably needs a few iterations with a good profiling tool). The second optimization step, proper template design, is the one that may determine the fate of FAST. FAST templates are the rules that guide the compression process. Specifically they describe what data is redundant or can be easily calculated by the receiver. These rules need to be created and tuned for each data feed, as the correct compression strategies depend highly on what type of data is expected on the wire. Comprehensive documentation from the FAST working group, and good tool support from the industry will likely help facilitate good FAST implementation, but those may take time.
But a good implementation, like the one built out by the Chicago Mercantile Exchange can produce impressive gains. Previously they had been using a proprietary protocol based on zlib (DEFLATE), and after moving to a new FAST/FIX-based implementation they saw a 70% improvement in bandwidth utilization (he hinted at significant CPU utilization gains as well, but did not give any hard numbers).
It’s no secret that we here at Marketcetera are FIX enthusiasts. We are excited about the possibilities that FAST presents. In the near future, we expect to see qualitative changes to the market data delivery landscape that Greg has predicted, as the quantitative advantages of FAST become apparent.