The HFT pursuit has triggered a technology arms race across our industry with tremendous capital (human and financial) expended chasing elusive microseconds.
In the midst of this full-throttle pursuit, it is worth considering some of the fundamental assumptions of great software design. One of the most basic (and often over-looked) questions is whether to pursue a proprietary system or opt for an open source approach.
History provides some useful context. In October 1991, a group of volunteers released Linux, a free operating system. The most recent release of Ubuntu Linux, one of the most popular distributions, is comprised of 100 million lines of code. As a point of comparison, Microsoft’s much postponed Windows Vista operating system was approximately 50 million lines of code. The software industry is strongly influenced by an engineering culture that values tight centralized planning, control and monitoring of the development process. How could a volunteer effort deliver a system twice the size of Microsoft’s offering?
The answer, I would argue, results from the “sunlight” benefits of the open source model. Development teams are organized not by management teams, but rather by the developers themselves, with individuals free to gravitate to tasks of personal interest and passion. This is coupled with the tremendously valuable process of peer review. Unlike proprietary systems, the open source projects freely publish their source code for other developers (and customers) to review. As Lerner and Tirole have documented, the open source model works especially well in communities—for example, software developers—where there are strong “reputation effects.” Instead of hiding the software’s source code as a company’s proprietary secret, open source programmers publish their code on the Internet for others to read, learn from, experiment with, modify, improve, and reuse.
The effect of this “openness” cannot be overstated. Today’s financial services industry is awash in a tangled alphabet soup of proprietary technologies, few of which are designed to integrate smoothly. In fact, many of these systems come from competing software companies with little or no interest in ensuring that their products work with those from their competitors (and often quite the opposite).
The complexity of today’s systems, the number of products we integrate, the speed at which we trade, and the secrecy that cloaks many of our operations threaten to create inherently unstable systems. Understanding the infamous “Flash Crash,” for example, consumed months of work by both regulators and industry participants and it has still become a touch point for people worried about the stability of today’s trading infrastructure. Outside our industry, both Airbus and Boeing provide object lessons on the limitations of the central planning model. Open source lacks the marketing muscle of richer proprietary cousins, but our model presents a genuine alternative to current industry practice, one where the balance of power resides with trading firms that gain the leverage to interrogate the source code of their most mission critical applications before they are deployed.