Crypto HFT - In Depth Guide to Optimization (I)
Continuing the last post on the market maker series -
we are going to do an exploration of the state and techniques for crypto specific hft optimisation, which differ from traditional hft outfits. Traditionally, tradfi hft involves buying access to colocation services which places their machine adjacent to the exchange servers. Exchanges typically spend a lot of effort ensuring fair access network connectivity and deterministic latency for these participants.
Therefore the role of the hft quant was primarily about server tuning, configuring kernel interactions such as buffers, IRQ coalescing and scheduler tasks, as well as app-level software t2t. To the extent of ultra low-latency settings - quant teams in-housed FPGA/DPDK solutions.
However, the crypto industry, and to an increasing extent, tradfi is turning to low latency services hosted by cloud infrastructure - for reasons including costs, scalability/elasticity and operational velocity.
Most crypto CEXes and DEXes matching engines/sequencers, market data gateways are hosted on cloud services such as AWS. Therefore, the role of colocation as purchasing exclusive proximity rights is secondary (although still present in a different form, as we will discuss) and the job of engineers here is now primarily the task of network optimisation, and the traditional skillset of tuning on-prem servers takes a backseat in terms of the leverage in latency gains to be had.
Given this backdrop, we explore a series of optimisation techniques that quant teams use in the crypto hft space. Of course, these techniques were not discovered by me, rather - they are based on a culmination of experience trading on these cloud infrastructures, conferences and consultations with cloud solutions architects and network engineers.