Low latency tricks for HFT and quantpylib
Quantpylib’s documentation has been moved to:
https://quantpylib.hangukquant.com/ and you can obtain the repo pass here.
Alternatively, obtain the annual research access for quantpylib.
In this post, we discuss
Trade imputation for implied orderbooks.
Canary orders for even latency edge.
Data redundancy and Feed arbitration.
LOB Data structure (B-tree/Sorted Array) and complexity.
Quantpylib choices (new features) and implementations.
For HFT firms, maintaining an accurate and responsive internal order book is critical; it serves many purposes including but not limited to
impact-modelling: simulate how agent’s orders affect market prices (for instance, in the funding arb lectures, the maker order size is a function of the taker exchange’s order book size and aggressiveness).
fill-probabilities: likelihood that orders are filled based on queue priority in the ob.
feature-generation: used for generating micro-structural features/’indicators’ such as order-book pressure/imbalance.
For most exchanges, data feeds come in variable types; order-book snapshots, delta updates, public trade prints, private fills etc.
For example, the Binance orderbook snapshots are available to limited depth at 100~500ms. If we want deeper levels, we would obtain an orderbook snapshot from the REST endpoint and stream delta updates to keep an internal orderbook state.
Trade Imputation
Here is a BBA quote from BTCUSDT/binance from the delta updates.
In green, we have trade arrival. In the image, the BBA lines are a right discontinuous jumps. Notice how some the ticks move before the delta updates; in the interval between the blue ticks and the next delta sequence, there is a latency in which our order state is calibrated on old data.
To prevent adverse fills, we can use public trade prints to impute the orderbook state between delta updates. For instance, if a trade print shows a sale at a price not previously visible in the ask side, the firm can deduce that an ask order was placed and matched.
This is now enabled in quantpylib. It is a matter-of setting a boolean flag (implied_orderbook=True) in the quantpylib data feed module
Canary Orders
A more, aggressive optimisation would be to use fill information from small, experimental orders designed to probe the market rather than to be profitably filled. Typically, the private fill information is sent out faster than public trade fills, so canary orders give information about drifts in the orderbook state even faster.
Imagine a firm placing a tiny buy order just below the best bid. If this order unexpectedly fills, it indicates that the market has shifted faster than the internal order book reflects. If this is on a leader, reference exchange, this information can be propagated to actions such as cancelling quotes on the laggard venues.
This feature is not available in quantpylib but we aim this to be a feature of the quantpylib OMS in the future.