In the previous post -
we looked at how to do kernel tuning - where most of the time is spent on host t2t (typically in crypto infrastructures). Quoting from that article
Other than the network topology…high value work comes at the level of the kernel.
This qualification is important - because network latency is order magnitudes more important to get right than software t2t - here is father El Jefe on that note:
That is, optimising software t2t before working on network latencies is putting the cart before the horse. Missing the forest for the trees. Reading “how to pleasure your woman” without a first date.
In a previous post -
we talked about box placement strategies and how different azs, placement groups, instance families can play a part in network optimisation. We also talked about endpoint selection and ALBs.
Here we will talk about single box network edge selection. It’s essentially the opposite side of the same coin in proximity search. Code examples are in the quantcplib repo. Link at the bottom.
What Happens
Let’s consider what happens, when we try to connect to say,
wss://fstream.binance.com/ws/btcusdt@bookTicker
The client presumably extracts the hostname fstream.binance.com, on port 443. This hostname must be translated into an IP address before we can open the TCP connection. The translation occurs via the libc function
::getaddrinfo(”fstream.binance.com”, “443”, &hints, &result)
This invokes the system’s configured name-resolution machinery. Depending on the host lookup configuration, the answer may come from a local hosts file, a resolver service or DNS. getaddrinfo gives us a linked list of socket addresses. The userspace network client then selects one of the addresses to connect to. Presumably, it walks the linked list and the first successful connection attempt is retained. The endpoint we use therefore depends on the addresses returned, their ordering and other less deliberate factors.
We can be more intentional about this edge resolution. For one, a wider set of candidates can be found via manual DNS querying than the heuristic set by the system resolver. Additionally, we are really interested in production-time market data latency, not arbitrary network latencies. We may question whether TCP handshake ranks translate to lower application layer ranks. We may also question whether such latency optimisations are persistent across time. And the answer will perhaps surprise.
Experiment
