People from the high-performance computing field have a clear understanding that performance of technical systems (of various nature) is characterized by two metrics: throughput and latency. People in other fields sometimes focus on either throughput or latency alone.
MV Colombo Express, one of the largest container ships in the world. Image source: Wikipedia
For many years it has been a well-accepted truth in the HPC field that latency in the computer network does matter. But with the proliferation of Gigabit Ethernet on desktops, people started to expect their network connection to function at a high bandwidth. It quickly turned out that big network latencies can easily limit available bandwidth. That’s how ordinary people learned about bandwidth, latency, and their interdependence.
Someone (who?) once jokingly defined supercomputer as “a computer whose performance is limited by input/output (I/O) rather than by the CPU” (link). Indeed, now that HPC is becoming increasingly dominated by data-crunching, not number-crunching, input/output starts to play a very big role. It is important to fetch vast amounts of data at high throughput, but it is equally important to perform a quick read of a small portion of data — and that is latency, not throughput.
Here is how Dimitris Krekoukias of RecoveryMonkey.org explains latency in storage systems:
Simply speaking, latency is a measure of how long it takes for a single I/O request to happen from the application’s viewpoint.
Therefore, if the latency of the storage system is big, then the computation process would block, waiting for the data to arrive (or be written back to storage), thus increasing the overall time to solution.
And here is another comparison of throughput versus latency. Take a look at the picture above: the container ship. The throughput (the load of cargo) is enormous, but the latency is big: the round-trip time from Europe to South-East Asia and back is 56 days. And you can’t trade throughput for latency, unless, of course, you shift to another means of transport, such as aviation.
Merry Christmas, and if your holiday presents are travelling via cargo ships, I hope they arrive on time!
