Measuring Burstiness in Data Center Applications

Buffer sizing is a tricky task --- it depends on a large number of variables, ranging from congestion control to traffic engineering. Still, the most unpredictable contributors are the workloads running in the network. The link utilization and burstiness of these workloads dictate the buffer depth needed by a switch. But what is a burst? Do traditional definitions still apply in the age in which switches transfer terabits of data and billions of packets every second? Unless we assess bursts correctly, we are unlikely to size buffers appropriately. In this work, we present a measurement-led evaluation of the burstiness of different data center applications. We address the question of "what is a burst?" and assert that common techniques cannot answer this question in modern data centers. We quantify the change in burstiness of the studied applications across multiple vectors, including latency and network perspective, and generalize our results to the common case. Our observations can inform future buffer sizing efforts and guide switch configurations. Our dataset is openly available for the benefit of the community.