Web100: extended TCP instrumentation for research, education and diagnosis

TCP has become the dominant protocol for all network data transport because it presents a simple uniform data delivery service which is sufficient for most applications over all types of lower network layers. By its very nature, TCP's adaption and retransmission strategies hide all of the details of the lower layers from the application. For example the only symptom of spurious packet loss (or nearly any other network problem) is longer elapsed time and lower performance.This information hiding is fundamentally important to the growth of the Internet because it decouples the evolution of applications from the evolution of link layers. However it also hides valuable information from researchers, educators, network administrators, and other people who would benefit from insight into the inner workings of TCP and the lower layers.In this paper, we present an architecture and infrastructure that provides for per-connection TCP instrumentation to expose otherwise hidden protocol events. We show examples how the infrastructure can be used in support of research, education and advanced network diagnostic tools.Our work was motivated by the observation that since about 1985 network data rates for typical novice network users have fallen by about three orders of magnitude behind expert users (who have kept up with Moore's Law). We use the term "Wizard Gap" to describe this phenomenon. The Web100 and Net100 projects were formed as one step in closing the Wizard Gap.