Abstract
Supporting multi-gigabit/s of iSCSI over TCP can quickly saturate the processing abilities of a SMP server today. Legacy OS designs and APIs are not designed for the multi-gigabit IO speeds. Most of industry’s efforts had been focused on offloading the extra processing and memory load to the network adapter (NIC). As an alternative, this paper shows a software implementation of iSCSI on generic OSes and processors. We discuss an asymmetric multiprocessing (AMP) architecture, where one of the processors is dedicated to serve as a TCP engine. The original purpose of our prototype was to leverage the flexibility and tools available in generic systems for extensive analyses of iSCSI. As work proceeded, we quickly realized the viability of generic processors to meet iSCSI requirements. Looking ahead to chip-multiprocessing, where multiple cores reside on each processor, understanding partitioning of work and scaling to cores will be important in future server platforms.
Chapter PDF
Similar content being viewed by others
References
Aiken, S., Grunwald, D., Andrew, P.: A performance analysis of the iSCSI protocol. In: Proceedings of the 20th IEEE Conf. on Mass Storage Systems and Technologies (2003)
Anand, V., Hartner, B.: TCPIP Network Stack Performance in Linux Kernel 2.4 and 2.5. In: Proc. of the Ottawa Linux Symposium, Ottawa (2002)
Brosky, S., Rotolo, S.: Shielded Processors: Guaranteeing Sub-millisecond Response in Standard Linux. In: 4th Real-Time Linux Workshop, Boston (December 2002)
Cameron, D., Regnier, G.: The Virtual Interface Architecture. Intel Press, Hillsboro (2002)
Chase, J., Gallatin, G., Yocum, K.: End system optimizations J. for high-speed TCP. IEEE Comms. Special Issue on high-speed TCP 39(4) (2001)
Foong, J.F., Huff, T., Hum, H., Patwardhan, J., Regnier, G.: TCP performance re-visited. In: Proc. of the IEEE ISPASS, Austin (March 2003)
Foong, J.F., Newell, D., Irelan, P., Lopez-Estrada, A., Abraham, S.: Architectural characterization of the impact of Processor Affinity on Network Processing. IEEE ISPASS (March 2005) (to appear)
Hammond, L., Nayfeh, B., Olukotun, K.: A single-chip multiprocessor. IEEE Computer (September 1997)
Hsu, J., Banerjee, P.: A message passing coprocessor for distributed memory multicomputers. Supercomputing, 720–729 (1990)
Kay, J., Pasquale, J.: The Importance of Non-Data Touching Processing Overheads in TCP/IP. In: Proc. of ACM SIGCOMM (1993)
Leroux, P.: Building Scalable Networking Equipment Using SMP. Dedicated Systems Magazine (2001)
Mogul, J.: TCP offload is a dumb idea whose time has come. In: Proc. of HotOS IX, Lihue (May 2003)
Muir, S., Smith, J.: AsyMOS: An asymmetric multiprocessor OS. In: Proceedings of OPENARCH 1998 (April 1998)
Pierce, P., Regnier, G.: The Paragon implementation of the NX message passing interface. In: Proc. of SHPCC 1994 (1994)
Rangarajan, M., Bohra, A., Banerjee, K., Carrera, E., Bianchini, R.: TCP servers: Offloading TCP processing in internet servers. Technical report. Rutgers University (2002)
Regnier, G., Minturn, D., McAlpine, G., Saletore, V., Foong, A.: ETA: Experience with an Intel Xeon Processor as a packet processing engine. IEEE Micro. (January 2004)
Sarkar, P., Uttamchandani, S., Voruganti, K.: Storage over IP: When does hardware support help? In: Proc. of the 2nd USENIX Conf. on File and Storage Technologies (2002)
Meredith, M., Vianney, D.: Linux 2.6 Performance in the Data Center. Linux World Expo (January 2004)
Iometer performance Analysis Tool, http://www.iometer.org .
Oprofile: A system-wide profiling tool for Linux, http://oprofile.sourceforge.net
UNH-iSCSI, Intel iSCSI reference stacks, http://sourceforge.net/
Scalable Networking: Eliminating the Receive Processing Bottleneck—Introducing RSS, http://www.microsoft.com/whdc/
Remote direct data placement protocol, http://www.ietf.org/html.charters/rddpcharter.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Foong, A., McAlpine, G., Minturn, D., Regnier, G., Saletore, V. (2005). An Architecture for Software-Based iSCSI: Experiences and Analyses. In: Boutaba, R., Almeroth, K., Puigjaner, R., Shen, S., Black, J.P. (eds) NETWORKING 2005. Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications Systems. NETWORKING 2005. Lecture Notes in Computer Science, vol 3462. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11422778_6
Download citation
DOI: https://doi.org/10.1007/11422778_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25809-4
Online ISBN: 978-3-540-32017-3
eBook Packages: Computer ScienceComputer Science (R0)