research-article

High-radix crossbar switches enabled by proximity communication

Authors:

Pedro J. Garcia,

Wladek OlesinskiAuthors Info & Claims

SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing

Article No.: 32, Pages 1 - 12

Published: 15 November 2008 Publication History

Abstract

We describe a novel way to implement high-radix crossbar switches. Our work is enabled by a new chip interconnect technology called Proximity Communication (PxC) that offers unparalleled chip IO density. First, we show how a crossbar architecture is topologically mapped onto a PxC-enabled multi-chip module (MCM). Then, we describe a first prototype implementation of a small-scale switch based on a PxC MCM. Finally, we present a performance analysis of two large-scale switch configurations with 288 ports and 1,728 ports, respectively, contrasting a 1-stage PxC-enabled switch and a multi-stage switch using conventional technology. Our simulation results show that (a) arbitration delays in a large 1-stage switch can be considerable, (b) multi-stage switches are extremely susceptible to saturation under non-uniform traffic, a problem that becomes worse for higher radices (1-stage switches, in contrast, are not affected by this problem).

References

[1]

F. Abel, C. Minkenberg, R. Luijten, M. Gusat, I. Iliadis, "A four-terabit packet switch supporting long round-trip times," IEEE Micro, vol. 23, no. 1, pp. 10--24, Jan./Feb. 2003.

Digital Library

[2]

ATCA Web Site, www.advancedtca.org.

[3]

V. Benes, Mathematical Theory of Connecting Networks and Telephone Traffic, New York, Academic Press, 1965.

[4]

C. Clos, "A Study of Non-blocking Switching Networks," Bell System Technical Journal, vol. 32, no. 5, pp. 406--424, March 1953.

[5]

R. Drost, R. Hopkins, I. Sutherland, "Proximity Communication," IEEE Custom Integrated Circuits Conference, pp. 469--472, Sep. 2003.

[6]

R. Drost, R. Hopkins, I. Sutherland, "Electronic Alignment for Proximity Communication," IEEE Solid-State Circuits Conference, pp. 144--518, Feb. 2004.

[7]

J. Duato, J. Flich, T. Nachiondo, "Cost-Effective Technique to Reduce HOL-blocking in Single-Stage and Multistage Switch Fabrics," Euromicro Conference on Parallel, Distributed and Network-based Processing, pp. 48--53, Feb. 2004.

[8]

Fulcrum Web Site, http://www.fulcrummicro.com/products/focalpoint/fm2224.htm.

[9]

C. Gomez, F. Gilabert, M. E. Gomez, P. Lopez and J. Duato, "Deterministic versus Adaptive Routing in Fat-Trees," Workshop on Communication Architecture for Clusters, pp. 1--8, March 2007.

[10]

D. Hopkins, A. Chow, R. Bosnyak, B. Coates, J. Ebergen, S. Fairbanks, J. Gainsley, R. Ho, J. Lexau, F. Liu, T. Ono, J. Schauer, I. Sutherland, R. Drost, "Circuit Techniques to Enable 430Gb/s/mm² Proximity Communication," IEEE International Solid-State Circuits Conference, pp. 368--369, Feb. 2007.

[11]

J. Kim, W. Dally, B. Towles, A. Gupta, "Microarchitecture of a High-Radix Router," IEEE/ACM 32nd International Symposium on Computer Architecture, pp. 420--43, June 2005.

Digital Library

[12]

N. McKeown, M. Izzard, A. Mekkittikul, W. Ellersick, M. Horowitz, "Tiny Tera: a Packet Switch Core," IEEE Micro, vol. 17, no. 1, pp. 26--33, Jan./Feb. 1997.

Digital Library

[13]

Mellanox Web Site, http://www.mellanox.com/products/switch\_silicon.php.

[14]

G. Mora, J. Flich, J. Duato, E. Baydal, P. Lpez, O. Lysne, "Towards an Efficient Switch Architecture for High-Radix Switches," ACM/IEEE Symposium on Architectures for Networking and Communications Systems, Dec. 2006.

Digital Library

[15]

W. Olesinski, H. Eberle, N. Gura, "OBIG: the Architecture of an Output Buffered Switch with Input Groups for Large Switches," IEEE GLOBECOM 2007, Nov. 2007.

[16]

W. Olesinski, H. Eberle, N. Gura: "PWWFA: The Parallel Wrapped Wave Front Arbiter for Large Switches," IEEE Workshop on High Performance Switching and Routing, May/June 2007.

[17]

W. Olesinski, N. Gura, H. Eberle, A. Mejia, "Low-Latency Scheduling in Large Switches," ACM/IEEE Symposium on Architectures for Networking and Communications Systems, Dec. 2007.

Digital Library

[18]

S. Scott, D. Abts, J. Kim, W. Dally, "The BlackWidow High-Radix Clos Network," IEEE/ACM 33rd International Symposium on Computer Architecture, pp. 16--28, June 2006.

Digital Library

[19]

Y. Tamir, H. C. Chi, "Symmetric Crossbar Arbiters for VLSI Communication Switches," IEEE Transactions on Parallel and Distributed Systems, vol. 4, issue 1, pp. 13--27, Jan. 1993.

Digital Library

[20]

Sun Microsystems, Inc., "The Path to Massive Scale High Performance Computing," www.sun.com/products/networking/datacenter/ds3456.

Cited By

Rezazadeh MSafaei FMoazez M(2015)FlatteningMicroelectronics Journal10.1016/j.mejo.2015.07.00346:10(875-892)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.mejo.2015.07.003
Villar JAndújar FAlfaro FSánchez JDuato J(2014)Formalization and configuration methodology for high-radix combined switchesThe Journal of Supercomputing10.1007/s11227-014-1223-969:3(1410-1444)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1007/s11227-014-1223-9
Ahn JSon YKim J(2013)Scalable high-radix router microarchitecture using a network switch organizationACM Transactions on Architecture and Code Optimization10.1145/251243310:3(1-25)Online publication date: 16-Sep-2013
https://dl.acm.org/doi/10.1145/2512433
Show More Cited By

Index Terms

High-radix crossbar switches enabled by proximity communication

Recommendations

Symmetric Crossbar Arbiters for VLSI Communication Switches

The design and implementation of symmetric crossbar arbiters are addressed. Severalarbiter designs are compared based on simulations of a multistage interconnectionnetwork. These simulations demonstrate the influence of the switch arbitration policy ...
Modeling and Design of High-Radix On-Chip Crossbar Switches
NOCS '15: Proceedings of the 9th International Symposium on Networks-on-Chip

The crossbar is a popular topology for on-chip networks that offers non-blocking connectivity and uniform latency. However, as the number of nodes increases, crossbars typically scale poorly in area, power, and latency/throughput. To better understand ...
The Combined Input-Output Queued Crossbar Architecture for High-Radix On-Chip Switches
High-radix, single-chip routers have emerged as efficient building blocks for interconnection networks. Researchers believe that hierarchical switch architectures are needed at high radices as crossbars scale with the square of the router radix. This ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing

November 2008

739 pages

ISBN:9781424428359

Conference Chair:
Patricia Teller

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Press

Publication History

Published: 15 November 2008

Check for updates

Qualifiers

Research-article

Funding Sources

CONSOLIDER-INGENIO 2010
Junta de Comunidades de Castilla-La Mancha
Ministerio de Educación, Cultura y Deporte

Conference

SC '08

Sponsor:

SIGARCH
IEEE-CS

SC '08: International Conference for High Performance Computing, Networking, Storage and Analysis

November 15 - 21, 2008

Texas, Austin

Acceptance Rates

SC '08 Paper Acceptance Rate 59 of 277 submissions, 21%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
293
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rezazadeh MSafaei FMoazez M(2015)FlatteningMicroelectronics Journal10.1016/j.mejo.2015.07.00346:10(875-892)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.mejo.2015.07.003
Villar JAndújar FAlfaro FSánchez JDuato J(2014)Formalization and configuration methodology for high-radix combined switchesThe Journal of Supercomputing10.1007/s11227-014-1223-969:3(1410-1444)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1007/s11227-014-1223-9
Ahn JSon YKim J(2013)Scalable high-radix router microarchitecture using a network switch organizationACM Transactions on Architecture and Code Optimization10.1145/251243310:3(1-25)Online publication date: 16-Sep-2013
https://dl.acm.org/doi/10.1145/2512433
Olesinski WEberle HGura N(2009)Scalable alternatives to virtual output queuingProceedings of the 2009 IEEE international conference on Communications10.5555/1817271.1817668(2129-2134)Online publication date: 14-Jun-2009
https://dl.acm.org/doi/10.5555/1817271.1817668

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents