research-article

Public Access

RotorNet: A Scalable, Low-complexity, Optical Datacenter Network

Authors:

William M. Mellette,

Rob McGuinness,

Alex Forencich,

Alex C. Snoeren,

George PorterAuthors Info & Claims

SIGCOMM '17: Proceedings of the Conference of the ACM Special Interest Group on Data Communication

Pages 267 - 280

https://doi.org/10.1145/3098822.3098838

Published: 07 August 2017 Publication History

Abstract

The ever-increasing bandwidth requirements of modern datacenters have led researchers to propose networks based upon optical circuit switches, but these proposals face significant deployment challenges. In particular, previous proposals dynamically configure circuit switches in response to changes in workload, requiring network-wide demand estimation, centralized circuit assignment, and tight time synchronization between various network elements--- resulting in a complex and unwieldy control plane. Moreover, limitations in the technologies underlying the individual circuit switches restrict both the rate at which they can be reconfigured and the scale of the network that can be constructed.

We propose RotorNet, a circuit-based network design that addresses these two challenges. While RotorNet dynamically reconfigures its constituent circuit switches, it decouples switch configuration from traffic patterns, obviating the need for demand collection and admitting a fully decentralized control plane. At the physical layer, RotorNet relaxes the requirements on the underlying circuit switches---in particular by not requiring individual switches to implement a full crossbar---enabling them to scale to 1000s of ports. We show that RotorNet outperforms comparably priced Fat Tree topologies under a variety of workload conditions, including traces taken from two commercial datacenters. We also demonstrate a small-scale RotorNet operating in practice on an eight-node testbed.

Supplementary Material

WEBM File (rotornetascalablelowcomplexityopticaldatacenternetwork.webm)

Download
127.00 MB

References

[1]

Dan Alistarh, Hitesh Ballani, Paolo Costa, Adam Funnell, Joshua Benjamin, Philip Watts, and Benn Thomsen. A High-Radix, Low-Latency Optical Switch for Data Centers (SIGCOMM '15).

[2]

Thomas Beth and Volker Hatz. 1991. A Restricted Crossbar Implementation and Its Applications. SIGARCH Comput. Archit. News 19, 6 (Dec. 1991).

Digital Library

[3]

Garrett Birkhoff. 1946. Tres Observaciones Sobre el Algebra Lineal. Univ. Nac. Tucumán Rev. Ser. A 5 (1946).

[4]

Shaileshh Bojja Venkatakrishnan, Mohammad Alizadeh, and Pramod Viswanath. Costly Circuits, Submodular Schedules and Approximate Carathéodory Theorems (SIGMETRICS '16).

[5]

Cheng-Shang Chang, Duan-Shin Lee, and Yi-Shean Jou. 2002. Load Balanced Birkhoff--von Neumann Switches, Part I: One-stage Buffering. Computer Communications 25, 6 (2002), 611--622.

Digital Library

[6]

Kai Chen, Ankit Singla, Atul Singh, Kishore Ramachandran, Lei Xu, Yueping Zhang, and Xitao Wen. OSA: An Optical Switching Architecture for Data Center Networks and Unprecedented Flexibility (NSDI '12).

[7]

Kai Chen, Xitao Wen, Xingyu Ma, Yan Chen, Yong Xia, Chengchen Hu, and Qunfeng Dong. WaveCube: A Scalable, Fault-tolerant, High-performance Optical Data Center Architecture (IEEE INFOCOM '15).

[8]

Li Chen, Kai Chen, Joshua Zhu, Minlan Yu, George Porter, Chunming Qiao, and Shan Zhong. Enabling Wide-Spread Communications on Optical Fabric with MegaSwitch (NSDI '17).

[9]

Jack Edmonds. 1965. Paths, trees, and flowers. Canad. J. Math. 17 (1965).

[10]

Facebook. 2014. Introducing data center fabric, the next-generation Facebook data center network. https://goo.gl/mvder2. (2014).

[11]

Facebook. 2015. FBFlow Dataset. https://www.facebook.com/network-analytics. (2015).

[12]

Nathan Farrington, Alex Forencich, George Porter, P-C Sun, Joseph E Ford, Yeshaiahu Fainman, George C Papen, and Amin Vahdat. 2013. A Multiport Microsecond Optical Circuit Switch for Data Center Networking. IEEE Photonics Technology Letters 25, 16 (2013).

[13]

Nathan Farrington, George Porter, Sivasankar Radhakrishnan, Hamid Bazzaz, Vikram Subramanya, Yeshaiahu Fainman, George Papen, and Amin Vahdat. Helios: A Hybrid Electrical/Optical Switch Architecture for Modular Data Centers (SIGCOMM '10).

[14]

David Gale and Lloyd S Shapley. 1962. College admissions and the stability of marriage. The American Mathematical Monthly 69, 1 (1962).

[15]

Monia Ghobadi, Ratul Mahajan, Amar Phanishayee, Nikhil Devanur, Janardhan Kulkarni, Gireeja Ranade, Pierre-Alexandre Blanche, Houman Rastegarfar, Madeleine Glick, and Daniel Kilper. ProjecToR: Agile Reconfigurable Data Center Interconnect (SIGCOMM '16).

[16]

Albert Greenberg, James R. Hamilton, Navendu Jain, Srikanth Kandula, Changhoon Kim, Parantap Lahiri, David A. Maltz, Parveen Patel, and Sudipta Sengupta. VL2: A Scalable and Flexible Data Center Network (SIGCOMM '09).

[17]

Navid Hamedazimi, Zafar Qazi, Himanshu Gupta, Vyas Sekar, Samir R. Das, Jon P. Longtin, Himanshu Shah, and Ashish Tanwer. FireFly: A Reconfigurable Wireless Data Center Fabric Using Free-space Optics (SIGCOMM '14).

[18]

Nikhil Handigol, Brandon Heller, Vimalkumar Jeyakumar, Bob Lantz, and Nick McKeown. Reproducible Network Experiments Using Container-based Emulation (CoNEXT '12).

[19]

Raj Jain. 1991. The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling.

[20]

He Liu, Feng Lu, Alex Forencich, Rishi Kapoor, Malveeka Tewari, Geoffrey M. Voelker, George Papen, Alex C. Snoeren, and George Porter. Circuit Switching Under the Radar with REACToR (NSDI '14).

[21]

He Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman, George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren. Scheduling Techniques for Hybrid Circuit/Packet Networks (CoNEXT '15).

[22]

He Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman, George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren. Scheduling Techniques for Hybrid Circuit/Packet Networks (CoNEXT '15).

[23]

Nick McKeown. 1999. The iSLIP Scheduling Algorithm for Input-Queued Switches. IEEE/ACM Transactions on Networking 7, 2 (1999).

Digital Library

[24]

William M. Mellette and Joseph Ford. 2015. Scaling Limits of MEMS Beam-Steering Switches for Data Center Networks. Journal of Lightwave Technology 33, 15 (Aug. 2015).

[25]

William M. Mellette, Glenn M. Schuster, George Porter, George Papen, and Joseph Ford. 2017. A scalable, partially configurable optical switch for data center networks. Journal of Lightwave Technology 35, 2 (Jan. 2017).

[26]

George Porter, Richard Strong, Nathan Farrington, Alex Forencich, Pang-Chen Sun, Tajana Rosing, Yeshaiahu Fainman, George Papen, and Amin Vahdat. Integrating Microsecond Circuit Switching into the Data Center (SIGCOMM '13).

[27]

Arjun Roy, Hongyi Zeng, Jasmeet Bagga, George Porter, and Alex C. Snoeren. Inside the Social Network's (Datacenter) Network (SIGCOMM '15).

[28]

Vishal Shrivastav, Asaf Valadarsky, Hitesh Ballani, Paolo Costa, Ki Suh Lee, Han Wang, Rachit Agarwal, and Hakim Weatherspoon. 2017. Shoal: a lossless newtork for high-density and disaggregated racks. Cornell Technical Report (2017).

[29]

Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network (SIGCOMM '15).

[30]

Leslie G. Valiant. 1982. A Scheme for Fast Parallel Communication. SIAM J. Comput. 11, 2 (1982).

[31]

John Von Neumann. 1953. A certain zero-sum two-person game equivalent to the optimal assignment problem. Contributions to the Theory of Games 2 (1953).

[32]

Guohui Wang, David G. Andersen, Michael Kaminsky, Konstantina Papagiannaki, T. S. Eugene Ng, Michael Kozuch, and Michael Ryan. c-Through: Part-time Optics in Data Centers (SIGCOMM '10).

[33]

David Zats, Tathagata Das, Prashanth Mohan, Dhruba Borthakur, and Randy Katz. DeTail: Reducing the Flow Completion Time Tail in Datacenter Networks (SIGCOMM '12).

Cited By

Zhu JSun WYe THu W(2024)Dynamic capacity sharing with multi-wavelength integrated transmitters in hybrid datacenter networksJournal of Optical Communications and Networking10.1364/JOCN.52844316:10(990)Online publication date: 19-Sep-2024
https://doi.org/10.1364/JOCN.528443
Qin LGu HYu XCai ZLiu J(2024)Orchid: enhancing HPC interconnection networks through infrequent topology reconfigurationJournal of Optical Communications and Networking10.1364/JOCN.51603116:6(644)Online publication date: 21-May-2024
https://doi.org/10.1364/JOCN.516031
Taka HInoue TOki E(2024)Design model of a twisted and folded Clos network with multi-step grouped intermediate switches guaranteeing admissible blocking probabilityJournal of Optical Communications and Networking10.1364/JOCN.51389816:3(328)Online publication date: 21-Feb-2024
https://doi.org/10.1364/JOCN.513898
Show More Cited By

Index Terms

RotorNet: A Scalable, Low-complexity, Optical Datacenter Network
1. Networks
  1. Network architectures

Recommendations

Enabling Instantaneous Relocation of Virtual Machines with a Lightweight VMM Extension
CCGRID '10: Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing

We are developing an efficient resource management system with aggressive virtual machine (VM) relocation among physical nodes in a data center. Existing live migration technology, however, requires a long time to change the execution host of a VM, it ...
DVM: towards a datacenter-scale virtual machine
VEE '12

As cloud-based computation becomes increasingly important, providing a general computational interface to support datacenter-scale programming has become an imperative research agenda. Many cloud systems use existing virtual machine monitor (VMM) ...
DVM: towards a datacenter-scale virtual machine
VEE '12: Proceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual Execution Environments

As cloud-based computation becomes increasingly important, providing a general computational interface to support datacenter-scale programming has become an imperative research agenda. Many cloud systems use existing virtual machine monitor (VMM) ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '17: Proceedings of the Conference of the ACM Special Interest Group on Data Communication

August 2017

515 pages

ISBN:9781450346535

DOI:10.1145/3098822

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

SIGCOMM '17

Sponsor:

SIGCOMM

SIGCOMM '17: ACM SIGCOMM 2017 Conference

August 21 - 25, 2017

CA, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

146
Total Citations
View Citations
2,863
Total Downloads

Downloads (Last 12 months)475
Downloads (Last 6 weeks)63

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu JSun WYe THu W(2024)Dynamic capacity sharing with multi-wavelength integrated transmitters in hybrid datacenter networksJournal of Optical Communications and Networking10.1364/JOCN.52844316:10(990)Online publication date: 19-Sep-2024
https://doi.org/10.1364/JOCN.528443
Qin LGu HYu XCai ZLiu J(2024)Orchid: enhancing HPC interconnection networks through infrequent topology reconfigurationJournal of Optical Communications and Networking10.1364/JOCN.51603116:6(644)Online publication date: 21-May-2024
https://doi.org/10.1364/JOCN.516031
Taka HInoue TOki E(2024)Design model of a twisted and folded Clos network with multi-step grouped intermediate switches guaranteeing admissible blocking probabilityJournal of Optical Communications and Networking10.1364/JOCN.51389816:3(328)Online publication date: 21-Feb-2024
https://doi.org/10.1364/JOCN.513898
De Marchi FLi JBai WXia YSekar VYu MSeneviratne AVeitch D(2024)POSTER: Opportunistic Credit-Based Transport for Reconfigurable Data Center Networks with TidalProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673714(4-6)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672202.3673714
Lei YDe Marchi FJoshi RLi JChandrasekaran BXia YSekar VYu MSeneviratne AVeitch D(2024)DEMO: An Open Research Framework for Optical Data Center NetworksProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673712(86-88)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672202.3673712
De Marchi FBai WLi JXia Y(2024)Rethinking Transport Protocols for Reconfigurable Data Centers: An Empirical StudyProceedings of the 1st SIGCOMM Workshop on Hot Topics in Optical Technologies and Applications in Networking10.1145/3672201.3674120(7-13)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672201.3674120
Mellette WForencich AAthapathu RSnoeren APapen GPorter GSekar VYu MSeneviratne AVeitch D(2024)Realizing RotorNet: Toward Practical Microsecond Scale Optical NetworkingProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672273(392-414)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672273
Amir DSaran NWilson TKleinberg RShrivastav VWeatherspoon HSekar VYu MSeneviratne AVeitch D(2024)Shale: A Practical, Scalable Oblivious Reconfigurable NetworkProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672248(449-464)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672248
Li JGong HDe Marchi FGong ALei YBai WXia YSekar VYu MSeneviratne AVeitch D(2024)Uniform-Cost Multi-Path Routing for Reconfigurable Data Center NetworksProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672245(433-448)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672245
Liang CSong XCheng JWang MLiu YLiu ZZhao SCui YSekar VYu MSeneviratne AVeitch D(2024)NegotiaToR: Towards A Simple Yet Effective On-demand Reconfigurable Datacenter NetworkProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672222(415-432)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672222
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents