Many scientific high-throughput applications can benefit from the elastic nature of Cloud resources, especially when there is a need to reduce time to completion. Cost considerations are usually a major issue in such endeavors, with networking often a major component; for data-intensive applications, egress networking costs can exceed the compute costs. Dedicated network links provide a way to lower the networking costs, but they do add complexity. In this paper we provide a description of a 100 fp32 PFLOPS Cloud burst in support of IceCube production compute, that used Internet2 Cloud Connect service to provision several logically-dedicated network links from the three major Cloud providers, namely Amazon Web Services, Microsoft Azure and Google Cloud Platform, that in aggregate enabled approximately 100 Gbps egress capability to on-prem storage. It provides technical details about the provisioning process, the benefits and limitations of such a setup and an analysis of the costs incurred.

References

[1]

R. Pordes, "The open science grid." Journal of Physics: Conference Series 78 pp 012057 (2007), https://doi.org/10.1088/1742-6596/78/1/012057

Crossref

Google Scholar

[2]

J. Towns, "XSEDE: Accelerating Scientific Discovery," in Computing in Science & Engineering, vol. 16, no. 5, pp. 62-74, Sept.-Oct. 2014, https://doi.org/10.1109/MCSE.2014.80

Crossref

Google Scholar

[3]

B. Posey, "Addressing the Challenges of Executing a Massive Computational Cluster in the Cloud," 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Washington, DC, 2018, pp. 253-262, https://doi.org/10.1109/CCGRID.2018.00040

Digital Library

Google Scholar

[4]

I. Sfiligoi, F.Würthwein, B. Riedel, and D. Schultz, "Running a Pre-exascale, Geographically Distributed, Multi-cloud Scientific Simulation." Lecture Notes in Computer Science, vol 12151. https://doi.org/10.1007/978-3-030-50743-5_2

Digital Library

Google Scholar

[5]

B. Posey, "On-Demand Urgent High Performance Computing Utilizing the Google Cloud Platform," IEEE/ACM HPC for Urgent Decision Making (UrgentHPC), Denver, CO, USA, 13-23 (2019), https://doi.org/10.1109/UrgentHPC49580.2019.00008

Crossref

Google Scholar

[6]

I. Sfiligoi, "Demonstrating a Pre-Exascale, Cost-Effective Multi-Cloud Environment for Scientific Computing." PEARC '20: Practice and Experience in Advanced Research Computing, July 2020, pp. 85-90, https://doi.org/10.1145/3311790.3396625

Digital Library

Google Scholar

[7]

Internet2, "Cloud Connect." https://internet2.edu/services/cloud-connect/

Google Scholar

[8]

P. Ruth and M. Cevik, "Experimenting with AWS Direct Connect using Chameleon, ExoGENI, and Internet2 Cloud Connect," 2019 IEEE 27th International Conference on Network Protocols (ICNP), Chicago, IL, USA, 2019, pp. 1-2, https://doi.org/10.1109/ICNP.2019.8888095

Crossref

Google Scholar

[9]

M. G. Aartsen, 2017. The IceCube Neutrino Observatory: instrumentation and online systems. J. Instrum. 12, P03012-P03012.

Crossref

Google Scholar

[10]

D. Chirkin, "Photon tracking with GPUs in IceCube." Nucl. Inst. and Meth. in Phys. Res. Sec. A, Vol. 725, 141-143 (2013), https://doi.org/10.1016/j.nima.2012.11.170

Crossref

Google Scholar

[11]

I. Sfiligoi, D. C. Bradley, B. Holzman, P. Mhashilkar, S. Padhi and F. Wurthwein, "The Pilot Way to Grid Resources Using glideinWMS," 2009 WRI World Congress on Computer Science and Information Engineering, Los Angeles, CA, 2009, pp. 428-432, https://doi.org/10.1109/CSIE.2009.950

Digital Library

Google Scholar

[12]

I. Sfiligoi, "Demonstrating 100 Gbps in and out of the public Clouds." PEARC '20: Practice and Experience in Advanced Research Computing, July 2020, pp. 495-499, https://doi.org/10.1145/3311790.3399612

Digital Library

Google Scholar

[13]

J. Chung, H. Owen and R. Clark, "SDX architectures: A qualitative analysis," SoutheastCon 2016, Norfolk, VA, 2016, pp. 1-8, https://doi.org/10.1109/SECON.2016.7506749

Crossref

Google Scholar

Cited By

View all

Adair MRodero IParashar MMelgar D(2023)Accelerating Data-Intensive Seismic Research Through Parallel Workflow Optimization and Federated CyberinfrastructureProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624276(1970-1977)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624276
Coleman AEser JMayotte ESarazin FSchröder FSoldin DVenters TAloisio RAlvarez-Muñiz JAlves Batista RBergman DBertaina MCaccianiga LDeligny ODembinski HDenton Pdi Matteo AGlobus NGlombitza JGolup GHaungs AHörandel JJaffe TKelley JKrizmanic JLu LMatthews JMariş IMussa ROikonomou FPierog TSantos ETinyakov PTsunesada YUnger MYushkov AAlbrow MAnchordoqui LAndeen KArnone EBarghini DBechtol EBellido JCasolino MCastellina ACazon LConceição RCremonini RDujmovic HEngel RFarrar GFenu FFerrarese SFujii TGardiol DGritsevich MHomola PHuege TKampert KKang DKido EKlimov PKotera KKozelov BLeszczyńska AMadsen JMarcelli LMarisaldi MMartineau-Huynh OMayotte SMulrey KMurase KMuzio MOgio SOlinto AOnel YPaul TPiotrowski LPlum MPont BReininghaus MRiedel BRiehn FRoth MSako TSchlüter FShoemaker DSidhu JSidelnik ITimmermans CTkachenko OVeberic DVerpoest SVerzi VVícha JWinn DZas EZotov M(2023)Ultra high energy cosmic rays The intersection of the Cosmic and Energy FrontiersAstroparticle Physics10.1016/j.astropartphys.2023.102819149(102819)Online publication date: Jul-2023
https://doi.org/10.1016/j.astropartphys.2023.102819
(2023)Publisher's Note:Astroparticle Physics10.1016/j.astropartphys.2022.102794147(102794)Online publication date: May-2023
https://doi.org/10.1016/j.astropartphys.2022.102794

Recommendations

Demonstrating a Pre-Exascale, Cost-Effective Multi-Cloud Environment for Scientific Computing: Producing a fp32 ExaFLOP hour worth of IceCube simulation data in a single workday
PEARC '20: Practice and Experience in Advanced Research Computing 2020: Catch the Wave

Scientific computing needs are growing dramatically with time and are expanding in science domains that were previously not compute intensive. When compute workflows spike well in excess of the capacity of their local compute resource, capacity should be ...
Pragmatic assessment of research intensive areas in cloud: a systematic review

Cloud computing is a name given to a set of systems for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly ...
Cost-Aware Elastic Cloud Provisioning for Scientific Workloads
CLOUD '15: Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing

Cloud computing provides an efficient model to host and scale scientific applications. While cloud-based approaches can reduce costs as users pay only for the resources used, it is often challenging to scale execution both efficiently and cost-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

PEARC '21: Practice and Experience in Advanced Research Computing 2021: Evolution Across All Dimensions

July 2021

310 pages

ISBN:9781450382922

DOI:10.1145/3437359

Editors:
Joseph Paris
Northwestern University
,
Jackie Milhans
Northwestern University
,
Betsy Hillery
Purdue University
,
Sharon Broude Geva
University of Michigan
,
Patrick Schmitz
Semper Cogito
,
Robert Sinkovits
San Diego Supercomputer Center

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

PEARC '21

Sponsor:

PEARC '21: Practice and Experience in Advanced Research Computing

July 18 - 22, 2021

MA, Boston, USA

Acceptance Rates

Overall Acceptance Rate 133 of 202 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
303
Total Downloads

Downloads (Last 12 months)138
Downloads (Last 6 weeks)20

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Adair MRodero IParashar MMelgar D(2023)Accelerating Data-Intensive Seismic Research Through Parallel Workflow Optimization and Federated CyberinfrastructureProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624276(1970-1977)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624276
Coleman AEser JMayotte ESarazin FSchröder FSoldin DVenters TAloisio RAlvarez-Muñiz JAlves Batista RBergman DBertaina MCaccianiga LDeligny ODembinski HDenton Pdi Matteo AGlobus NGlombitza JGolup GHaungs AHörandel JJaffe TKelley JKrizmanic JLu LMatthews JMariş IMussa ROikonomou FPierog TSantos ETinyakov PTsunesada YUnger MYushkov AAlbrow MAnchordoqui LAndeen KArnone EBarghini DBechtol EBellido JCasolino MCastellina ACazon LConceição RCremonini RDujmovic HEngel RFarrar GFenu FFerrarese SFujii TGardiol DGritsevich MHomola PHuege TKampert KKang DKido EKlimov PKotera KKozelov BLeszczyńska AMadsen JMarcelli LMarisaldi MMartineau-Huynh OMayotte SMulrey KMurase KMuzio MOgio SOlinto AOnel YPaul TPiotrowski LPlum MPont BReininghaus MRiedel BRiehn FRoth MSako TSchlüter FShoemaker DSidhu JSidelnik ITimmermans CTkachenko OVeberic DVerpoest SVerzi VVícha JWinn DZas EZotov M(2023)Ultra high energy cosmic rays The intersection of the Cosmic and Energy FrontiersAstroparticle Physics10.1016/j.astropartphys.2023.102819149(102819)Online publication date: Jul-2023
https://doi.org/10.1016/j.astropartphys.2023.102819
(2023)Publisher's Note:Astroparticle Physics10.1016/j.astropartphys.2022.102794147(102794)Online publication date: May-2023
https://doi.org/10.1016/j.astropartphys.2022.102794

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Demonstrating a Pre-Exascale, Cost-Effective Multi-Cloud Environment for Scientific Computing: Producing a fp32 ExaFLOP hour worth of IceCube simulation data in a single workday

Pragmatic assessment of research intensive areas in cloud: a systematic review

Cost-Aware Elastic Cloud Provisioning for Scientific Workloads