research-article

The Pacific Research Platform: Making High-Speed Networking a Reality for the Scientist

Authors:

Camille Crittenden,

Thomas DeFanti,

Philip Papadopoulos,

Frank WürthweinAuthors Info & Claims

PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing: Seamless Creativity

Article No.: 29, Pages 1 - 8

https://doi.org/10.1145/3219104.3219108

Published: 22 July 2018 Publication History

Abstract

While NSF's recent Campus Cyberinfrastructure investments have catalyzed an enormous leap in campus networking capabilities, it remains necessary to integrate these capabilities into the routine workflows of researchers transferring data among their remote collaborators, data repositories, and visualization facilities. The Pacific Research Platform (PRP) is a program to develop a science-driven, data-centric "freeway system" by federating campus Science DMZs into a regional Science DMZ. Research collaborations across PRP partners serve as use cases for deploying the hardware/software and addressing security/policy/social issues to achieve high-performance data transfers between researchers. The regional PRP represents a manageable scale for initial deployment efforts and capturing lessons learned, but the PRP also informs national deployments --i.e., an eventual National Research Platform (NRP). The recent First NRP Workshop included many campuses and networking organizations and addressed scientific requirements, scaling approaches, and socio-technical issues to extend the PRP to a national level; a second NRP Workshop will be held in August 2018. In this paper, we describe efforts to build the PRP, the social and technical approaches, the domain science applications of PRP's capabilities, and results of the recent NRP workshop.

References

[1]

NSF Campus Cyberinfrastructure program, including solicitations 18--508, 16--567, 15--534, 14--521 and predecessors. Retrieved from https://www.nsf.gov/funding/pgm_summ.jsp?pims_id=504748.

[2]

ESNet. Science DMZ: A Scalable Network Design Model for Optimizing Science Data Transfers. Retrieved from https://fasterdata.es.net/science-dmz/.

[3]

The First National Research Platform Workshop: Toward a National Big Data Superhighway, August 7--8, 2017, Bozeman MT. Retrieved from http://prp.ucsd.edu/workshop-reports/NRP%20Workshop%20Report_FINAL-23-Oct-17.pdf/view.

[4]

ESNet. Science Engagement. Retrieved from https://www.es.net/science-engagement/.

[5]

ESNet. CrossConnects Workshop Series. Retrieved from https://www.es.net/science-engagement/programs-and-workshops/crossconnects-workshop-series/.

[6]

Building the Pacific Research Platform: A Workshop Towards Deploying a Science-Driven Regional 'Big Data Freeway', October 14--16, 2015, La Jolla, CA. Retrieved from http://prp.ucsd.edu/workshop-reports/2015-prp-workshop-report/view.

[7]

Pacific Research Platform Version 2 (PRPv2) Workshop: Summary of Workshop Presentation, February 21, 2017, La Jolla, CA. Retrieved from http://prp.ucsd.edu/workshop-reports/prp-v2-workshop-report-2-21-17-final.pdf/view

[8]

Big Data and the Earth Sciences: Grand Challenges Workshop, May 31-June 2, 2017, La Jolla, CA. Retrieved from http://prp.ucsd.edu/workshop-reports/BigDataWorkshop2017_Report_FINAL_082417.pdf/view.

[9]

Larry Smarr, Andrew A. Chien, Tom DeFanti, Jason Leigh, Philip M. Papadopoulos, 2003. The OptIPuter. Comm. of the ACM, 46, 58--67.

Digital Library

[10]

Phil Papadopoulos, B. Dunne, Larry Smarr, J. Ford, and S. Fainman. Quartzite: A Campus-Scale Hybrid Networking Infrastructure. Optical Fiber Communication/National Fiber Optic Engineers Conference 2008, 24--28 Feb. 2008, pp 1--3.

[11]

Viewing Research Bandwidth through a New Prism. Retrieved from ucsdnews.ucsd.edu/pressrelease/viewing_research_through_a_new_prism.

[12]

CENIC Network. Retrieved from https://cenic.org/network/network-overview.

[13]

ESNet. Science DMZ: Data Transfer Nodes. Retrieved from https://fasterdata.es.net/science-dmz/DTN/.

[14]

UCSD PRP. Summer 2017 FIONA build. Retrieved from http://prp.ucsd.edu/solution-resources.

[15]

Brian Tierney, Ezra Kissel, Martin Swany, and Eric Pouyol. 2012. Efficient Data Transfer Protocols for Big Data. IEEE 8th International Conference on e-Science, 8--12 October, 2012, Retrieved from https://www.es.net/assets/pubs_presos/eScience-networks.pdf.

[16]

Se-Yeung Yu, Nevil Brownlee, Aniket Mahanti. 2013. Comparative performance analysis of high-speed transfer protocols for big data. IEEE 38th Conference on Local Computer Networks, 21--24 October 2013, Retrieved from http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6761252.

[17]

ESnet. Data Transfer Tools. Retrieved from https://fasterdata.es.net/data-transfer-tools/.

[18]

Globus. Data Transfer with Globus. Retrieved from https://www.globus.org/data-transfer.

[19]

CERN. Fast Data Transfer -- FDT. Retrieved from monalisa.cern.ch/FDT/.

[20]

KeyCDN. UDP File Transfer -- Explained. Retrieved from https://www.keycdn.com/support/udp-file-transfer/.

[21]

UDT: UDP-based Data Transfer. Retrieved from http://udt.sourceforge.net/.

[22]

ESnet. MaDDash: Monitoring and Debugging Dashboard. Retrieved from software.es.net/maddash/.

[23]

Van Jacobson, Internet Hall of Fame Innovator. Retrieved from internethalloffame.org/inductees/van-jacobson.

[24]

CENIC. CENIC perfSONAR Dashboard. Retrieved from https://ps-dashboard.cenic.net/.

[25]

Kubernetes: Production-Grade Container Orchestration. Retrieved from https://kubernetes.io.

[26]

JupyterHub. Retrieved from http://jupyterhub.readthedocs.io/en/latest/.

[27]

UCSD PRP. To see the latest PRP traceroute visualization, visit the Kubernetes traceroute tool page at https://traceroute.nautilus.optiputer.net/.

[28]

CIO Magazine. What are containers and why do you need them? Retrieved from www.cio.com/article/2924995/software/what-are-containers-and-why-do-you-need-them.html.

[29]

The Linux Foundation. Cloud Native Computing Foundation. Retrieved from www.cncf.io/.

[30]

Techcrunch, Retrieved from techcrunch.com/2017/09/20/kubernetes-gains-momentum-as-big-name-vendors-join-cloud-native-computing-foundation/.

[31]

Rook. File, Block and Object Storage Services for your Cloud-Native Environments. Retrieved from https://rook.io/.

[32]

Wikipedia. Ceph (software). Retrieved from https://en.wikipedia.org/wiki/Ceph_(software).

[33]

Bro. The Bro Network Security Monitor. Retrieved from www.bro.org.

[34]

Trusted Computing Group. Trusted Platform Module (TPM). Retrieved from https://trustedcomputinggroup.org/trusted-platform-module-tpm-summary/.

[35]

Github. CoreOS/Kubernetes-dtc-admission. Retrieved from https://github.com/coreos/kubernetes-dtc-admission/.

[36]

William Poehlman, Alex Feltus, Stephen Ficklin, 2017. Large-Scale Genomics Experiments Enabled by OSG Connect Resources. 2017 OSG All-Hands Meeting, 6--9 March 2017, La Jolla, CA. Retrieved from https://indico.fnal.gov/event/12973/session/28/contribution/62.

[37]

High Performance Wireless Research & Education Network. Retrieved from hpwren.ucsd.edu.

[38]

Trisha Gorman 2018. Science Engagement: Best Practices and Scalability in the 21st Century. Retrieved from http://prp.ucsd.edu/Science%20Engagement-PRP-NRP_Report-FINAL%203-20-2018b.pdf.

[39]

CHASE-CI NSF award 1730158, retrieved from https://nsf.gov/awardsearch/showAward?AWD_ID=1730158.

[40]

The Quilt Advanced Regional Networking, retrieved from https://www.thequilt.net.

Cited By

Gough E(2024)Evaluation of Kubernetes Schedulers for a Community Cloud Computing ModelPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670520(1-7)Online publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1145/3626203.3670520
Sfiligoi IWürthwein FDost JLin BSchultz D(2024)Demand-driven provisioning of Kubernetes-like resources in OSGEPJ Web of Conferences10.1051/epjconf/202429507014295(07014)Online publication date: 6-May-2024
https://doi.org/10.1051/epjconf/202429507014
Gonzalez-Ferrer JLehrer JO’Farrell APaten BTeodorescu MHaussler DJonsson VMostajo-Radji M(2024)SIMS: A deep-learning label transfer tool for single-cell RNA sequencing analysisCell Genomics10.1016/j.xgen.2024.1005814:6(100581)Online publication date: Jun-2024
https://doi.org/10.1016/j.xgen.2024.100581
Show More Cited By

Index Terms

The Pacific Research Platform: Making High-Speed Networking a Reality for the Scientist
1. Networks
  1. Network performance evaluation
    1. Network performance analysis
  2. Network properties
    1. Network manageability

Recommendations

The Organization and Management of Grid Infrastructures

Grid computing technology has become fundamental to e-Science. As the virtual organizations established by scientific communities progress from testing their applications to more routine usage, maintaining reliable and adaptive grid infrastructures ...
Distributed Computing Education, Part 4: Training Infrastructure

Teaching distributed computing requires environments that provide adequate training infrastructure, or t-infrastructure. In practice, t-infrastructure includes the computing equipment, digital communications, software, data, and support staff necessary ...
APCHI '12: Proceedings of the 10th asia pacific conference on Computer human interaction

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing: Seamless Creativity

July 2018

652 pages

ISBN:9781450364461

DOI:10.1145/3219104

General Chair:
Sergiu Sanielevici
Pittsburgh Supercomputing Center

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

PEARC '18

PEARC '18: Practice and Experience in Advanced Research Computing

July 22 - 26, 2018

PA, Pittsburgh, USA

Acceptance Rates

PEARC '18 Paper Acceptance Rate 79 of 123 submissions, 64%;

Overall Acceptance Rate 133 of 202 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
186
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gough E(2024)Evaluation of Kubernetes Schedulers for a Community Cloud Computing ModelPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670520(1-7)Online publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1145/3626203.3670520
Sfiligoi IWürthwein FDost JLin BSchultz D(2024)Demand-driven provisioning of Kubernetes-like resources in OSGEPJ Web of Conferences10.1051/epjconf/202429507014295(07014)Online publication date: 6-May-2024
https://doi.org/10.1051/epjconf/202429507014
Gonzalez-Ferrer JLehrer JO’Farrell APaten BTeodorescu MHaussler DJonsson VMostajo-Radji M(2024)SIMS: A deep-learning label transfer tool for single-cell RNA sequencing analysisCell Genomics10.1016/j.xgen.2024.1005814:6(100581)Online publication date: Jun-2024
https://doi.org/10.1016/j.xgen.2024.100581
Vega CKfoury EGomez JPezoa JFigueroa MCrichigno J(2024)Machine learning controller for data rate management in science DMZ networksComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110237242:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110237
Lee SRaza Shah SSeok WMoon JKim KRaza Shah S(2023)An Optimal Network-Aware Scheduling Technique for Distributed Deep Learning in Distributed HPC PlatformsElectronics10.3390/electronics1214302112:14(3021)Online publication date: 10-Jul-2023
https://doi.org/10.3390/electronics12143021
Roten DWells LCrawl DParsons RMarcozzi ALinn RHiers KAltintas I(2023)TrueTrees: A Scalable Workflow for the Integration of Airborne LiDAR Scanning Data into Fuel Models for Prescribed Fire Simulations2023 IEEE 19th International Conference on e-Science (e-Science)10.1109/e-Science58273.2023.10254923(1-10)Online publication date: 9-Oct-2023
https://doi.org/10.1109/e-Science58273.2023.10254923
Altintas IBlock JCrawl DCallafon R(2023)Using Dynamic Data-Driven Cyberinfrastructure for Next-Generation Wildland Fire IntelligenceHandbook of Dynamic Data Driven Applications Systems10.1007/978-3-031-27986-7_17(451-474)Online publication date: 31-Mar-2023
https://doi.org/10.1007/978-3-031-27986-7_17
Hadish JBiggs TShealy BBender MMcKnight CWytko CSmith MFeltus FHonaas LFicklin S(2022)GEMmaker: process massive RNA-seq datasets on heterogeneous computational infrastructureBMC Bioinformatics10.1186/s12859-022-04629-723:1Online publication date: 2-May-2022
https://doi.org/10.1186/s12859-022-04629-7
Sfiligoi ISchultz DWürthwein FRiedel BMishin D(2022)The anachronism of whole-GPU accountingPractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3535125(1-5)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3535125
Sfiligoi IDeFanti TWürthwein F(2022)Auto-scaling HTCondor pools using Kubernetes compute resourcesPractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3535123(1-4)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3535123
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents