research-article

Open access

XSEDE value added, cost avoidance, and return on investment

Authors:

Craig A. Stewart,

Richard Knepper,

Richard L. Moore,

Timothy M. CockerillAuthors Info & Claims

XSEDE '15: Proceedings of the 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure

Article No.: 23, Pages 1 - 8

https://doi.org/10.1145/2792745.2792768

Published: 26 July 2015 Publication History

Abstract

It is difficult for large research facilities to quantify a return on the investments that fund their operations. This is because there can be a time lag of years or decades between an innovation or discovery and the realization of its value through practical application. This report presents a three-part methodology that attempts to assess the value of federal investment in XSEDE: 1) a qualitative examination of the areas where XSEDE adds value to the activities of the open research community, 2) a "thought model" examining the cost avoidance realized by the National Science Foundation (NSF) through the centralization and coordination XSEDE provides, and 3) an assessment of the value XSEDE provides to Service Providers in the XD ecosystem. XSEDE adds significantly to the US research community because it functions as a unified interface to the XD ecosystem and because of its scale. A partly quantitative, partly qualitative analysis suggests the Return on Investment of NSF spending on XSEDE is greater than 1.0. -- indicating that the aggregate value received by the nation from XSEDE is greater than the cost of direct federal investment in XSEDE.

References

[1]

National Science Foundation. 2006. Cyberinfrastructure: From Supercomputing to the TeraGrid. http://www.nsf.gov/news/special_reports/cyber/fromsctotg.jsp

[2]

Stewart, C. A., R. D. Knepper, M. R. Link, M. A. Pierce, E. A. Wernert, N. Wilkins-Diehr. 2014. Cyberinfrastructure, Science Gateways, Campus Bridging, and Cloud Computing. In: Encyclopedia of Information Science and Technology. Hershey, PA. http://hdl.handle.net/2022/18608

[3]

Stewart, C. A., S. Simms, B. Plale, M. R. Link, D. Hancock and G. C. Fox. What is Cyberinfrastructure? In: Proceedings of SIGUCCS 2010. Norfolk, VA, 24--27 Oct 2010. http://dx.doi.org/10.1145/1878335.1878347

Digital Library

[4]

Catlett, C., W. E. Allcock, P. Andrews, R. Aydt, R. Bair, N. Balac, B. Banister, T. Barker, M. Bartelt, P. Beckman, F. Berman, G. Bertoline, A. Blatecky, J. Boisseau, J. Bottum, S. Brunett, J. Bunn, M. Butler, D. Carver, J. Cobb, T. Cockerill, P. F. Couvares, M. Dahan, D. Diehl, T. Dunning, I. Foster, K. Gaither, D. Gannon, S. Goasguen, M. Grobe, D. Hart, M. Heinzel, C. Hempel, W. Huntoon, J. Insley, C. Jordan, I. Judson, A. Kamrath, N. Karonis, C. Kesselman, P. Kovatch, L. Lane, S. L. Lathrop, D. Lifka, L. Liming, M. Livny, R. Loft, D. Marcusiu, J. Marsteller, S. Martin, D. S. McCaulay, J. McGee, L. McGinnis, M. A. McRobbie, P. Messina, R. Moore, J. P. Navarro, J. Nichols, M. E. Papka, R. Pennington, G. Pike, J. Pool, R. Reddy, D. Reed, T. Rimovsky, E. Roberts, R. Roskies, S. Sanielevici, J. R. Scott, A. Shankar, M. Sheddon, M. Showerman, D. Simmel, A. Singer, D. Skow, S. Smallen, W. Smith, C. Song, R. Stevens, C. A. Stewart, R. B. Stock, N. Stone, J. Towns, T. Urban, M. Vildibill, E. Walker, V. Welch, N. Wilkins-Diehr, R. Williams, L. Winkler, L. Zhao and A. Zimmerman. 2008. TeraGrid: Analysis of Organization, System Architecture, and Middleware Enabling New Types of Applications. L. Grandinetti, ed. Advances in Parallel Computing. Amsterdam, The Netherlands. http://hdl.handle.net/2022/14524

[5]

National Science Foundation. 2008. TeraGrid Phase III: eXtreme Digital Resources for Science and Engineering (XD) Program Solicitation NSF 08-571. http://www.nsf.gov/pubs/2008/nsf08571/nsf08571.htm

[6]

National Science Foundation. 2014. XSEDE: eXtreme Science and Engineering Discovery Environment (Award Abstract). http://www.nsf.gov/awardsearch/showAward?AWD_ID=1053575

[7]

National Science Foundation. 2014. Evaluating and Enhancing the eXtreme Digital (XD) Cyberinfrastructure for Maximum Usability and Science Impact (Award Abstract). http://www.nsf.gov/awardsearch/showAward?AWD_ID=0946505

[8]

Towns, J., T. Cockerill, M. Dahan, I. Foster, K. Gaither, A. Grimshaw, V. Hazlewood, S. Lathrop, D. Lifka, G. D. Peterson, R. Roskies, J. R. Scott and N. Wilkins-Diehr. 2014. XSEDE: Accelerating Scientific Discovery. In: Computing in Science & Engineering 16(5): 62--74.

[9]

Furlani, T., M. Green, G. von Laszewski, V. Chaudhary and M. Jones. 2010. Technology Audit and Insertion Service for TeraGrid (Award Abstract). http://www.nsf.gov/awardsearch/showaward?awd_id=1025159

[10]

XSEDE. Service Provider Forum: Charter, Membership, and Governance, V12, 20 July 2014. http://hdl.handle.net/2142/49980

[11]

XSEDE. Requesting Membership in the XSEDE Federation, V2, 10 July 2014. http://hdl.handle.net/2142/49981

[12]

National Science Foundation. 2014. High Performance Computing System Acquisition: Continuing the Building of a More Inclusive Computing Environment for Science and Engineering Program Solicitation NSF 14-536. http://www.nsf.gov/pubs/2014/nsf14536/nsf14536.htm

[13]

National Science Foundation. 2014. Major Research Instrumentation Program: (MRI) Instrument Acquistion or Development Program Solicitation NSF 15-504. http://www.nsf.gov/pubs/2015/nsf15504/nsf15504.htm

[14]

Jones, A. Keynote Presentation. TeraGrid '07 (Madison, WI, 4 June 2007. http://www.union.wisc.edu/teragrid07/

[15]

National Science Foundation. 2015. High Performnace Computing System Acquisition: Enhancing the Petascale Computing Environment for Science and Engineering Program Solicitation 11-511. http://www.nsf.gov/pubs/2011/nsf11511/nsf11511.htm

[16]

National Science Foundation. 2014. Directorate for Computer and Information Science and Engineering (CISE): CISE Funding. http://www.nsf.gov/about/budget/fy2014/pdf/18_fy2014.pdf

[17]

Kinney, M. F. and C. A. Raiborn. 2011. Cost Accounting. South-Western, Mason, OH, 2011. ISBN: 978-1-111-97172.

[18]

Newby, G., A. Apon, N. Berente, R. Eigenmann, S. Fratkin, D. Lifka and C. A. Stewart. 2014. Return on Investment from Academic Supercomputing: Panel presented at: SC14 (Supercomputing 2014), New Orleans, LA, November 21, 2014. http://hdl.handle.net/2022/19242

[19]

Apon, A., S. Ahalt, V. Dantuluri, C. Gurdgiev, M. Limayem, L. Ngo and M. Stealey. 2010. High Performance Computing Instrumentation and Research Productivity in U.S. Universities. Journal of Information Technology Impact, 10(2), 87--98. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1679248

[20]

Apon, A. W., L. B. Ngo, M. E. Payne and P. W. Wilson. 2014. Assessing the effect of high performance computing capabilities on academic research output. Empirical Economics 48: 283--312.

[21]

Joseph, E. C., C. Dekate and S. Conway. Real-World Examples of Supercomputers Used For Economic and Societal Benefits: A Prelude to What the Exascale Era Can Provide (Special Study). International Data Corporation (IDC). 2014. http://casc.org/wp-content/uploads/2014/07/IDCReportRealWorldExamplesOfBenefitsOfSupercomputers.pdf

[22]

Tabor Griffin Communications. High-Performance Computing Contributions to Society. Tabor Griffin Communications, 1998. http://www.tgc.com/hpcbook/toc1.html

[23]

PRACE. PRACE (Partnership for Advanced Computing in Europe). Home page. http://www.prace-ri.eu

[24]

Compute Canada. Compute Canada. Home Page. https://www.computecanada.ca

[25]

XSEDE. Extended Collaborative Support (ECSS). http://www.xsede.org/ecss

[26]

The Arabidopsis Information Resource (TAIR). Home Page. http://www.arabidopsis.org

[27]

NCGAS. National Center for Genome Analysis Support (NCGAS). Home Page. http://ncgas.org

[28]

Wilkins-Diehr, N. 2007. Special Issue: Science Gateways --- Common Community Interfaces to Grid Resources: Editorials. Concurr. Comput. : Pract. Exper., 19(6), 743--749. http://dl.acm.org/citation.cfm?id=1232528

Digital Library

[29]

XSEDE. XSEDE Campus Champions. http://www.xsede.org/campus-champions

[30]

XSEDE. XSEDE Conferences. https://conferences.xsede.org

[31]

Venkatesh, V., Morris, M. G., Davis, F. D., Davis, G. B. 2003. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425--478. http://csdltechreports.googlecode.com/svn/trunk/techreports/2005/05-06/doc/Venkatesh2003.pdf

Digital Library

[32]

O'Hara, M., C. Yao and M. Ye. What's Not There: Odd Lots and Market Data. 2014. The Journal of Finance, 69(5), 2199--2236. http://onlinelibrary.wiley.com/doi/10.1111/jofi.12185/abstract

Cited By

Snapp-Childs WHart DCosta CWernert JJankowski HTowns JStewart C(2024)Evaluating Return on Investment for Cyberinfrastructure Using the International Integrated Reporting FrameworkSN Computer Science10.1007/s42979-024-02889-z5:5Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1007/s42979-024-02889-z
Stewart CCosta CWernert JSnapp-Childs WBland MBlood PCampbell TCouvares PFischer JHancock DHart DJankowski HKnepper RMcMullen DMehringer SPierce MRogers GSinkovits RTowns J(2023)Use of accounting concepts to study research: return on investment in XSEDE, a US cyberinfrastructure serviceScientometrics10.1007/s11192-022-04539-8128:6(3225-3255)Online publication date: 14-Feb-2023
https://doi.org/10.1007/s11192-022-04539-8
Stewart CCosta CWernert JHancock DMcMullen DBlood PSinkovits RMehringer SKnepper RFischer JBland MRogers GCouvares PCampbell TJankowski HSnapp-Childs WTowns J(2022)Metrics of financial effectiveness: Return On Investment in XSEDE, a national cyberinfrastructure coordination and support organizationPractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3530287(1-9)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3530287
Show More Cited By

Index Terms

XSEDE value added, cost avoidance, and return on investment
1. General and reference
  1. Cross-computing tools and techniques
    1. Metrics
2. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. System management
        Technology audits

Recommendations

Assessment of financial returns on investments in cyberinfrastructure facilities: A survey of current methods
PEARC '19: Practice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)

In recent years, considerable attention has been given to assessing the value of investments in cyberinfrastructure (CI). This paper includes a survey of current methods for the assessment of financial returns on investment (ROI) in CI. Applying the ...
Return on Investment in Research Cyberinfrastructure: State of the Art
PEARC '22: Practice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You

“What is the Return On Investment (ROI) for a cyberinfrastructure system or service?” seems like a natural question to ask. Existing literature shows strong evidence of good return on investment in cyberinfrastructure. This paper summarizes key points ...
Metrics of financial effectiveness: Return On Investment in XSEDE, a national cyberinfrastructure coordination and support organization
PEARC '22: Practice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You

This paper explores the financial effectiveness of a national advanced computing support organization within the United States (US) called the eXtreme Science and Engineering Discovery Environment (XSEDE). XSEDE was funded by the National Science ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

XSEDE '15: Proceedings of the 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure

July 2015

296 pages

ISBN:9781450337205

DOI:10.1145/2792745

General Chair:
Gregory D. Peterson
National Institute of Computational Sciences

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

San Diego Super Computing Ctr: San Diego Super Computing Ctr
HPCWire: HPCWire
Omnibond: Omnibond Systems, LLC
SGI
Internet2
Indiana University: Indiana University
CASC: The Coalition for Academic Scientific Computation
NICS: National Institute for Computational Sciences
Intel: Intel
DDN: DataDirect Networks, Inc
DELL
CORSA: CORSA Technology
ALLINEA: Allinea Software
Cray
RENCI: Renaissance Computing Institute

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

XSEDE '15

Sponsor:

San Diego Super Computing Ctr
HPCWire
Omnibond
Indiana University
CASC
NICS
Intel
DDN
CORSA
ALLINEA
RENCI

XSEDE '15: Extreme Science Engineering Discovery Environment 2015 Conference

July 26 - 30, 2015

Missouri, St. Louis

Acceptance Rates

XSEDE '15 Paper Acceptance Rate 49 of 70 submissions, 70%;

Overall Acceptance Rate 129 of 190 submissions, 68%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
462
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)7

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Snapp-Childs WHart DCosta CWernert JJankowski HTowns JStewart C(2024)Evaluating Return on Investment for Cyberinfrastructure Using the International Integrated Reporting FrameworkSN Computer Science10.1007/s42979-024-02889-z5:5Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1007/s42979-024-02889-z
Stewart CCosta CWernert JSnapp-Childs WBland MBlood PCampbell TCouvares PFischer JHancock DHart DJankowski HKnepper RMcMullen DMehringer SPierce MRogers GSinkovits RTowns J(2023)Use of accounting concepts to study research: return on investment in XSEDE, a US cyberinfrastructure serviceScientometrics10.1007/s11192-022-04539-8128:6(3225-3255)Online publication date: 14-Feb-2023
https://doi.org/10.1007/s11192-022-04539-8
Stewart CCosta CWernert JHancock DMcMullen DBlood PSinkovits RMehringer SKnepper RFischer JBland MRogers GCouvares PCampbell TJankowski HSnapp-Childs WTowns J(2022)Metrics of financial effectiveness: Return On Investment in XSEDE, a national cyberinfrastructure coordination and support organizationPractice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You10.1145/3491418.3530287(1-9)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3491418.3530287
Stewart CKnepper RLink MPierce MWernert EWilkins-Diehr N(2019)Cyberinfrastructure, Cloud Computing, Science Gateways, Visualization, and Cyberinfrastructure Ease of UseAdvanced Methodologies and Technologies in Network Architecture, Mobile Computing, and Data Analytics10.4018/978-1-5225-7598-6.ch012(157-170)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-7598-6.ch012
Stewart CHancock DWernert JFurlani TLifka DSill ABerente NMcMullen DCheatham TApon APayne RSlavin SFurlani T(2019)Assessment of financial returns on investments in cyberinfrastructure facilitiesPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3332228(1-8)Online publication date: 28-Jul-2019
https://dl.acm.org/doi/10.1145/3332186.3332228
Stewart CKnepper RLink MPierce MWernert EWilkins-Diehr N(2018)Cyberinfrastructure, Cloud Computing, Science Gateways, Visualization, and Cyberinfrastructure Ease of UseEncyclopedia of Information Science and Technology, Fourth Edition10.4018/978-1-5225-2255-3.ch092(1063-1074)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-2255-3.ch092
Stewart CKrefeldt MSanielevici S(2018)Your Good Health is a Workforce IssueProceedings of the Practice and Experience on Advanced Research Computing: Seamless Creativity10.1145/3219104.3219107(1-8)Online publication date: 22-Jul-2018
https://dl.acm.org/doi/10.1145/3219104.3219107
Stewart CHancock DWernert JLink MWilkins-Diehr NMiller TGaither KSnapp-Childs W(2018)Return on Investment for Three Cyberinfrastructure Facilities: A Local Campus Supercomputer, the NSF-Funded Jetstream Cloud System, and XSEDE (the eXtreme Science and Engineering Discovery Environment)2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC.2018.00031(223-236)Online publication date: Dec-2018
https://doi.org/10.1109/UCC.2018.00031
Thota AFulton BWeakley LHenschel RHancock DAllen MTillotson JLink MStewart CFox LKing CWhite S(2016)A PetaFLOPS Supercomputer as a Campus ResourceProceedings of the 2016 ACM SIGUCCS Annual Conference10.1145/2974927.2974956(61-68)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1145/2974927.2974956

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents