research-article

Open access

Sustained Software for Cyberinfrastructure: Analyses of Successful Efforts with a Focus on NSF-funded Software

Authors:

Craig A. Stewart,

William K. Barnett,

Eric A. Wernert,

Julie A. Wernert,

Richard KnepperAuthors Info & Claims

SCREAM '15: Proceedings of the 1st Workshop on The Science of Cyberinfrastructure: Research, Experience, Applications and Models

Pages 63 - 72

https://doi.org/10.1145/2753524.2753533

Published: 16 June 2015 Publication History

Abstract

Reliable software that provides needed functionality is clearly essential for an effective distributed cyberinfrastructure (CI) that supports comprehensive, balanced, and flexible distributed CI. Effective distributed cyberinfrastructure, in turn, supports science and engineering applications. The purpose of this study was to understand what factors lead to software projects being well sustained over the long run, focusing on software created with funding from the US National Science Foundation (NSF) and/or used by researchers funded by the NSF. We surveyed NSF-funded researchers and performed in-depth studies of software projects that have been sustained over many years. Successful projects generally used open-source software licenses and employed good software engineering practices and test practices. However, many projects that have not been well sustained over time also met these criteria. The features that stood out about successful projects included deeply committed leadership and some sort of user forum or conference at least annually. In some cases, software project leaders have employed multiple financial strategies over the course of a decades-old software project. Such well-sustained software is used in major distributed CI projects that support thousands of users, and this software is critical to the operation of major distributed CI facilities in the US. The findings of our study identify some characteristics of software that is relevant to the NSF-supported research community, and that has been sustained over many years.

References

[1]

Dreher, P., V. Agarwala, S. Ahalt, G. Almes, S. Fratkin, T. Hauser, J. Odegard, J. Pepin, and C.A. Stewart. 2009. Developing a Coherent Cyberinfrastructure from Local Campuses to National Facilities: Challenges and Strategies. Available from: http://hdl.handle.net/2022/5122.

[2]

NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging. 201 Final Report. Available from: http://www.nsf.gov/cise/aci/taskforces/TaskForceReport_CampusBridging.pdf.

[3]

NASA Earth Science Data Systems - Software Reuse Working Group. 2010. Reuse Readiness Levels (RRLs), Version 1.0. Available from: https://earthdata.nasa.gov/sites/default/files/esdswg/reuse/Resources/rrls/RRLs_v1.0.pdf

[4]

Chang, V., H. Mills, and S. Newhouse. 2007. From Open Source to long-term sustainability: Review of Business Models and Case studies. All Hands Meeting 2007, OMII-UK Workshop 2007; Available from: http://eprints.soton.ac.uk/263925/.

[5]

Katz, D.S. and D. Proctor. 2014. A Framework for Discussing e-Research Infrastructure Sustainability. Available from: http://dx.doi.org/10.5334/jors.av.

[6]

National Science Foundation. 2015. Award Search. Available from: http://www.nsf.gov/awardsearch/.

[7]

Wernert, J., E Wernert, J. Fischer, H. Terhune, A. Bowers, T. Miller, C.A. Stewart. 2014. Best Practices and Models for Sustainability for Robust Cyberinfrastructure Software - Survey Dataset and Analyses. Indiana University. Available from http://hdl.handle.net/2022/17312

[8]

University of Chicago Computation Institute. 2015. Home Page. Available from: https://www.ci.uchicago.edu/.

[9]

Globus Project. 2015. Home Page. Available from: https://www.globus.org/.

[10]

Stewart, C.A., Richard Knepper, Matthew R. Link, Marlon Pierce, Eric Wernert, Nancy Wilkins-Diehr. 2014. Cyberinfrastructure, Science Gateways, Campus Bridging, and Cloud Computing. In: Encyclopedia of Information Science and Technology, Third Edition (10 Volumes) 2014; pp 6562--6572}. Available from: http://hdl.handle.net/2022/18608.

[11]

Slashdot Media. 2015. Home page. Available from: http://slashdotmedia.com/.

[12]

SourceForge. 2015. Home page. Available from: http://sourceforge.net/.

[13]

R Studio. 2015. R Studio Pricing. Available from: http://www.rstudio.com/pricing/.

[14]

National Center for Biotechnology Information. 2015. BLAST® Basic Alignment Search Tool. Available from: http://blast.ncbi.nlm.nih.gov/Blast.cgi.

[15]

Rivard, R. 2014. Kuali Tries to Compete https://www.insidehighered.com/news/2013/10/15/colleges-prepare-major-software-upgrades-kuali-tries-woo-them-corporate-vendors

[16]

Internet2. 2015. Internet2 NET+. Available from: http://www.internet2.edu/netplus/.

[17]

Raymond, E.S. 2010. The Cathedral and the Bazaar. http://www.catb.org/esr/writings/cathedral-bazaar/.

Digital Library

[18]

Stewart, C.A., Knepper, R. D., Link, M. R., Pierce, M., Wernert, E. A., Wilkins-Diehr, N. 2014. Cyberinfrastructure, Science Gateways, Campus Bridging, and Cloud Computing. In: Encyclopedia of Information Science and Technology, Third Edition. IGI Global. Hershey. PA. Available from: http://hdl.handle.net/2022/18608

[19]

Open Source Initiative. 2015. Home page. Available from: http://www.opensource.org.

[20]

Apache Foundation. 2015. Home Page. Available from: https://www.apache.org.

[21]

iRODS. 2015. Home page. Available from: https://www.irods.org/.

[22]

Felsenstein, J., Evolutionary trees from DNA sequences: a maximum likelihood approach. J Mol Evol, 1981. 17(6): p. 368--76.

[23]

Olsen, G. J., H. Matsuda, R. Hagstrom, and R. Overbeek. 1994. fastDNAml: A tool for construction of phylogenetic trees of DNA sequences using maximum likelihood. Comput. Appl. Biosci. 10: 41--48.

[24]

Stewart, C.A., D. Hart, D. K. Berry, G. J. Olsen, E. Wernert, W. Fischer. 2001. Parallel implementation and performance of fastDNAml - a program for maximum likelihood phylogenetic inference. Proceedings of SC2001, Denver, CO, November 2001. http://portal.acm.org/citation.cfm?id=582054

Digital Library

[25]

Stamatakis, A. 2014. RAxML Version 8: A tool for Phylogenetic Analysis and Post-Analysis of Large Phylogenies. Bioinformatics (open access). Available from:

[26]

Stodden, V. 2012. Reproducible research for scientific computing: Tools and strategies for changing the culture. Computing in Science & Engineering, vol.14, no. 4, pp. 13--17, July/August 2012.

Digital Library

[27]

The Open Group. Tar File Archiver. Available from: http://pubs.opengroup.org/onlinepubs/7990989799/xcu/tar.html.

[28]

Jenkins. Home Page. Available from: http://jenkins-ci.org/.

[29]

BaTLab. 2015. Home Page. Available from: https://www.batlab.org/.

[30]

SWAMP (Software Assurance Marketplace ). 2015. Home page. Available from: https://continuousassurance.org/.

[31]

Open Science Grid. 2015. Home page. Available from: http://www.opensciencegrid.org/.

[32]

Pordes, R. 2008. Challenges Facing Production Grids. In: High Performance Computing and Grids in Action. Advances in Parallel Computing, 16th ed. IOS Press: Amsterdam. p. 506--521.

[33]

Pordes, R., W. Kramer, M. Livny, P. Avery, K. Blackburn, T. Wenaus, F. Würthwein, I. Foster, R. Gardner, M. Wilde, A. Blatecky, J. McGee, R. Quick. 2007. The Open Science Grid--Its Status and Implementation Architecture. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP 07). Available from: http://tinyurl.com/pun3vbj

[34]

XSEDE. 2015. Home page. Available from: https://www.xsede.org/.

[35]

XSEDE. 2015. XSEDE Resources Overview. Available from: https://www.xsede.org/web/guest/resources.

[36]

Towns, J., T.Cockerill, M. Dahan, I. Foster, K. Gaither, A. Grimshaw, V. Hazlewood, S. Lathrop, D. Lifka, G.D. Peterson, R. Roskies, J.R. Scott, N. Wilkins-Diehr, XSEDE: Accelerating Scientific Discovery. Computing in Science & Engineering. 16:62--74. Available from:

[37]

XSEDE. 2015.What We Do. Available from: https://www.xsede.org/what-we-do.

[38]

Software Carpentry. Home page. Available from: http://software-carpentry.org/.

[39]

National Science Foundation. Characteristics of Potentially Transformative Research. 2010 31 Jan 2011}; Available from: http://tinyurl.com/q9s6tln.

Cited By

Imran AKosar T(2021)The Impact of Human Factors on Software SustainabilitySoftware Sustainability10.1007/978-3-030-69970-3_12(287-300)Online publication date: 6-Oct-2021
https://doi.org/10.1007/978-3-030-69970-3_12
Kee KLe BJitkajornwanich K(2021)If you build it, promote it, and they trust you, then they will come: Diffusion strategies for science gateways and cyberinfrastructure adoption to harness big data in the science, technology, engineering, and mathematics (STEM) communityConcurrency and Computation: Practice and Experience10.1002/cpe.619233:19Online publication date: 20-Jan-2021
https://doi.org/10.1002/cpe.6192
Stewart CApon AHancock DFurlani TSill AWernert JLifka DBerente NCheatham TSlavin SStewart C(2019)Assessment of non-financial returns on cyberinfrastructureProceedings of the Humans in the Loop: Enabling and Facilitating Research on Cloud Computing10.1145/3355738.3355749(1-10)Online publication date: 29-Jul-2019
https://dl.acm.org/doi/10.1145/3355738.3355749

Index Terms

Sustained Software for Cyberinfrastructure: Analyses of Successful Efforts with a Focus on NSF-funded Software
1. Software and its engineering

Recommendations

A Comparison of Software Quality Characteristics and Software Sustainability Characteristics
ISCSIC 2019: Proceedings of the 2019 3rd International Symposium on Computer Science and Intelligent Control

Software sustainability has generated much interest in the software engineering field in recent times, and has been widely investigated across different fields and from different standpoints. The relationship between software quality and software ...
Sustainable software engineering: Reflections on advances in research and practice
Abstract Context:
Modern societies are highly dependent on complex, large-scale, software-intensive systems that increasingly operate within an environment of continuous availability, which are challenging to maintain, and evolve in response to changes in ...
Towards sustainable software systems: A software sustainability analysis framework
Abstract Context:
In today’s rapidly evolving technological landscape designing sustainable software systems requires considering the software impacts and its long-term viability. For professionals, a significant barrier lies in the need for practical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SCREAM '15: Proceedings of the 1st Workshop on The Science of Cyberinfrastructure: Research, Experience, Applications and Models

June 2015

82 pages

ISBN:9781450335669

DOI:10.1145/2753524

General Chairs:
Shantenu Jha
Rutgers University, USA
,
Daniel S. Katz
University of Chicago & Argonne National Laboratory, USA
,
Jon Weissman
University of Minnesota, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 June 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

HPDC'15

Sponsor:

University of Arizona
SIGARCH

HPDC'15: The 24th International Symposium on High-Performance Parallel and Distributed Computing

June 16, 2015

Oregon, Portland, USA

Acceptance Rates

SCREAM '15 Paper Acceptance Rate 8 of 12 submissions, 67%;

Overall Acceptance Rate 8 of 12 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
333
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)9

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Imran AKosar T(2021)The Impact of Human Factors on Software SustainabilitySoftware Sustainability10.1007/978-3-030-69970-3_12(287-300)Online publication date: 6-Oct-2021
https://doi.org/10.1007/978-3-030-69970-3_12
Kee KLe BJitkajornwanich K(2021)If you build it, promote it, and they trust you, then they will come: Diffusion strategies for science gateways and cyberinfrastructure adoption to harness big data in the science, technology, engineering, and mathematics (STEM) communityConcurrency and Computation: Practice and Experience10.1002/cpe.619233:19Online publication date: 20-Jan-2021
https://doi.org/10.1002/cpe.6192
Stewart CApon AHancock DFurlani TSill AWernert JLifka DBerente NCheatham TSlavin SStewart C(2019)Assessment of non-financial returns on cyberinfrastructureProceedings of the Humans in the Loop: Enabling and Facilitating Research on Cloud Computing10.1145/3355738.3355749(1-10)Online publication date: 29-Jul-2019
https://dl.acm.org/doi/10.1145/3355738.3355749

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents