Article

Validation methods for calibrating software effort models

Authors:

Sherry StukesAuthors Info & Claims

ICSE '05: Proceedings of the 27th international conference on Software engineering

Pages 587 - 595

https://doi.org/10.1145/1062455.1062559

Published: 15 May 2005 Publication History

Abstract

COCONUT calibrates effort estimation models using an ex-haustive search over the space of calibration parameters in a COCOMO I model. This technique is much simpler than other effort estimation method yet yields PRED levels com-parable to those other methods. Also, it does so with less project data and fewer attributes (no scale factors). How-ever, a comparison between COCONUT and other methods is complicated by differences in the experimental methods used for effort estimation. A review of those experimental methods concludes that software effort estimation models should be calibrated to local data using incremental hold-out (not jack knife) studies, combined with randomization and hypothesis testing, repeated a statistically significant number of times.

References

[1]

B. Boehm. Software Engineering Economics. Prentice Hall, 1981.

Digital Library

[2]

Barry Boehm, Ellis Horowitz, Ray Madachy, Donald Reifer, Bradford K. Clark, Bert Steece, A. Winsor Brown, Sunita Chulani, and Chris Abts. Software Cost Estimation with Cocomo II. Prentice Hall, 2000.

Digital Library

[3]

G. Boetticher. When will it be done? the 300 billion dollar question, machine learner answers. IEEE Intelligent Systems, June 2003.

Digital Library

[4]

S. Chulani, B. Boehm, and B. Steece. Bayesian analysis of empirical software engineering cost models. IEEE Transaction on Software Engineerining, 25(4), July/August 1999.

Digital Library

[5]

S. Chulani, B. Boehm, and B. Steece. From multiple regression to bayesian analysis for calibrating COCOMO II. Journal of Parametrics, 15(2):175--188, 1999.

[6]

B. Clark. The Effects of Process Maturity on Software Development Effort. PhD thesis, University of Southern California, 1997. Available from http://sunset.usc.edu/~bkclark/Research/PMAT990406.pdf.

Digital Library

[7]

P.R. Cohen. Empirical Methods for Artificial Intelligence. MIT Press, 1995.

Digital Library

[8]

D. Ferens and D. Christensen. Calibrating software cost models to Department of Defense Database: A review of ten studies. Journal of Parametrics, 18(1):55--74, November 1998.

[9]

H. Habib-agahi, S. Malhotra, and J. Quirk. Estimating software productivity and cost for NASA projects. Journal of Parametrics, pages 59--71, November 1998.

[10]

R.C. Holte. Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11:63, 1993.

Digital Library

[11]

C.F. Kemerer. An empirical validation of software cost estimation models. Communications of the ACM, 30(5):416--429, May 1987.

Digital Library

[12]

K. Lum, J. Powell, and J. Hihn. Validation of spacecraft cost estimation models for flight and ground systems. In ISPA Conference Proceedings, Software Modeling Track, May 2002.

[13]

Carolyn Mair, Gada Kadoda, Martin Lefley, Keith Phalp, Chris Sch ofield1, Martin Shepperd, and Steve Webster. An investigation of machine learning based prediction systems. The Journal of Systems and Software, 53(1):23--29, 2000.

Digital Library

[14]

M. Shepperd and C. Schofield. Estimating software project effort using analogies. IEEE Transactions on Software Engineering, 23(12), November 1997. Available from http://www.utdallas.edu/~rbanker/SE_XII.pdf.

Digital Library

[15]

K. Srinivasan and D. Fisher. Machine learning approaches to estimating software development effort. IEEE Trans. Soft. Eng., pages 126--137, February 1995.

Digital Library

[16]

The Standish Group Report: Chaos, 1995. Available from http://www4.in.tum.de/lehre/vorlesungen/vse/WS2004/1995_Standish_Chaos.pdf.

[17]

S. Stukes and H. Apgar. Applications oriented software data collection: Software model calibration report,TR-9007/549-1, management consulting and research, 15, March 1991.

[18]

S. Stukes and D. Ferens. Software cost model calibration. Journal of Parametrics, 18(1):77--98, 1998.

[19]

I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, 1999.

Digital Library

[20]

G. Wittig and G. Finnie. Estimating software development effort with connectionist models. Information and Software Technology, 39(7):469--476, 1997.

Cited By

Sarro FPetrozziello A(2018)Linear Programming as a Baseline for Software Effort EstimationACM Transactions on Software Engineering and Methodology10.1145/323494027:3(1-28)Online publication date: 17-Sep-2018
https://dl.acm.org/doi/10.1145/3234940
Hira ABoehm BReddy YVarma VCleland JBellur USengupta SSharma NLoganathan RSharma RSarkar S(2018)COSMIC Function Points Evaluation for Software MaintenanceProceedings of the 11th Innovations in Software Engineering Conference10.1145/3172871.3172874(1-11)Online publication date: 9-Feb-2018
https://dl.acm.org/doi/10.1145/3172871.3172874
Srivastava ASingh SAbbas S(2018)Performance Measure of the Proposed Cost Estimation Model: Advance Use Case Point MethodSoft Computing: Theories and Applications10.1007/978-981-13-0589-4_21(223-233)Online publication date: 31-Aug-2018
https://doi.org/10.1007/978-981-13-0589-4_21
Show More Cited By

Index Terms

Validation methods for calibrating software effort models
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
      2. Software management

Recommendations

Issues in calibrating effort estimation models

This paper discusses some important issues when calibrating, or developing effort estimation models for software projects. The author has developed and calibrated several effort estimation models within Nokia Corporation. The paper takes use of the ...
A novel fuzzy based approach for effort estimation in software development

Accurate and credible software effort estimation is always a challenge for academic research and software industry. In the beginning, estimation was carried out using only human expertise or algorithmic models, but more recently, interest has turned to ...
An empirical comparison of validation methods for software prediction models
Abstract
Model validation methods (e.g., k‐fold cross‐validation) use historical data to predict how well an estimation technique (e.g., random forest) performs on the current (or future) data. Studies in the contexts of software development effort ...

Investigating cross‐validation (CV) and bootstrap validation methods in software development effort estimation (SDEE) and software fault prediction (SFP). Ten‐fold CV with SDEE and optimistic boot with SFP data provide better predictions in a greater ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '05: Proceedings of the 27th international conference on Software engineering

May 2005

754 pages

ISBN:1581139632

DOI:10.1145/1062455

General Chair:
Gruia-Catalin Roman
Washington University in St. Louis
,
Program Chairs:
William Griswold
University of California, San Diego
,
Bashar Nuseibeh
The Open University, UK

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE05

Sponsor:

ICSE05: 27th International Conference on Software Engineering

May 15 - 21, 2005

MO, St. Louis, USA

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
1,082
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sarro FPetrozziello A(2018)Linear Programming as a Baseline for Software Effort EstimationACM Transactions on Software Engineering and Methodology10.1145/323494027:3(1-28)Online publication date: 17-Sep-2018
Hira ABoehm BReddy YVarma VCleland JBellur USengupta SSharma NLoganathan RSharma RSarkar S(2018)COSMIC Function Points Evaluation for Software MaintenanceProceedings of the 11th Innovations in Software Engineering Conference10.1145/3172871.3172874(1-11)Online publication date: 9-Feb-2018
Srivastava ASingh SAbbas S(2018)Performance Measure of the Proposed Cost Estimation Model: Advance Use Case Point MethodSoft Computing: Theories and Applications10.1007/978-981-13-0589-4_21(223-233)Online publication date: 31-Aug-2018
Gautam SSingh V(2018)The state-of-the-art in software development effort estimationJournal of Software: Evolution and Process10.1002/smr.1983(e1983)Online publication date: 29-Aug-2018
Menzies TYang YMathew GBoehm BHihn J(2017)Negative results for software effort estimationEmpirical Software Engineering10.1007/s10664-016-9472-222:5(2658-2683)Online publication date: 1-Oct-2017
PHANNACHITTA PMONDEN AKEUNG JMATSUMOTO K(2016)LSA-X: Exploiting Productivity Factors in Linear Size Adaptation for Analogy-Based Software Effort EstimationIEICE Transactions on Information and Systems10.1587/transinf.2015EDP7237E99.D:1(151-162)Online publication date: 2016
Qi FJing XZhu XWu FCheng LLo DApel SKhurshid S(2016)Privacy preserving via interval covering based subclass division and manifold learning based bi-directional obfuscation for effort estimationProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970302(75-86)Online publication date: 25-Aug-2016
Hira ASharma SBoehm BPerry DRaffo D(2016)Calibrating COCOMO® II for projects with high personnel turnoverProceedings of the International Conference on Software and Systems Process10.1145/2904354.2904367(51-55)Online publication date: 14-May-2016
Menzies TKocaguneli ETurhan BMinku LPeters F(2014)Sharing Data and Models in Software EngineeringundefinedOnline publication date: 22-Dec-2014
Ponnalagu KNarendra NLeavens G(2012)Automated trendline generation for accurate software effort estimationProceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity10.1145/2384716.2384774(203-212)Online publication date: 19-Oct-2012
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents