Article

Effort estimation: how valuable is it for a web company to use a cross-company data set, compared to using its own single-company data set?

Authors:

Sergio Di Martino,

Filomena Ferrucci,

Carmine GravinoAuthors Info & Claims

WWW '07: Proceedings of the 16th international conference on World Wide Web

Pages 963 - 972

https://doi.org/10.1145/1242572.1242702

Published: 08 May 2007 Publication History

Abstract

Previous studies comparing the prediction accuracy of effort models built using Web cross- and single-company data sets have been inconclusive, and as such replicated studies are necessary to determine under what circumstances a company can place reliance on a cross-company effort model.

This paper therefore replicates a previous study by investigating how successful a cross-company effort model is: i) to estimate effort for Web projects that belong to a single company and were not used to build the cross-company model; ii) compared to a single-company effort model. Our single-company data set had data on 15 Web projects from a single company and our cross-company data set had data on 68 Web projects from 25 different companies. The effort estimates used in our analysis were obtained by means of two effort estimation techniques, namely forward stepwise regression and case-based reasoning.

Our results were similar to those from the replicated study, showing that predictions based on the single-company model were significantly more accurate than those based on the cross-company model.

References

[1]

Briand, L. C., K. El-Emam, K. Maxwell, D. Surmann, I. Wieczorek. An assessment and comparison of common cost estimation models, in Proceedings of the 21st International Conference on Software Engineering, ICSE 99, 1999, 313--322.

Digital Library

[2]

Briand, L. C., T. Langley, I. Wieczorek. A replicated assessment of common software cost estimation techniques, in Proceedings of the 22nd International Conference on Software Engineering, ICSE 20, 2000, 377--386.

Digital Library

[3]

Christodoulou, S. P., P. A. Zafiris, T. S. Papatheodorou, WWW 2000: The Developer's view and a practitioner's approach to Web Engineering, in Proceedings of Second ICSE Workshop on Web Engineering, 4 and 5 June 2000, Limerick, Ireland, 2000, 75--92.

Digital Library

[4]

Conte, S. D., Dunsmore, H. E., Shen, V. Y. Software Engineering Metrics and Models, Benjamin-Cummins, 1986.

Digital Library

[5]

Cook, R. D. Detection of influential observations in linear regression. Technometrics, 19, 1977, 15--18.

[6]

G. Costagliola, S. Di Martino, F. Ferrucci, C. Gravino, G. Vitiello, G. Tortora, "A COSMIC-FFP Approach to Predict Web Application Development Effort", Journal of Web Engineering (Rinton Press), 5(2), 2006, 93--120.

Digital Library

[7]

G. Costagliola, S. Di Martino, F. Ferrucci, C. Gravino, G. Tortora, G. Vitiello, "Effort estimation modeling techniques: a case study for web applications", in Proceedings of International Conference on Web Engineering (ICWE'06), Palo Alto, California, USA, 2006, 9--16.

Digital Library

[8]

Jeffery, R., M. Ruhe and I. Wieczorek. A Comparative Study of Two Software Development Cost Modeling Techniques using Multi-organizational and Company-specific Data. Information and Software Technology, 42, 2000, 1009--1016.

[9]

Jeffery, R., M. Ruhe and I. Wieczorek. Using public domain metrics to estimate software development effort, in Proceedings Metrics' 01, London, 2001, 16--27.

Digital Library

[10]

Kemerer, C. F. An empirical validation of software cost estimation models. Communications ACM, 30(5), 1987.

Digital Library

[11]

B. A. Kitchenham, "A Procedure for Analyzing Unbalanced Datasets", IEEE Transactions on Software Engineering, 24(4), 1998, 278--301.

Digital Library

[12]

Kitchenham, B. A., and E. Mendes. Software Productivity Measurement Using Multiple Size Measures, IEEE Transactions on Software Engineering, 30(12), 1023--1035.

Digital Library

[13]

Kitchenham, B. A., and E. Mendes. A Comparison of Cross-company and Single-company Effort Estimation Models for Web Applications, in Proceedings EASE 2004, 2004, 47--55.

[14]

Kitchenham, B. A. and N. R. Taylor. Software Cost Models. ICL Technical Journal, May 1984, 73--102.

[15]

Kitchenham, B. A., E. Mendes, and G. Travassos, A Systematic Review of Cross - and Within-company Cost Estimation Studies, 2006, Proceedings of Empirical Assessment in Software Engineering, 89--98.

Digital Library

[16]

Lefley, M., and M. J. Shepperd, Using Genetic Programming to Improve Software Effort Estimation Based on General Data Sets, Proceedings of GECCO 2003, LNCS 2724, Springer-Verlag, 2003, 2477--2487.

Digital Library

[17]

Lokan, C. J., and E. Mendes, Cross-company and Single-company Effort Models Using the ISBSG Database: a Further Replicated Study, Proceedings of the ISESE'06, 2006, 75--84.

Digital Library

[18]

Maxwell, K. Applied Statistics for Software Managers. Software Quality Institute Series, Prentice Hall, 2002.

[19]

Maxwell, K., L. V. Wassenhove, and S. Dutta, Performance Evaluation of General and Company Specific Models in Software Development Effort Estimation, Management Science, 45(6), June, 1999, 787--803.

Digital Library

[20]

Mendes, E. and B. A. Kitchenham, Further Comparison of Cross-Company and Within Company Effort Estimation Models for Web Applications, in Proceedings Metrics'04, Chicago, Illinois September 11-17th 2004, IEEE Computer Society, 2004, 348--357.

Digital Library

[21]

Mendes, E., N. Mosley, and S. Counsell, Investigating Early Web Size Measures for Web Cost Estimation, in Proceedings of EASE' 2003 Conference, Keele, April, 2003, 1-22.

[22]

Mendes, E., Lokan, C., Harrison, R., and Triggs, C. A Replicated Comparison of Cross-company and Within-company Effort Estimation models using the ISBSG Database, in Proceedings of Metrics' 05, Como, 2005.

Digital Library

[23]

I. Myrtveit, E. Stensrud, "A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models", IEEE Transactions on Software Engineering, 25(4), 1999, 510--525.

Digital Library

[24]

Shepperd, M. J., and G. Kadoda, Using Simulation to Evaluate Prediction Techniques, in Proceedings IEEE 7th International Software Metrics Symposium, London, UK, 2001, 349--358.

Digital Library

[25]

Wieczorek, I. and M. Ruhe. How Valuable is Company-Specific Data Compared to Multi-Company Data for Software Cost Estimation?, in Proceedings Metrics' 02, Ottawa, June 2002, 237--246.

Digital Library

[26]

Wilcoxon, F. Individual Comparisons by Ranking Methods. Biometrics, 1, 1945, 80--83.

Cited By

Bosu MMacdonell S(2019)ExperienceJournal of Data and Information Quality10.1145/332874611:4(1-38)Online publication date: 19-Aug-2019
https://dl.acm.org/doi/10.1145/3328746
Xu ZZhang TZhang YTang YLiu JLuo XKeung JCui X(2019)Identifying Crashing Fault Residence Based on Cross Project Model2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2019.00027(183-194)Online publication date: Oct-2019
https://doi.org/10.1109/ISSRE.2019.00027
Ferrucci FGravino C(2019)Can Expert Opinion Improve Effort Predictions When Exploiting Cross-Company Datasets? - A Case Study in a Small/Medium CompanyProduct-Focused Software Process Improvement10.1007/978-3-030-35333-9_20(280-295)Online publication date: 18-Nov-2019
https://doi.org/10.1007/978-3-030-35333-9_20
Show More Cited By

Index Terms

Effort estimation: how valuable is it for a web company to use a cross-company data set, compared to using its own single-company data set?
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Implementation management
        Pricing and resource allocation
      2. Project and people management
2. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Reusability
        Software product lines
    2. Software verification and validation
      1. Process validation

Recommendations

Web effort estimation: the value of cross-company data set compared to single-company data set
PROMISE '12: Proceedings of the 8th International Conference on Predictive Models in Software Engineering

This study investigates to what extent Web effort estimation models built using cross-company data sets can provide suitable effort estimates for Web projects belonging to another company, when compared to Web effort estimates obtained using that company'...
A Comparison of Cross-Versus Single-Company Effort Prediction Models for Web Projects
SEAA '14: Proceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications

Background: In order to address the challenges in companies having no or limited effort datasets of their own, cross-company models have been a focus of interest for previous studies. Further, a particular domain of investigation has been Web projects. ...
A replicated study comparing web effort estimation techniques
WISE'07: Proceedings of the 8th international conference on Web information systems engineering

The objective of this paper is to replicate two previous studies that compared at least three techniques for Web effort estimation in order to identify the one that provides best prediction accuracy. We employed the three effort estimation techniques ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '07: Proceedings of the 16th international conference on World Wide Web

May 2007

1382 pages

ISBN:9781595936547

DOI:10.1145/1242572

General Chairs:
Carey Williamson
University of Calgary, Canada
,
Mary Ellen Zurko
IBM, USA
,
Program Chairs:
Peter Patel-Schneider
Bell Labs Research, USA
,
Prashant Shenoy
University of Massachusetts at Amherst, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

WWW'07

Sponsor:

ACM

WWW'07: 16th International World Wide Web Conference

May 8 - 12, 2007

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
505
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 20 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bosu MMacdonell S(2019)ExperienceJournal of Data and Information Quality10.1145/332874611:4(1-38)Online publication date: 19-Aug-2019
https://dl.acm.org/doi/10.1145/3328746
Xu ZZhang TZhang YTang YLiu JLuo XKeung JCui X(2019)Identifying Crashing Fault Residence Based on Cross Project Model2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2019.00027(183-194)Online publication date: Oct-2019
https://doi.org/10.1109/ISSRE.2019.00027
Ferrucci FGravino C(2019)Can Expert Opinion Improve Effort Predictions When Exploiting Cross-Company Datasets? - A Case Study in a Small/Medium CompanyProduct-Focused Software Process Improvement10.1007/978-3-030-35333-9_20(280-295)Online publication date: 18-Nov-2019
https://doi.org/10.1007/978-3-030-35333-9_20
Silva SC�rtes M(2017)Use of Non-functional Requirements in Software Effort Estimation: Systematic Review and Experimental Results2017 5th International Conference in Software Engineering Research and Innovation (CONISOFT)10.1109/CONISOFT.2017.00008(1-9)Online publication date: Oct-2017
https://doi.org/10.1109/CONISOFT.2017.00008
Lee TWei KAbd Ghani A(2016)Systematic literature review on effort estimation for Open Sources (OSS) web application development2016 Future Technologies Conference (FTC)10.1109/FTC.2016.7821748(1158-1167)Online publication date: Dec-2016
https://doi.org/10.1109/FTC.2016.7821748
Minku LSarro FMendes EFerrucci F(2015)How to Make Best Use of Cross-Company Data for Web Effort Estimation?2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1109/ESEM.2015.7321199(1-10)Online publication date: Oct-2015
https://doi.org/10.1109/ESEM.2015.7321199
Di Martino SGravino C(2015)Estimating Web Application Development Effort Using Cosmic-Ffp MethodInternational Journal of Computers and Applications10.1080/1206212X.2009.1144193631:3(153-158)Online publication date: 11-Jul-2015
https://doi.org/10.1080/1206212X.2009.11441936
Corazza ADi Martino SFerrucci FGravino CSarro F(2015)From Function Points to COSMIC - A Transfer Learning Approach for Effort EstimationProceedings of the 16th International Conference on Product-Focused Software Process Improvement - Volume 945910.1007/978-3-319-26844-6_19(251-267)Online publication date: 2-Dec-2015
https://dl.acm.org/doi/10.1007/978-3-319-26844-6_19
Matos OConte TMendes EShepperd MHall TMyrtveit I(2014)Is there a place for qualitative studies when identifying effort predictors?Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering10.1145/2601248.2601281(1-10)Online publication date: 13-May-2014
https://dl.acm.org/doi/10.1145/2601248.2601281
Turhan BMendes E(2014)A Comparison of Cross-Versus Single-Company Effort Prediction Models for Web ProjectsProceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications10.1109/SEAA.2014.41(285-292)Online publication date: 27-Aug-2014
https://dl.acm.org/doi/10.1109/SEAA.2014.41
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents