Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Systematic literature review of ensemble effort estimation

Published: 01 August 2016 Publication History

Abstract

Systematic review of 24 selected studies on Ensemble Effort Estimation.Studies analyzing based on six review questions.EEE techniques usually yield acceptable estimation accuracy than single models.There is no EEE technique performing better than others in all situations. The need to overcome the weaknesses of single estimation techniques for prediction tasks has given rise to ensemble methods in software development effort estimation (SDEE). An ensemble effort estimation (EEE) technique combines several of the single/classical models found in the SDEE literature. However, to the best of our knowledge, no systematic review has yet been performed with a focus on the use of EEE techniques in SDEE. The purpose of this review is to analyze EEE techniques from six viewpoints: single models used to construct ensembles, ensemble estimation accuracy, rules used to combine single estimates, accuracy comparison of EEE techniques with single models, accuracy comparison between EEE techniques and methodologies used to construct ensemble methods. We performed a systematic review of EEE studies published between 2000 and 2016, and we selected 24 of them to address the questions raised in this review. We found that EEE techniques may be separated into two types: homogeneous and heterogeneous, and that the machine learning single models are the most frequently employed in constructing EEE techniques. We also found that EEE techniques usually yield acceptable estimation accuracy, and in fact are more accurate than single models.

References

[1]
D. Azhar, P. Riddle, E. Mendes, N. Mittas, L. Angelis, Using ensembles for web effort estimation, in: 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, 2013, pp. 173-182.
[2]
M. Azzeh, A.B. Nassif, L.L. Minku, An empirical evaluation of ensemble adjustment methods for analogy-based effort estimation, J. Syst. Softw., 103 (2015) 36-52.
[3]
P. Braga, A. Oliveira, G. Ribeiro, S. Meira, Bagging predictors for estimation of software project effort, in: Proceedings of International Joint Conference on Neural Networks, 2007, pp. 14-19.
[4]
L. Breiman, Bagging predictors, Mach. Learn., 26 (1996) 123-140.
[5]
A. Chandra, X. Yao, Ensemble learning using multi-objective evolutionary algorithms, J. Math. Model. Algorithms, 5 (2006) 417-445.
[6]
R.N. Charette, Why software fails, IEEE Spectr, 42 (2005) 42-49.
[7]
M.O. Elish, Assessment of voting ensemble for estimating software development effort, in: IEEE Symposium on Computational Intelligence and Data Mining (CIDM), 2013, pp. 316-321.
[8]
M.O. Elish, Improved estimation of software project effort using multiple additive regression trees, Expert Syst. Appl., 36 (2009) 10774-10778.
[9]
M.O. Elish, T. Helmy, M.I. Hussain, Empirical study of homogeneous and heterogeneous ensemble models for software development effort estimation, Math. Probl. Eng, 2013 (2013).
[10]
P.Brooks Frederick, The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley, ISBN: 0-201-00650-2, Boston, 1975.
[11]
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I.H. Witten, The WEKA data mining software, ACM SIGKDD Explor, 11 (2009) 10-18.
[12]
T. Hastie, J. Friedman, R. Tibshirani, Neural networks, Springer, New York, 2009.
[13]
C.-J. Hsu, N.U. Rodas, C.-Y. Huang, K.-L. Peng, A study of improving the accuracy of software effort estimation using linearly weighted combinations, in: Proceedings of the 34th IEEE Annual Computer Software and Applications Conference Workshops, 2010, pp. 98-103.
[14]
A. Idri, F.A. Amazal, A. Abran, Analogy-based software development effort estimation: a systematic mapping and review, Inf. Softw. Technol., 58 (2015) 206-230.
[15]
A. Idri, A. Abran, DocumentA fuzzy logic based set of measures for software project similarity: validation and possible improvements, in: International Software Metrics Symposium, Proceedings, 2001, pp. 85-96.
[16]
A. Idri, T.M. Khosgoftaar, A. Abran, Investigating soft computing in case-based reasoning for software cost estimation, Int. J. Eng. Intell. Syst. Elec. Eng. Commun., 10 (2002) 147-157.
[17]
M. Jørgensen, Forecasting of software development work effort: Evidence on expert judgement and formal models, Int. J. Forecast., 23 (2007) 449-462.
[18]
M. Jorgensen, M. Shepperd, A systematic review of software development cost estimation studies, IEEE Trans. Softw. Eng., 33 (2007) 33-53.
[19]
T.M. Khoshgoftaar, P. Rebours, N. Seliya, Software quality analysis by combining multiple projects and learners, Softw. Qual. J., 17 (2009) 25-49.
[20]
B. Kitchenham, S. Charters, Guidelines for performing systematic literature reviews in software engineering, Engineering, 2 (2007) 1051.
[21]
E. Kocaguneli, Y. Kultur, A.B. Bener, Combining multiple learners induced on multiple datasets for software effort prediction, in: Proceedings of International Symposium on Software Reliability Engineering, 2009.
[22]
E. Kocaguneli, T. Menzies, J.W. Keung, On the Value of Ensemble Effort Estimation, IEEE Trans. Softw. Eng., 38 (2012) 1403-1416.
[23]
Y. Kultur, B. Turhan, A. Bener, Ensemble of neural networks with associative memory (ENNA) for estimating software development costs, Knowl.-Based Syst, 22 (2009) 395-402.
[24]
Q. Li, Q. Wang, Y. Yang, M. Li, Reducing biases in individual software effort estimations: a combining approach, in: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement - ESEM '08, 2008, pp. 223.
[25]
Y. Liu, X. Yao, Ensemble learning via negative correlation, Neural Netw., 12 (1999) 1399-1404.
[26]
S.G. MacDonell, M.J. Shepperd, Combining techniques to optimize effort predictions in software project management, J. Syst. Softw., 66 (2003) 91-98.
[27]
S. McConnell, Software Estimation - Black Art Book, Microsoft Press, ISBN: 0735637032, USA, 2006.
[28]
T. Menzies, Z. Chen, J. Hihn, K. Lum, Selecting best practices for effort estimation, IEEE Trans. Softw. Eng., 32 (2006) 883-895.
[29]
L.L. Minku, X. Yao, Ensembles and locality: Insight on improving software effort estimation, Inf. Softw. Technol., 55 (2013) 1512-1528.
[30]
L.L. Minku, X. Yao, Software effort estimation as a multiobjective learning problem, ACM Trans.Softw.Eng.Methodol, 22 (2013).
[31]
L.L. Minku, X. Yao, An analysis of multi-objective evolutionary algorithms for training ensemble models based on different performance measures in software effort estimation, in: Proceedings of the 9th International Conference on Predictive Models in Software Engineering - PROMISE '13, 2013, pp. 1-10.
[32]
N. Mittas, M. Athanasiades, L. Angelis, Improving analogy-based software cost estimation by a resampling method, Inf. Softw. Technol., 50 (2008) 221-230.
[33]
A.B. Nassif, M. Azzeh, L.F. Capretz, D. Ho, A comparison between decision trees and decision tree forest models for software development effort estimation, in: 2013 Third International Conference on Communications and Information Technology (ICCIT), 2013, pp. 220-224.
[34]
A.B. Nassif, L.F. Capretz, D. Ho, M. Azzeh, A treeboost model for software effort estimation based on use case points, in: 2012 11th International Conference on Machine Learning and Applications, 2012, pp. 314-319.
[35]
F. Nayebi, A. Abran, J.-M. Desharnais, Automated selection of a software effort estimation model based on accuracy and uncertainty, Artif. Intell. Res., 4 (2015) 45-54.
[36]
J.S. Pahariya, V. Ravi, M. Carr, Software cost estimation using computational intelligence techniques, in: Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, 2009, pp. 848-853.
[37]
S.K. Rath, B.P. Acharya, S.M. Satapathy, Early stage software effort estimation using random forest technique based on use case points, IET Softw, 10 (2016) 10-17.
[38]
S.M. Satapathy, M. Kumar, S.K. Rath, Class point approach for software effort estimation using soft computing techniques, in: 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2013, pp. 178-183.
[39]
Seni, G., Elder, J.F., 2010. Ensemble methods in data mining: improving accuracy through combining predictions.
[40]
M.J. Shepperd, G. Kadoda, Comparing software prediction techniques using simulation, IEEE Trans. Softw. Eng., 27 (2001) 1014-1022.
[41]
L. Song, L.L. Minku, X. Yao, The impact of parameter tuning on software effort estimation using learning machines, in: Proceedings of the 9th International Conference on Predictive Models in Software Engineering, 2013.
[42]
M.C.K Vinaykumar, V. Ravi, Software cost estimation using soft computing approaches, IGI-Global, 2009.
[43]
J. Wen, S. Li, Z. Lin, Y. Hu, C. Huang, Systematic literature review of machine learning based software development effort estimation models, Inf. Softw. Technol., 54 (2012) 41-59.
[44]
P.A. Whigham, C.A. Owen, S.G. Macdonell, A baseline model for software effort estimation, ACM Trans. Softw. Eng. Methodol., 24 (2015) 20.
[45]
D. Wu, J. Li, Y. Liang, Linear combination of multiple case-based reasoning with optimized weight for software effort estimation, J. Supercomput., 64 (2013) 898-918.

Cited By

View all

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Systems and Software
Journal of Systems and Software  Volume 118, Issue C
August 2016
297 pages

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 August 2016

Author Tags

  1. Ensemble effort estimation
  2. Software development effort estimation
  3. Systematic literature review

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in AgileACM Computing Surveys10.1145/366336556:11(1-37)Online publication date: 28-Jun-2024
  • (2024)Ensemble effort estimation for novice agile teamsInformation and Software Technology10.1016/j.infsof.2024.107447170:COnline publication date: 1-Jun-2024
  • (2024)A random forest model for early-stage software effort estimation for the SEERA datasetInformation and Software Technology10.1016/j.infsof.2024.107413169:COnline publication date: 1-May-2024
  • (2024)Software Effort Estimation Using Stacked Ensemble Technique and Hybrid Principal Component Regression and Multivariate Adaptive Regression SplinesWireless Personal Communications: An International Journal10.1007/s11277-024-11010-9134:4(2259-2278)Online publication date: 1-Feb-2024
  • (2024)Software effort estimation modeling and fully connected artificial neural network optimization using soft computing techniquesCluster Computing10.1007/s10586-023-03979-y27:1(737-760)Online publication date: 1-Feb-2024
  • (2024)Towards effective feature selection in estimating software effort using machine learningJournal of Software: Evolution and Process10.1002/smr.258836:5Online publication date: 25-Apr-2024
  • (2023)Software effort estimation using stacked ensembled techniques and proposed stacking ensemble using principal component regression as super learnerJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-23067645:6(9697-9713)Online publication date: 1-Jan-2023
  • (2023)Ensemble Effort EstimationJournal of Systems and Software10.1016/j.jss.2022.111542195:COnline publication date: 1-Jan-2023
  • (2023)Application of neuro-fuzzy ensembles across domainsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.106582124:COnline publication date: 1-Sep-2023
  • (2023)On the value of parameter tuning in stacking ensemble model for software regression test effort estimationThe Journal of Supercomputing10.1007/s11227-023-05334-979:15(17123-17145)Online publication date: 6-May-2023
  • Show More Cited By

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media