Abstract
Basically, this chapter tries to find answers for the following fundamental questions in experimental research.
-
(Q-1)
How can problem instances be generated?
-
(Q-2)
How can experimental results be generalized?
The chapter is structured as follows. Section 56.2 introduces real-world and artificial optimization problems. Algorithms are described in Sect. 56.3. Objective functions and statistical models are introduced in Sect. 56.4; these models take problem and algorithm features into consideration. Section 56.5 presents case studies that illustrate our methodology. The chapter closes with a summary and an outlook.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Abbreviations
- ANOVA:
-
analysis of variance
- CI:
-
computational intelligence
- EA:
-
evolutionary algorithm
- ES:
-
evolution strategy
- FX:
-
foreign exchange
- i.i.d.:
-
independent, identically distributed
- MAMP:
-
multiple algorithms, multiple problems
- MAMS:
-
multiple algorithms and multiple problem instances
- MASP:
-
multiple algorithms and one single problem
- MSG:
-
max-set of Gaussian landscape generator
- Q–Q:
-
quantile–quantile
- REML:
-
restricted maximum likelihood estimator
- SAMP:
-
one single algorithm and multiple problems
- SASP:
-
one single algorithm and one single problem
- SPOT:
-
sequential parameter optimization toolbox
- US EPA:
-
United States Environmental Protection Agency
References
P.J. Brockwell, R.A. Davis: Introduction to Time Series and Forecasting (Springer, New York 2002)
D.C. Montgomery: Design and Analysis of Experiments (Wiley, New York 2001)
M. Chiarandini, Y. Goegebeur: Mixed models for the analysis of optimization algorithms. In: Experimental Methods for the Analysis of Optimization Algorithms, ed. by T. Bartz-Beielstein, M. Chiarandini, L. Paquete, M. Preuss (Springer, Berlin Heidelberg 2010) pp. 225–264, preliminary version available as Tech. Rep. DMF-2009-07-001 at the The Danish Mathematical Society
C.C. McGeoch: Toward an experimental method for algorithm simulation, INFORMS J. Comput. 8(1), 1–15 (1996)
M. Birattari: On the Estimation of the Expected Performance of a Metaheuristic on a Class of Instances 2004), Tech. Rep. (IRIDIA, Bruxelles 2004)
M. Gallagher, B. Yuan: A general-purpose tunable landscape generator, IEEE Trans. Evol. Comput. 10(5), 590–603 (2006)
I. Castiñeiras, M.D. Cauwer, B. O'Sullivan: Weibull-based benchmarks for bin packing, Lect. Notes Comput. Sci. 7514, 207–222 (2012)
G.E.P. Box, G.M. Jenkins, G.C. Reinsel: Time Series Analysis, Forecasting and Control (Holden-Day, San Francisco 1976)
T. Bartz-Beielstein: Beyond Particular Problem Instances: How to Create Meaningful and Generalizable Results, Tech. Rep. TR 03/2012 (Cologne University of Applied Sciences, 2012)
T. Bartz-Beielstein, M. Friese, B. Naujoks, M. Zaefferer: SPOT applied to non-stochastic optimization problems – An experimental study, Genet. Evol. Comput. Conf. (GECCO 2012) (ACM, Philadelphia 2012)
M. Zaefferer: Optimization and Empirical Analysis of an Event Detection Software for Water Quality Monitoring, Master Thesis (University of Applied Sciences, Cologne 2012)
O. Flasch, T. Bartz-Beielstein, D. Bicker, W. Kantschik, C. von Strachwitz: Results of the GECCO 2011 Industrial Challenge: Optimizing Foreign Exchange Trading Strategies. CIOP Tech. Rep. 10/11, Res. Center CIOP (Cologne University of Applied Science, Cologne 2011)
E. Vladislavleva: Model–based Problem Solving through Symbolic Regression via Pareto Genetic Programming, Ph.D. Thesis (Tilburg University, Tilburg 2008)
H.-G. Beyer, H.-P. Schwefel: Evolution strategies – A comprehensive introduction, Nat. Comput. 1, 3–52 (2002)
H.-P. Schwefel: Numerical Optimization of Computer Models (Wiley, Chichester 1981)
T. Bartz-Beielstein: Experimental Research in Evolutionary Computation – The New Experimentalism, Natural Computing Series (Springer, Berlin, Heidelberg, New York 2006)
T. Bartz-Beielstein, M. Preuss: Automatic and interactive tuning of algorithms, Proc. 13th Annu. Conf. Companion Genet. Evol. Comput. (GECCO) (ACM, New York 2011) pp. 1361–1380
J. Pinheiro, D. Bates: Mixed-Effects Models in S and S-PLUS (Springer, New York 2000)
T. Bartz-Beielstein, M. Zaefferer: A Gentle Introduction to Sequential Parameter Optimization, Tech. Rep. TR 01/2012 CIplus Bd. 1/2012 (Cologne University of Applied Sciences, Cologne 2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bartz-Beielstein, T. (2015). How to Create Generalizable Results. In: Kacprzyk, J., Pedrycz, W. (eds) Springer Handbook of Computational Intelligence. Springer Handbooks. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43505-2_56
Download citation
DOI: https://doi.org/10.1007/978-3-662-43505-2_56
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43504-5
Online ISBN: 978-3-662-43505-2
eBook Packages: EngineeringEngineering (R0)