Abstract
The first international worst-case execution time (WCET) Tool Challenge in 2006 used benchmark programs to evaluate academic and commercial WCET tools. It aimed to study the state-of-the-art in WCET analysis. The WCET Tool Challenge comprised two parallel evaluation approaches: an internal evaluation by the respective tool developers and an external test by a neutral person of an independent institute. The latter was conducted by the author of this paper. Focusing on the external test, we describe the rules, benchmarks, participants and discuss the obtained results.
Similar content being viewed by others
References
Altium Tasking Compiler. http://www.altium.com/TASKING/
ARM9. http://www.arm.com/products/CPUs/families/ARM9Family.html
ASCET. http://www.etas.com/de/products/ascet_software_products.php
Bound-T. http://www.tidorum.fi/bound-t/
C-Lab. http://www.c-lab.de/
Cosmic. http://www.cosmicsoftware.com/
Freescale MPC565. http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC565
Gaisler BCC Compiler. http://gaisler.com/doc/bcc.pdf
GNU C Compiler. http://gcc.gnu.org/
Graphviz. http://www.graphviz.org/
HCS12. http://www.freescale.com/webapp/sps/site/overview.jsp?nodeId=02Wcbf8WD69BXm
IAR. http://www.iar.com/
Infineon. http://www.infineon.com/cms/en/product/channel.html?channel=ff80808112ab681d0112ab6b2f42075b
Mälardalen Benchmarks http://www.mrtc.mdh.se/projects/wcet/sweet.html
Matlab/Simulink. http://www.mathworks.com
MIPS. http://www.mips.com/
PapaBench. http://www.irit.fr/recherches/ARCHI/MARCH/rubri-que.php3?id_rubrique=97/
Paparazzi Project. http://paparazzi.enac.fr/wiki/index.php/Main_Page
SCADE. http://www.esterel-technologies.com/products/scade-suite/
SimpleScalar. http://www.simplescalar.com/
SPARCV7/V8. http://www.sparc.org/specificationsDownload.html
SPEC95. http://www.spec.org/cpu95/
TMS470. http://focus.ti.com/mcu/docs/mcuprodoverview.tsp?sectionId=95&tabId=203familyId=454
WCET Discussion Group. http://tech.groups.yahoo.com/group/wcet/
WindRiver Compiler. http://www.windriver.com/products/development_suite/wind_river_compiler/.
WCET Tool Challenge 2006. Internet, 2006. http://www.idt.mdh.se/personal/jgn/challenge/.
Berkelaar, M.: lp solve: A mixed integer linear program solver. Tech. rep., Eindhoven University of Technology (1997)
Engblom, J.: Processor Pipelines and Static Worst-Case Execution Time Analysis. PhD Thesis, Uppsala University
Ferdinand, C.: Cache Behavior Prediction for Real-Time Systems. PhD Thesis, Universität des Saarlandes (1997) http://rw4.cs.uni-sb.de/~ferdi/publications.html
Gustafsson, J.: The WCET Tool Challenge 2006. In: Steffen Tiziana Margaris, B., Philippeu, A. (eds.) Second International Symposium on Leveraging Applications of Formal Methods (ISOLA’06), pp. 233–240, November 2007
Harel D.: Statecharts: A visual formalism for complex systems. Sci. Comput. Program. 8(3), 231–274 (1987)
Seoul National University Real-Time Research Group. SNU-RT. http://realtime.snu.ac.kr/realtime/
University of Maryland. The Omega project. http://www.cs.umd.edu/projects/omega/
Tan, L.: The Worst-Case Execution Time Tool Challenge 2006: The External Test. In: Steffen Tiziana Margaris, B., Philippeu, A., (eds.) Second International Symposium on Leveraging Applications of Formal Methods (ISOLA’06), pp. 241–248, November 2007
Wenzel, I., Kirner, R., Rieder, B., Puschner, P.P.: Measurement-Based Worst-Case Execution Time Analysis. In: SEUS, pp. 7–10. IEEE Computer Society, New York (2005)
Wenzel, I., Rieder, B., Kirner, R., Puschner, P.: Automatic timing model generation by cfg partitioning and model checking. In: DATE ’05: Proceedings of the conference on Design, Automation and Test in Europe, pp. 606–611, Washington, DC, USA. IEEE Computer Society, New York (2005)
Wilhelm R., Engblom J., Ermedahl A., Holsti N., Thesing S., Whalley D., Bernat G., Ferdinand C., Heckmann R., Mitra T., Mueller F., Puaut I., Puschner P., Staschulat J., Stenström P.: The worst-case execution-time problem—overview of methods and survey of tools. Trans. Embedded Comput. Syst. 7(3), 1–53 (2008)
Wilhelm, R., Engblom, J., Thesing, S., Whalley, D.B.: Industrial Requirements for WCET Tools—Answers to the ARTIST Questionnaire. In: Gustafsson, J. (ed.) WCET, volume MDH-MRTC-116/2003-1-SE, pp. 39–43. Department of Computer Science and Engineering, Mälardalen University, Västerås, Sweden (2003)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the ARTIST2 European Network of Excellence.
Rights and permissions
About this article
Cite this article
Tan, L. The worst-case execution time tool challenge 2006. Int J Softw Tools Technol Transfer 11, 133–152 (2009). https://doi.org/10.1007/s10009-008-0095-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10009-008-0095-9