article

Free access

Why SpecInt95 should not be used to benchmark embedded systems tools

Author:

Jakob EngblomAuthors Info & Claims

ACM SIGPLAN Notices, Volume 34, Issue 7

Pages 96 - 103

https://doi.org/10.1145/315253.314470

Published: 01 May 1999 Publication History

Abstract

The SpecInt95 benchmark suite is often used to evaluate the performance of programming tools, including those used for embedded systems programming. Embedded applications, however, are often targeting 8- or 16-bit processors with limited functionality, whereas SpecInt95 has no particular target architecture and a bias towards 32-bit systems. Hence, there are reasons to question the use of SpecInt95 for the evaluation of tools for embedded systems.We present a comparative study of the static properties of a set of embedded application and the SpecInt95 benchmarks. The properties studied include: variable types, function argument lists, type of operations, and the use of local and global memory.The study provides-clear evidence that embedded applications and the SpecInt95 program suite differs significantly in several important areas. Hence, we conclude that using SpecInt95 to evaluate or compare tools for embedded systems is likely to be irrelevant or misleading, and that there is a clear need for a benchmark suite tailored for the embedded applications area.

References

[1]

Homepage for ASTEC: http: {/~w-w. does. uu. se/astec.

[2]

ARM7 Data Sheet, ARM DDI 0020C, Advanced RISC Machines Ltd, December 1994.

[3]

Homepage for the EDN Embedded Microprocessor Benchmarking Consortium: bttp://~, combe, org.

[4]

J. Engblom: Static Properties of Commercial Real-Time and Embedded Systems- Results from the MARE Project, ASTEC Technical Report 98/05, Uppsala University, October 1998. Available on the web: http://www, docs. uu. se/astec/Report s/tr_$ndex, shtml.

[5]

J. Engblom, "Static Properties of Commercial Embedded Real-Time Programs, and Their Implications for Worst-Case Execution Time Analysis", To be published in Proceedings of the Fifth IEEE Real-Time Technology and Applications Symposium (RTAS '99), IEEE Computer Society Press, June 1999.

Digital Library

[6]

Homepage for IAR Systems: http://w~'w.iax.com.

[7]

Information about the IAR Systems 68HCll compiler: wvw. iar. com/download/ew681 I. pdf.

[8]

MCS 51 Microcontroller Family User's Manual, Order No. 272383-002, Intel Corporation, February 1994.

[9]

N. Jones: "Efficient C code for eight-bit MCUs", Embedded Systems Programming Europe, Miller Freeman Ltd, London, February 1999, pp. 18-30.

[10]

D. Lafreniere, "An efficient dynamic storage allocator", Embedded Systems Programming Europe, Miller Freeman Ltd, London, November 1998, pp. 34--42.

[11]

J. Lampe: "Statistics about Modules of the Oberon System", Software-Concepts and Tools, (1997) 18, Springer Verlag, pp. 27-34.

[12]

D. C. Lee, P. J. Crowley, J-L. Baer, T. E. Anderson, and B. N. Bershad: "Execution Characteristics of Desktop Applications on Windows NT". In Proceedings of the 25th Annual International Symposium on Computer Architecture, published as A CM SIGARCH Computer Architecture News, Vol. 26, No. 3 (June 1998), Pages 27-38.

Digital Library

[13]

PowerPC Microprocessor Family: The Programming Environment For 32-Bit Microprocessors, Rev 1. Order No. MPCFPE32B/AD, Motorola Inc, January 19997.

[14]

V. Sepp~inen, A-M. K~ihkSnen, M. Oivo, H. Perunka, P. Isomursu, and P. Pulli: Strategic Needs and Future Trends of Embedded Software, TEKES Technology Review 48/96, 1996. To order, check http://w,r,q, tekes .fi.

[15]

Homapage for SPEC (The Standard Performance Evaluation Corporation): http://www, spec. org.

Cited By

Masud ALisper B(2021)Semantic Correctness of Dependence-based Slicing for Interprocedural, Possibly Nonterminating ProgramsACM Transactions on Programming Languages and Systems10.1145/343448942:4(1-56)Online publication date: 4-Jan-2021
https://doi.org/10.1145/3434489
Zhao CMei KZheng N(2014)Design of write merging and read prefetching buffer in DRAM controller for embedded processorMicroprocessors and Microsystems10.1016/j.micpro.2014.03.01038:5(451-457)Online publication date: Jul-2014
https://doi.org/10.1016/j.micpro.2014.03.010
Masud ALisper B(2021)Semantic Correctness of Dependence-based Slicing for Interprocedural, Possibly Nonterminating ProgramsACM Transactions on Programming Languages and Systems10.1145/343448942:4(1-56)Online publication date: 4-Jan-2021
https://dl.acm.org/doi/10.1145/3434489
Show More Cited By

Index Terms

Why SpecInt95 should not be used to benchmark embedded systems tools
1. Software and its engineering
  1. Software creation and management
    1. Designing software
      1. Software implementation planning
        Software design techniques
    2. Software development process management
  2. Software organization and properties
    1. Contextual software domains
      1. Operating systems

Recommendations

Why SpecInt95 should not be used to benchmark embedded systems tools
LCTES '99: Proceedings of the ACM SIGPLAN 1999 workshop on Languages, compilers, and tools for embedded systems

The SpecInt95 benchmark suite is often used to evaluate the performance of programming tools, including those used for embedded systems programming. Embedded applications, however, are often targeting 8- or 16-bit processors with limited functionality, ...
Introduction to Embedded Systems
ParMiBench - An Open-Source Benchmark for Embedded Multiprocessor Systems

Multicore processors are the main computing platform in laptops, desktop, and servers today, and are making their way into the embedded systems market also. Using benchmarks is a common approach to evaluate the performance of a system. However, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 34, Issue 7

LCTES '99. Languages, compilers, and tools for embedded systems: proceedings of the ACM SIGPLAN 1999 workshop

July 1999

104 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/315253

Editor:
A. Michael Berman
Rowan Univ., Glassboro, NJ

Issue’s Table of Contents

LCTES '99: Proceedings of the ACM SIGPLAN 1999 workshop on Languages, compilers, and tools for embedded systems
May 1999
120 pages
ISBN:1581131364
DOI:10.1145/314403
Chairmen:
Y. Annie Liu,
Reinhard Wilhelm

Copyright © 1999 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 1999

Published in SIGPLAN Volume 34, Issue 7

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
572
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)20

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Masud ALisper B(2021)Semantic Correctness of Dependence-based Slicing for Interprocedural, Possibly Nonterminating ProgramsACM Transactions on Programming Languages and Systems10.1145/343448942:4(1-56)Online publication date: 4-Jan-2021
https://doi.org/10.1145/3434489
Zhao CMei KZheng N(2014)Design of write merging and read prefetching buffer in DRAM controller for embedded processorMicroprocessors and Microsystems10.1016/j.micpro.2014.03.01038:5(451-457)Online publication date: Jul-2014
https://doi.org/10.1016/j.micpro.2014.03.010
Masud ALisper B(2021)Semantic Correctness of Dependence-based Slicing for Interprocedural, Possibly Nonterminating ProgramsACM Transactions on Programming Languages and Systems10.1145/343448942:4(1-56)Online publication date: 4-Jan-2021
https://dl.acm.org/doi/10.1145/3434489
Leibson S(2016)Using Performance Metrics to Select Microprocessor Cores for IC DesignsElectronic Design Automation for IC System Design, Verification, and Testing10.1201/b19569-13(199-228)Online publication date: 14-Apr-2016
https://doi.org/10.1201/b19569-13
Zhao CMei KZheng N(2014)Design of write merging and read prefetching buffer in DRAM controller for embedded processorMicroprocessors & Microsystems10.1016/j.micpro.2014.03.01038:5(451-457)Online publication date: 1-Jul-2014
https://dl.acm.org/doi/10.1016/j.micpro.2014.03.010
Sundmark DErmedahl AStarner J(2009)Pinpointing Interrupts in Embedded Real-Time Systems Using Hashed Execution ContextsIEEE Transactions on Industrial Informatics10.1109/TII.2009.20249355:3(257-266)Online publication date: Aug-2009
https://doi.org/10.1109/TII.2009.2024935
Gustafsson JErmedahl ALisper B(2005)Towards a Flow Analysis for Embedded System C ProgramsProceedings of the 10th IEEE International Workshop on Object-Oriented Real-Time Dependable Systems10.1109/WORDS.2005.53(287-300)Online publication date: 2-Feb-2005
https://dl.acm.org/doi/10.1109/WORDS.2005.53
Tal Oved Weiss S(2003)Embedded instruction memory in automotive engine controllersIEEE Transactions on Vehicular Technology10.1109/TVT.2002.80712752:1(173-183)Online publication date: Jan-2003
https://doi.org/10.1109/TVT.2002.807127
Oh HKim HJung HLee JKim BJung JMin BLim JLee HKwon K(2000)AE32000: an embedded microprocessor coreProceedings of Second IEEE Asia Pacific Conference on ASICs. AP-ASIC 2000 (Cat. No.00EX434)10.1109/APASIC.2000.896957(255-258)Online publication date: 2000
https://doi.org/10.1109/APASIC.2000.896957
Engblom J(1999)Static properties of commercial embedded real-time programs, and their implication for worst-case execution time analysisProceedings of the Fifth IEEE Real-Time Technology and Applications Symposium10.1109/RTTAS.1999.777660(46-55)Online publication date: 1999
https://doi.org/10.1109/RTTAS.1999.777660

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Issue’s Table of Contents