research-article

Open access

Bifröst: Visualizing and Checking Behavior of Embedded Systems across Hardware and Software

Authors:

Majeed Kazemitabaar,

Mitchell Karchemsky,

Björn HartmannAuthors Info & Claims

UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology

Pages 299 - 310

https://doi.org/10.1145/3126594.3126658

Published: 20 October 2017 Publication History

Abstract

The Maker movement has encouraged more people to start working with electronics and embedded processors. A key challenge in developing and debugging custom embedded systems is understanding their behavior, particularly at the boundary between hardware and software. Existing tools such as step debuggers and logic analyzers only focus on software or hardware, respectively. This paper presents a new development environment designed to illuminate the boundary between embedded code and circuits. Bifröst automatically instruments and captures the progress of the user's code, variable values, and the electrical and bus activity occurring at the interface between the processor and the circuit it operates in. This data is displayed in a linked visualization that allows navigation through time and program execution, enabling comparisons between variables in code and signals in circuits. Automatic checks can detect low-level hardware configuration and protocol issues, while user-authored checks can test particular application semantics. In an exploratory study with ten participants, we investigated how Bifröst influences debugging workflows.

Supplementary Material

suppl.mov (uistf4553-file3.mp4)

Supplemental video

Download
84.69 MB

References

[1]

Marzieh Ahmadzadeh, Dave Elliman, and Colin Higgins. 2005. An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin 37, 3 (2005), 84--88.

Digital Library

[2]

Arduino. 2017. Arduino Zero. https://www.arduino.cc/en/Main/ArduinoBoardZero. (2017).

[3]

Tracey Booth, Simone Stumpf, Jon Bird, and Sara Jones. 2016. Crossed Wires: Investigating the Problems of End-User Developers in a Physical Computing Task. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 3485--3497.

Digital Library

[4]

Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3 data-driven documents. IEEE transactions on visualization and computer graphics 17, 12 (2011), 2301--2309.

Digital Library

[5]

Cloud9. 2017. Ace - The High Performance Code Editor for the Web. https://ace.c9.io/. (2017).

[6]

Digilent. 2016. Digilent Electronics Explorer Board. (2016). https://reference.digilentinc.com/electronics_explorer:electronics_explorer Online; accessed 30-March-2016.

[7]

Daniel Drew, Julie L. Newcomb, William McGrath, filip Maksimovic, David Mellis, and Björn Hartmann. 2016. The Toastboard: Ubiquitous Instrumentation and Automated Checking of Breadboarded Circuits. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 677--686.

Digital Library

[8]

Mireille Ducassé. 1993. A pragmatic survey of automated debugging. Automated and Algorithmic Debugging (1993), 1--15.

Digital Library

[9]

L. Gugerty and G. Olson. 1986. Debugging by Skilled and Novice Programmers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '86). ACM, NY, NY, USA, 171--174.

Digital Library

[10]

Björn Hartmann, Scott R Klemmer, and Michael Bernstein. 2005. d. tools: Integrated prototyping for physical interaction design. IEEE Pervasive Computing 4 (2005).

[11]

Joshua Hibschman and Haoqi Zhang. 2015. Unravel: Rapid Web Application Reverse Engineering via Interaction Recording, Source Tracing, and Library Detection. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, NY, NY, USA, 270--279.

Digital Library

[12]

Joshua Hibschman and Haoqi Zhang. 2016. Telescope: fine-Tuned Discovery of Interactive Web UI Feature Implementation. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 233--245.

Digital Library

[13]

Jane Hoffswell, Arvind Satyanarayan, and Jeffrey Heer. 2016. Visual debugging techniques for reactive data visualization. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 271--280.

Digital Library

[14]

Jun Kato, Sean McDirmid, and Xiang Cao. 2012. DejaVu: Integrated Support for Developing Interactive Camera-based Programs. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, NY, NY, USA, 189--196.

Digital Library

[15]

Cory Kissinger, Margaret Burnett, Simone Stumpf, Neeraja Subrahmaniyan, Laura Beckwith, Sherry Yang, and Mary Beth Rosson. 2006. Supporting End-user Debugging: What Do Users Want to Know?. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '06). ACM, NY, NY, USA, 135--142.

Digital Library

[16]

Andrew J Ko, Robin Abraham, Laura Beckwith, Alan Blackwell, Margaret Burnett, Martin Erwig, Chris Scaffidi, Joseph Lawrance, Henry Lieberman, Brad Myers, and others. 2011. The state of the art in end-user software engineering. ACM Computing Surveys (CSUR) 43, 3 (2011), 21.

Digital Library

[17]

Andrew J. Ko and Brad A. Myers. 2008. Debugging Reinvented: Asking and Answering Why and Why Not Questions About Program Behavior. In Proceedings of the 30th International Conference on Software Engineering (ICSE '08). ACM, NY, NY, USA, 301--310.

Digital Library

[18]

Andrew J Ko, Brad A Myers, and Htet Htet Aung. 2004. Six learning barriers in end-user programming systems. In Visual Languages and Human Centric Computing, 2004 IEEE Symposium on. IEEE, 199--206.

Digital Library

[19]

Thomas D LaToza and Brad A Myers. 2010. Developers ask reachability questions. In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering-Volume 1. ACM, 185--194.

Digital Library

[20]

Bil Lewis. 2003. Debugging backwards in time. arXiv preprint cs/0310016 (2003).

[21]

Tom Lieber, Joel R. Brandt, and Rob C. Miller. 2014. Addressing Misconceptions About Code with Always-on Programming Visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2481--2490.

Digital Library

[22]

D Mellis, Massimo Banzi, David Cuartielles, and Tom Igoe. 2007. Arduino: An open electronic prototyping platform. In Proc. CHI, Vol. 2007.

[23]

Bernhard Plattner and Juerg Nievergelt. 1981. Special feature: Monitoring program execution: A survey. Computer 14, 11 (1981), 76--93.

Digital Library

[24]

Guillaume Pothier, Éric Tanter, and José Piquer. 2007. Scalable omniscient debugging. ACM SIGPLAN Notices 42, 10 (2007), 535--552.

Digital Library

[25]

Markus Stumptner and Franz Wotawa. 1998. A survey of intelligent debugging. AI Communications 11, 1 (1998), 35--51.

Digital Library

[26]

Daniel Tetteroo, Iris Soute, and Panos Markopoulos. 2013. Five key challenges in end-user development for tangible and embodied interaction. In Proceedings of the 15th ACM on International conference on multimodal interaction. ACM, 247--254.

Digital Library

[27]

Twitter. 2017. Bootstrap. http://getbootstrap.com. (2017).

[28]

David Ungar, Henry Lieberman, and Christopher Fry. 1997. Debugging and the Experience of Immediacy. Commun. ACM 40, 4 (April 1997), 38--43.

Digital Library

[29]

Bret Victor. 2012. Learnable programming. (2012). http://worrydream.com/LearnableProgramming/

[30]

Bret Victor. 2014. Seeing Spaces. (2014). http://worrydream.com/SeeingSpaces/

[31]

W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707--740.

Digital Library

[32]

Andreas Zeller. 2009. Why programs fail: a guide to systematic debugging. Elsevier.

Digital Library

Cited By

Lima ATrevisan D(2024)Challenges and Opportunities in Using Electronic Prototyping as a Maker ToolProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702092(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702092
Bianchi AYap ZLertjaturaphat PHenley AMoon KKim Y(2024)Inline Visualization and Manipulation of Real-Time Hardware Log for Supporting Debugging of Embedded ProgramsProceedings of the ACM on Human-Computer Interaction10.1145/36602508:EICS(1-26)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660250
Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Show More Cited By

Index Terms

Bifröst: Visualizing and Checking Behavior of Embedded Systems across Hardware and Software
1. Human-centered computing

Recommendations

WiFröst: Bridging the Information Gap for Debugging of Networked Embedded Systems
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

The rise in prevalence of Internet of Things (IoT) technologies has encouraged more people to prototype and build custom internet connected devices based on low power microcontrollers. While well-developed tools exist for debugging network communication ...
Inline Visualization and Manipulation of Real-Time Hardware Log for Supporting Debugging of Embedded Programs
EICS

The advent of user-friendly embedded prototyping systems, exemplified by platforms like Arduino, has significantly democratized the creation of interactive devices that combine software programs with electronic hardware. This interconnection between ...
Minimally Invasive HW/SW Co-debug Live Visualization on Architecture Level
GLSVLSI '21: Proceedings of the 2021 Great Lakes Symposium on VLSI

We present a tool that allows developers to debug hard- and software and their interaction in an early design stage. We combine a SystemC virtual prototype (VP) with an easily configurable and interactive graphical user interface and a standard software ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology

October 2017

870 pages

ISBN:9781450349819

DOI:10.1145/3126594

General Chair:
Krzysztof Gajos
Harvard University
,
Program Chairs:
Jennifer Mankoff
Carnegie Mellon University
,
Chris Harrison
Carnegie Mellon University

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2017

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

TerraSwarm
NSF

Conference

UIST '17

Sponsor:

UIST '17: The 30th Annual ACM Symposium on User Interface Software and Technology

October 22 - 25, 2017

QC, Québec City, Canada

Acceptance Rates

UIST '17 Paper Acceptance Rate 73 of 324 submissions, 23%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
1,139
Total Downloads

Downloads (Last 12 months)195
Downloads (Last 6 weeks)20

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lima ATrevisan D(2024)Challenges and Opportunities in Using Electronic Prototyping as a Maker ToolProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702092(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702092
Bianchi AYap ZLertjaturaphat PHenley AMoon KKim Y(2024)Inline Visualization and Manipulation of Real-Time Hardware Log for Supporting Debugging of Embedded ProgramsProceedings of the ACM on Human-Computer Interaction10.1145/36602508:EICS(1-26)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660250
Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Hähnlein FBernstein GSchulz A(2024)Understanding and Supporting Debugging Workflows in CADProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676353(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676353
Methfessel PBeckmann TRein PRamson SHirschfeld R(2024)MµSE: Supporting Exploration of Software-Hardware Interactions Through ExamplesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642186(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642186
Hodges SKristensson PHester JKrüger AMankoff JOlivier PRogers Y(2023)Hardware is Hard—is it Worth it?Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3583751(1-4)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3583751
Bianchi AHodges SCuartielles DOh HLambrichts MRoudaut A(2023)Beyond prototyping boards: future paradigms for electronics toolkitsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3573792(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3573792
Jiang PSun FXia H(2023)Log-it: Supporting Programming with Interactive, Contextual, Structured, and Visual LogsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581403(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581403
Brocker ASchäfer RRemy CVoelker SBorchers J(2023)Flowboard: How Seamless, Live, Flow-Based Programming Impacts Learning to Code for Embedded ElectronicsACM Transactions on Computer-Human Interaction10.1145/353301530:1(1-36)Online publication date: 7-Mar-2023
https://dl.acm.org/doi/10.1145/3533015
Qiao WWu JRen HZhang ZTan B(2023)An Efficient DSP Simulator Design Supporting Multi-Core Debugging2023 IEEE 23rd International Conference on Communication Technology (ICCT)10.1109/ICCT59356.2023.10419585(90-95)Online publication date: 20-Oct-2023
https://doi.org/10.1109/ICCT59356.2023.10419585
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents