research-article

FS-Pad: Video Game Interactions Using Force Feedback Gamepad

Authors:

Youngbo Aram Shim,

Geehyuk LeeAuthors Info & Claims

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

Pages 938 - 950

https://doi.org/10.1145/3379337.3415850

Published: 20 October 2020 Publication History

Abstract

Force feedback has not been fully explored in modern gaming environments where a gamepad is the main interface. We developed various game interaction scenarios where force feedback through the thumbstick of the gamepad can be effective, and categorized them into five themes. We built a haptic device and control system that can support all presented interactions. The resulting device, FS-Pad, has sufficient fidelity to be used as a haptic game interaction design tool. To verify the presented interactions and effectiveness of the FS-Pad, we conducted a user study with game players, developers, and designers. The subjects used an FS-Pad while playing a demo game and were then interviewed. Their feedback revealed the actual needs for the presented interactions as well as insight into the potential design of game interactions when applying FS-Pad.

Supplementary Material

VTT File (3379337.3415850.vtt)

Download
5.65 KB

MP4 File (ufp4940pv.mp4)

Preview video

Download
27.96 MB

MP4 File (ufp4940vf.mp4)

Video figure

Download
36.24 MB

MP4 File (3379337.3415850.mp4)

Presentation Video

Download
57.63 MB

References

[1]

Yoshitaka Adachi, Takahiro Kumano, and Kouichi Ogino. 1995. Intermediate representation for stiff virtual objects. In Proceedings Virtual Reality Annual International Symposium'95. IEEE, 203--210.

[2]

Sheldon Andrews, Javier Mora, Jochen Lang, and Won Sook Lee. 2006. Hapticast: a physically-based 3D game with haptic feedback. Proceedings of FuturePlay (2006).

[3]

M Angerilli, A Frisoli, F Salsedo, S Marcheschi, and M Bergamasco. 2001. Haptic simulation of an automotive manual gearshift. In Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No. 01TH8591). IEEE, 170--175.

[4]

Entertainment Software Association and others. 2019. Essential facts about the computer and video game industry. (2019).

[5]

Byunghoon Bae, Taeoh Koo, Kyihwan Park, and Yongdae Kim. 2001. Design and control of a two degree of freedom haptic device for the application of PC video games. In Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180), Vol. 3. IEEE, 1738--1743.

[6]

Raghav Bhardwaj. 2016. The ergonomic development of video game controllers. Journal of Ergonometrics 7 (2016), 209--214.

[7]

David B Camarillo, Thomas M Krummel, and J Kenneth Salisbury Jr. 2004. Robotic technology in surgery: past, present, and future. The American Journal of Surgery 188, 4 (2004), 2--15.

[8]

Lo"ic Caroux, Katherine Isbister, Ludovic Le Bigot, and Nicolas Vibert. 2015. Player--video game interaction: A systematic review of current concepts. Computers in Human Behavior 48 (2015), 366--381.

Digital Library

[9]

Dean Chang. 2002. Haptics: gaming's new sensation. Computer 35, 8 (2002), 84--86.

Digital Library

[10]

Daniela Constantinescu, Septimiu E Salcudean, and Elizabeth A Croft. 2005. Local model of interaction for haptic manipulation of rigid virtual worlds. The International Journal of Robotics Research 24, 10 (2005), 789--804.

Digital Library

[11]

C Doerrer and R Werthschuetzky. 2002. Simulating push-buttons using a haptic display: Requirements on force resolution and force-displacement curve. In Proc. EuroHaptics. 41--46.

[12]

Mohamad Eid, Ahmad El Issawi, and Abdulmotaleb El Saddik. 2014. Slingshot 3D: A synchronous haptic-audio-video game. Multimedia tools and applications 71, 3 (2014), 1635--1649.

[13]

David Feygin, Madeleine Keehner, and R Tendick. 2002. Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. In Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE, 40--47.

[14]

Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: dynamic physical affordances and constraints through shape and object actuation. In Uist, Vol. 13. 2501988--2502032.

[15]

Antonio Frisoli, Carlo Alberto Avizzano, and Massimo Bergamasco. 2001. Simulation of a manual gearshift with a 2-DOF force-feedback joystick. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), Vol. 2. IEEE, 1364--1369.

[16]

R Brent Gillespie, M O'Modhrain, Philip Tang, David Zaretzky, and Cuong Pham. 1998. The virtual teacher. In Proceedings of the ASME Dynamic Systems and Control Division, Vol. 64. American Society of Mechanical Engineers, 171--178.

[17]

Ashley L Guinan, Nathaniel A Caswell, Frank A Drews, and William R Provancher. 2013. A video game controller with skin stretch haptic feedback. In 2013 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 456--457.

[18]

Ashley L Guinan, Rebecca L Koslover, Nathaniel A Caswell, and William R Provancher. 2012a. Bi-manual skin stretch feedback embedded within a game controller. In 2012 IEEE Haptics Symposium (HAPTICS). IEEE, 255--260.

[19]

Ashley L Guinan, Markus N Montandon, Nathaniel A Caswell, and William R Provancher. 2012b. Skin stretch feedback for gaming environments. In 2012 IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2012) Proceedings. IEEE, 101--106.

[20]

Peter Gyory, Clement Zheng, Daniel Leithinger, and Ellen Yi-Luen Do. 2019. HOT SWAP: Probing Embodied Game Interfaces With Reconfigurable Controllers. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion. ACM, 183--187.

Digital Library

[21]

Christian Hatzfeld and Thorsten A Kern. 2016. Engineering Haptic Devices. Springer.

[22]

Kazuyuki Henmi and Tsuneo Yoshikawa. 1998. Virtual lesson and its application to virtual calligraphy system. In Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146), Vol. 2. IEEE, 1275--1280.

[23]

Jingun Jung, Eunhye Youn, and Geehyuk Lee. 2017. PinPad: touchpad interaction with fast and high-resolution tactile output. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2416--2425.

Digital Library

[24]

AJ Kelley and SE Salcudean. 1994. On the development of a force-feedback mouse and its integration into a graphical user interface. ASME Dynamic Systems and Control 1, 55--1 (1994), 287--294.

[25]

Hyunyoung Kim, Céline Coutrix, and Anne Roudaut. 2018. KnobSlider: Design of a Shape-Changing UI for Parameter Control. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[26]

Sung Min Kim and Mee Young Sung. 2014. A Haptic Gaming System for Tactile Textures and 3D Shapes Discrimination. International Journal of Multimedia and Ubiquitous Engineering 9, 9 (2014), 319--334.

[27]

H Igo Krebs, Neville Hogan, Mindy L Aisen, and Bruce T Volpe. 1998. Robot-aided neurorehabilitation. IEEE transactions on rehabilitation engineering 6, 1 (1998), 75--87.

[28]

Yi-Chi Liao, Sunjun Kim, Byungjoo Lee, and Antti Oulasvirta. 2020. Press' Em: Simulating Varying Button Tactility via FDVV Models. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--4.

Digital Library

[29]

Karon E MacLean. 2008. Haptic interaction design for everyday interfaces. Reviews of Human Factors and Ergonomics 4, 1 (2008), 149--194.

[30]

Thomas H Massie, J Kenneth Salisbury, and others. 1994. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems, Vol. 55. Chicago, IL, 295--300.

[31]

William A McNeely, Kevin D Puterbaugh, and James J Troy. 2005. Six degree-of-freedom haptic rendering using voxel sampling. In ACM SIGGRAPH 2005 Courses. 42--es.

Digital Library

[32]

Stefano Mintchev, Marco Salerno, Alexandre Cherpillod, Simone Scaduto, and Jamie Paik. 2019. A portable three-degrees-of-freedom force feedback origami robot for human--robot interactions. Nature Machine Intelligence 1, 12 (2019), 584--593.

[33]

Dan Morris, Neel Joshi, and Kenneth Salisbury. 2004. Haptic battle pong: High-degree-of-freedom haptics in a multiplayer gaming environment. In Experimental gameplay workshop, GDC, Vol. 192.

[34]

Dan Morris, Hong Tan, Federico Barbagli, Timothy Chang, and Kenneth Salisbury. 2007. Haptic feedback enhances force skill learning. In Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07). IEEE, 21--26.

Digital Library

[35]

Konstantinos Moustakas, Georgios Nikolakis, Konstantinos Kostopoulos, Dimitrios Tzovaras, and Michael G Strintzis. 2007. Haptic rendering of visual data for the visually impaired. IEEE MultiMedia 14, 1 (2007), 62--72.

Digital Library

[36]

Keith V Nesbitt and Ian Hoskens. 2008. Multi-sensory game interface improves player satisfaction but not performance. In Proceedings of the ninth conference on Australasian user interface-Volume 76. 13--18.

[37]

Mauricio Orozco, Juan Silva, Abdulmotaleb El Saddik, and Emil Petriu. 2012. The role of haptics in games. In Haptics rendering and applications. IntechOpen.

[38]

Ming Ouhyoung, Wu-Nan Tsai, Ming-Chang Tsai, Jiann-Rong Wu, Chung-Hsi Huang, and Tzong-Jer Yang. 1995. A low-cost force feedback joystick and its use in PC video games. IEEE Transactions on Consumer Electronics 41, 3 (1995), 787--794.

Digital Library

[39]

M Sile O'Modhrain and Brent Gillespie. 1997. The moose: A haptic user interface for blind persons. In Proc. Third WWW6 Conference.

[40]

Sabrina Paneels and Jonathan C Roberts. 2009. Review of designs for haptic data visualization. IEEE Transactions on Haptics 3, 2 (2009), 119--137.

Digital Library

[41]

Kyihwan Park, Byunghoon Bae, and Taeoh Koo. 2004. A haptic device for PC video game application. Mechatronics 14, 2 (2004), 227--235.

[42]

Wei Peng and Gary Hsieh. 2012. The influence of competition, cooperation, and player relationship in a motor performance centered computer game. Computers in Human Behavior 28, 6 (2012), 2100--2106.

Digital Library

[43]

Niklas Ravaja, Timo Saari, Marko Turpeinen, Jari Laarni, Mikko Salminen, and Matias Kivikangas. 2006. Spatial presence and emotions during video game playing: Does it matter with whom you play? Presence: Teleoperators and Virtual Environments 15, 4 (2006), 381--392.

Digital Library

[44]

L Reinstein. 1983. de Quervain's stenosing tenosynovitis in a video games player. Archives of physical medicine and rehabilitation 64, 9 (1983), 434--435.

[45]

Anne Roudaut, Andreas Rau, Christoph Sterz, Max Plauth, Pedro Lopes, and Patrick Baudisch. 2013. Gesture output: eyes-free output using a force feedback touch surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2547--2556.

Digital Library

[46]

Diego C Ruspini, Krasimir Kolarov, and Oussama Khatib. 1997. The haptic display of complex graphical environments. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 345--352.

Digital Library

[47]

Marco Salerno, Stefano Mintchev, Alexandre Cherpillod, Simone Scaduto, and Jamie Paik. 2018. Stiffness Perception of Virtual Objects Using FOLDAWAY-Touch. In International AsiaHaptics conference. Springer, 139--143.

[48]

Marco Salerno, Jamie Paik, and Stefano Mintchev. 2019. FOLDAWAY DroneSense, a controller for haptic information encoding for drone pilots. (2019). http://infoscience.epfl.ch/record/267463

[49]

Kenneth Salisbury, Francois Conti, and Federico Barbagli. 2004. Haptic rendering: introductory concepts. IEEE computer graphics and applications 24, 2 (2004), 24--32.

[50]

Oliver Schneider, Jotaro Shigeyama, Robert Kovacs, Thijs Jan Roumen, Sebastian Marwecki, Nico Boeckhoff, Daniel Amadeus Gloeckner, Jonas Bounama, and Patrick Baudisch. 2018. DualPanto: A Haptic Device that Enables Blind Users to Continuously Interact with Virtual Worlds. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 877--887.

Digital Library

[51]

Ali Shahrokni, Julio Jenaro, Tomas Gustafsson, Andreas Vinnberg, Johan Sandsjö, and Morten Fjeld. 2006. One-dimensional force feedback slider: going from an analogue to a digital platform. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles. 453--456.

Digital Library

[52]

Thomas N Smyth and Arthur E Kirkpatrick. 2006. A new approach to haptic augmentation of the GUI. In Proceedings of the 8th international conference on Multimodal interfaces. 372--379.

Digital Library

[53]

Jorge Solis, Carlo Alberto Avizzano, and Massimo Bergamasco. 2002. Teaching to write Japanese characters using a haptic interface. In Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE, 255--262.

[54]

Penelope Sweetser and Peta Wyeth. 2005. GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE) 3, 3 (2005), 3--3.

Digital Library

[55]

Chee Leong Teo, Etienne Burdet, and HP Lim. 2002. A robotic teacher of Chinese handwriting. In Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE, 335--341.

[56]

Nicolas Villar, Kiel Gilleade, Devina Ramdunyellis, and Hans Gellersen. 2007. The VoodooIO gaming kit: a real-time adaptable gaming controller. Computers in Entertainment (CIE) 5, 3 (2007), 7.

Digital Library

[57]

Christopher R Wagner, Nicholas Stylopoulos, Patrick G Jackson, and Robert D Howe. 2007. The benefit of force feedback in surgery: Examination of blunt dissection. Presence: teleoperators and virtual environments 16, 3 (2007), 252--262.

[58]

Yasuyoshi Yokokohji, Ralph L Hollis, Takeo Kanade, Kazuyuki Henmi, and Tsuneo Yoshikawa. 1996. Toward machine mediated training of motor skills. Skill transfer from human to human via virtual environment. In Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA. IEEE, 32--37.

[59]

Wai Yu, Ramesh Ramloll, and Stephen Brewster. 2000. Haptic graphs for blind computer users. In International Workshop on Haptic Human-Computer Interaction. Springer, 41--51.

Digital Library

[60]

Craig B Zilles and J Kenneth Salisbury. 1995. A constraint-based god-object method for haptic display. In Proceedings 1995 ieee/rsj international conference on intelligent robots and systems. Human robot interaction and cooperative robots, Vol. 3. IEEE, 146--151.

Cited By

Kojima MYoshimoto SYamamoto A(2024)Design of a haptic device for presenting pressure and skin stretching stimuli to the palmFrontiers in Mechanical Engineering10.3389/fmech.2024.141788110Online publication date: 29-Jul-2024
https://doi.org/10.3389/fmech.2024.1417881
Wu YYi AMa CChen L(2023)Artificial intelligence for video game visualization, advancements, benefits and challengesMathematical Biosciences and Engineering10.3934/mbe.202368620:8(15345-15373)Online publication date: 2023
https://doi.org/10.3934/mbe.2023686
Shim YLee SYi HLee G(2023)ExpanStick: Augmenting Gamepad Thumbstick using Force at the LimitProceedings of the ACM on Human-Computer Interaction10.1145/36110427:CHI PLAY(588-610)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3611042
Show More Cited By

Index Terms

FS-Pad: Video Game Interactions Using Force Feedback Gamepad
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Tactile and hand-based interfaces
      1. Haptic devices
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction devices
      1. Haptic devices
      2. Pointing devices

Recommendations

Demonstrating Gamepad with Programmable Haptic Texture Analog Buttons
UIST '18 Adjunct: Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

We demonstrate a haptic feedback method to generate multiple virtual textures on analog buttons of the gamepad. The method utilizes the haptic illusion evoked from proper haptic cues in respect of the analog button's movement to change the perceived ...
ExpanStick: Augmenting Gamepad Thumbstick using Force at the Limit

We present ExpanStick, a novel thumbstick that augments video game interaction using the force that the user applies at the physical limit of the gamepad. Gamers frequently manipulate a thumbstick leaning on the limit, or physical boundary around it, ...
Gamepad vs. touchscreen: a comparison of action selection interfaces in computer games
WASA '12: Proceedings of the Workshop at SIGGRAPH Asia

In this paper, we compare gamepad and touchscreen interfaces for action selection tasks in computer games. Touchscreens are now widely used for computer games on tablets, smartphones and hand-held game consoles. However, in general, game players are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

October 2020

1297 pages

ISBN:9781450375146

DOI:10.1145/3379337

General Chairs:
Shamsi Iqbal
Microsoft Research, USA
,
Karon MacLean
University of British Columbia, Canada
,
Program Chairs:
Fanny Chevalier
University of Toronto, Canada
,
Stefanie Mueller
MIT CSAIL, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministry of Science and ICT, South Korea

Conference

UIST '20

Sponsor:

UIST '20: The 33rd Annual ACM Symposium on User Interface Software and Technology

October 20 - 23, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
846
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)20

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kojima MYoshimoto SYamamoto A(2024)Design of a haptic device for presenting pressure and skin stretching stimuli to the palmFrontiers in Mechanical Engineering10.3389/fmech.2024.141788110Online publication date: 29-Jul-2024
https://doi.org/10.3389/fmech.2024.1417881
Wu YYi AMa CChen L(2023)Artificial intelligence for video game visualization, advancements, benefits and challengesMathematical Biosciences and Engineering10.3934/mbe.202368620:8(15345-15373)Online publication date: 2023
https://doi.org/10.3934/mbe.2023686
Shim YLee SYi HLee G(2023)ExpanStick: Augmenting Gamepad Thumbstick using Force at the LimitProceedings of the ACM on Human-Computer Interaction10.1145/36110427:CHI PLAY(588-610)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3611042
Kojima MYoshimoto SYamamoto A(2023)Grip-Type Pseudo Force Display with Normal and Tangential Skin StimulationNew Trends in Medical and Service Robotics10.1007/978-3-031-32446-8_7(63-70)Online publication date: 19-May-2023
https://doi.org/10.1007/978-3-031-32446-8_7
Lin HHe LSong FLi YCheng TZheng CWang WOh H(2022)FlexHaptics: A Design Method for Passive Haptic Inputs Using Planar Compliant StructuresProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502113(1-13)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502113
Islam MAli RHaider AKim H(2022)QoS Provisioning: Key Drivers and Enablers Toward the Tactile Internet in Beyond 5G EraIEEE Access10.1109/ACCESS.2022.319790010(85720-85754)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3197900
Fukuda A(2021)Onomatangiplay : A Game Experience with a Tangible Auditory User Interface Created by Representing Game Effects with a Real-world Sound SourceExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3451853(1-4)Online publication date: 8-May-2021
https://dl.acm.org/doi/10.1145/3411763.3451853

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents