research-article

JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation

Authors:

Shunichi Kasahara,

Jun RekimotoAuthors Info & Claims

AH '14: Proceedings of the 5th Augmented Human International Conference

Article No.: 46, Pages 1 - 8

https://doi.org/10.1145/2582051.2582097

Published: 07 March 2014 Publication History

Abstract

JackIn is a new human-human communication framework for connecting two or more people. With first-person view video streaming from a person (called Body) wearing a transparent head-mounted display and a head-mounted camera, the other person (called Ghost) participates in the shared first-person view. With JackIn, people's activities can be shared and assistance or guidance can be given through other peoples expertise. This can be applied to daily activities such as cooking lessons, shopping navigation, education in craft-work or electrical work, and sharing experiences of sporting and live events. For a better viewing experience with frist-person view, we developed the out-of-body view in which first-person images are integrated to construct a scene around a Body, and a Ghost can virtually control the viewpoint to look around the space surrounding the Body. We also developed a tele-pointing gesture interface. We conducted an experiment to evaluate how effective this framework is and found that Ghosts can understand the spatial situation of the Body.

References

[1]

Tachi, S., Arai, H., Maeda, T., Oyama, E., Tsunemoto, N. and Inoue, Y., "Tele-existence in real world and virtual world," Advanced Robotics, 1991.193--198

[2]

Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proc, CHI 2013. ACM Press (2013), pp.179--188.

Digital Library

[3]

Tecchia, F., Alem, L. and Huang, W. 2012. 3D helping hands: a gesture based MR system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI '12). pp.323--328.

Digital Library

[4]

Kuzuoka, H., Spatial workspace collaboration: a SharedView video support system for remote collaboration capability, CHI 1992, pp.533--540, 1992.

Digital Library

[5]

Fussell, S. R., Setlock, L. D. and Kraut, R. E. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks, CHI 2003, pp. 513--520. 2003.

Digital Library

[6]

Kirk, D., Rodden, T. and Fraser, D. S. 2007. Turn it this way: grounding collaborative action with remote gestures. CHI 2007, pp.1039--1048, 2007.

Digital Library

[7]

Steve M., Telepointer: Hands-Free Completely Self Contained Wearable Visual Augmented Reality without Headwear and without any Infrastructural Reliance. In Proceedings of ISWC '00, 177--178. 2000.

Digital Library

[8]

Bauer, M., Kortuem, G. Segall, Z. "Where Are You Pointing At?" A Study of Remote Collaboration in a Wearable Videoconference System. ISWC 1999, pp.151--.158, 1999.

Digital Library

[9]

Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H. and Billinghurst, M. 2004. Remote Collaboration using a Shoulder-Worn Active Camera/Laser. In Proc. ISWC 2004. 62--69.

Digital Library

[10]

Gauglitz, S., Lee, C., Turk M. and Hllerer, T. Integrating the physical environment into mobile remote collaboration. In Proc. MobileHCI '12. ACM Press (2012) 241--250.

Digital Library

[11]

Ou, J., Fussell, S. R., Chen, X., Setlock, L. D. and Yang, J. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proc. ICMI '03. ACM Press (2003), 242--249.

Digital Library

[12]

Snavely, N., Seitz, S. and Szeliski, R. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (TOG) 2006. 83--846

Digital Library

[13]

SmartAR (http://www.sony.net/SonyInfo/News/Press/201105/11-058E/index.html)

[14]

LUMUS (http://www.lumus-optical.com/)

[15]

Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A. and Fitzgibbon, A. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. UIST '11. ACM Press (2011) 559--568.

Digital Library

[16]

L. Cheng and J. Robinson, Dealing with speed and robustness issues for video-based registration on a wearable computing platform. In Proc. ISWC98, pages 84.91, 1998.

Digital Library

[17]

Baudisch, P. and Rosenholtz, R. Halo: a technique for visualizing off-screen objects. In Proc. CHI '03. ACM Press (2013) 481--488.

Digital Library

[18]

Benson, Alan J. (2002). "Motion Sickness". In Kent B. Pandoff and Robert E. Burr. Medical Aspects of Harsh Environments 2. Washington, D.C.: Borden Institute. pp. 1048--1083. Retrieved 4 Dec 2012.

[19]

Goldberg, K., Song, D., Khor, Y., Pescovitz, D., Levandowski, A., Himmelstein, J., Shih, J., Ho, A., Paulos, E. and Donath, J. Collaborative Online Teleoperation with Spatial Dynamic Voting and a Human "Tele-Actor" In Proc. ICRA '02. 1179--1184.

[20]

Tamaki, E., Miyaki, T. and Rekimoto, J. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proc. CHI2011, ACM Press (2011) 543--552.

Digital Library

[21]

Leap Motion (https://www.leapmotion.com/)

[22]

William Gibson, Neuromancer, Ace Science Fiction, 1984.

[23]

GoPro Official Website, http://gopro.com/

[24]

Leonard, J. J. and Durrant-whyte, H. F., Simultaneous map building and localization for an autonomous mobile robot. Proceedings IROS'91. IEEE/RSJ International Workshop, pp.1442--1447, 1991.

[25]

Rekimoto, J. "Traxion: A Tactile Interaction Device with Virtual Force Sensation, ACM UIST 2013, pp.427--432, 2013.

Digital Library

[26]

Gerhard, R., Ethan, E., and Tom, W., D. Semi-automatic Annotations in Unknown Environments. In Proceeding of ISMAR '2007, pp. 67--70, 2007.

Cited By

Monnai YMuramoto GYazaki TNaitou RSaito HMaekawa AInami M(2024)共進化するテラヘルツ無線技術と人間拡張工学IEICE Communications Society Magazine10.1587/bplus.18.14518:2(145-157)Online publication date: 2024
https://doi.org/10.1587/bplus.18.145
Rastegar Sani MZollmann SLanglotz T(2024)Train Me: Exploring Mobile Sports Capture and Replay for Immersive Sports CoachingAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680239(1-7)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680239
Pfeil KBadillo-Urquiola KBelga JSera-Josef JLaviola JWisniewski P(2024)Using Co-Design with Streamers and Viewers to Identify Values and Resolve Tensions in the Design of Interpersonal Wearable Telepresence SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374258:CSCW1(1-21)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637425
Show More Cited By

Index Terms

JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices
      2. Touch screens
    2. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Recommendations

JackIn space: designing a seamless transition between first and third person view for effective telepresence collaborations
AH '17: Proceedings of the 8th Augmented Human International Conference

Traditional telepresence systems only supported first person view and users had difficulty in recognizing the surrounding situation of their remote workspace. JackIn Space is a telepresence system that solves this problem by seamlessly integrating first ...
JackIn head: an immersive human-human telepresence system
SA '15: SIGGRAPH Asia 2015 Emerging Technologies

Sharing full immersive experience in real-time has been the one of ultimate goals of telecommunication. Possible application can include various applications such as entertainment, sports viewing, education, social network and professional assistance. ...
Augmented Human: Augmented Reality and Beyond
AltMM'18: Proceedings of the 3rd International Workshop on Multimedia Alternate Realities

Will Augmented Reality (AR) allow us to access digital information, experience others' stories, and thus explore alternate realities? AR has recently attracted attention again due to the rapid advances in related multimedia technologies as well as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

AH '14: Proceedings of the 5th Augmented Human International Conference

March 2014

249 pages

ISBN:9781450327619

DOI:10.1145/2582051

General Chair:
Tsutomu Terada
Kobe University, Japan
,
Program Chairs:
Masahiko Inami
Keio University, Japan
,
Kai Kunze
Osaka Prefecture University, Japan
,
Takuya Nojima
University of Electro-communications, Japan

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

MEET IN KOBE 21st Century: MEET IN KOBE 21st Century

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AH '14

Sponsor:

MEET IN KOBE 21st Century

AH '14: 5th Augmented Human International Conference

March 7 - 8, 2014

Kobe, Japan

Acceptance Rates

Overall Acceptance Rate 121 of 306 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

105
Total Citations
View Citations
1,542
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)11

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Monnai YMuramoto GYazaki TNaitou RSaito HMaekawa AInami M(2024)共進化するテラヘルツ無線技術と人間拡張工学IEICE Communications Society Magazine10.1587/bplus.18.14518:2(145-157)Online publication date: 2024
https://doi.org/10.1587/bplus.18.145
Rastegar Sani MZollmann SLanglotz T(2024)Train Me: Exploring Mobile Sports Capture and Replay for Immersive Sports CoachingAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680239(1-7)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680239
Pfeil KBadillo-Urquiola KBelga JSera-Josef JLaviola JWisniewski P(2024)Using Co-Design with Streamers and Viewers to Identify Values and Resolve Tensions in the Design of Interpersonal Wearable Telepresence SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374258:CSCW1(1-21)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637425
Doerr NLee BBaricova KSchmalstieg DSedlmair M(2024)Visual Highlighting for Situated Brushing and LinkingComputer Graphics Forum10.1111/cgf.1510543:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15105
Li YLuo GXu YHe YZhang FZhang S(2024)AdaPIP: Adaptive picture-in-picture guidance for 360° film watchingComputational Visual Media10.1007/s41095-023-0347-310:3(487-503)Online publication date: 2-May-2024
https://doi.org/10.1007/s41095-023-0347-3
Huang WBillinghurst MAlem LXiao CRasmussen THuang WBillinghurst MAlem LXiao CRasmussen T(2024)Evaluating Augmented Reality Remote Guidance SystemsComputer‐Supported Collaboration10.1002/9781119719830.ch6(143-171)Online publication date: 24-May-2024
https://doi.org/10.1002/9781119719830.ch6
Yazaki TUriu DWatanabe YTakagi RKashino ZInami M(2023)“Oh, could you also grab that?”: A case study on enabling elderly person to remotely explore a supermarket using a wearable telepresence systemProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627799(340-352)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627799
Yazaki TWatanabe YKong LInami M(2023)Design and Field Study of Syn-Leap: A Symmetric Telepresence System for Immersion Switching and Walking Across Multiple LocationsProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627772(353-365)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627772
Zaman FAnslow CRhee T(2023)Vicarious: Context-aware Viewpoints Selection for Mixed Reality CollaborationProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615709(1-11)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3611659.3615709
Cho HKomar MLindlbauer D(2023)RealityReplayProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108887:3(1-25)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610888
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents