1 Introduction

In recent years, we all have witnessed a rapid evolution of user interfaces and a fundamental shift in the goals that researchers and practitioners of human–computer interaction (HCI) pursue. Traditionally, HCI focused on the personal computer as a desktop system or other stand-alone devices with single displays. Even emerging or recently established styles of interaction such as multi-touch input are mostly restricted to single device usage. Today, however, a great deal of HCI research work follows a new and more holistic notion of what an interactive system is and where its boundaries are: Single devices are used as parts of greater and more complex systems that constitute entire “interactive spaces,” i.e., physical rooms or spaces in which several or even dozens of devices and displays of different sizes and form factors are used complementarily to serve a single user or a group of users as one dynamic user interface. By this, HCI researchers increasingly realize Mark Weiser’s vision of a ubiquitous computing that is based on multiple tabs, pads, and boards and where “the real power of the concept comes not from any of these devices; it emerges from the interaction of all of them” [1].

User input to these systems happens simultaneously at different locations and on different devices and also includes multi-touch, pen, gestural, eye gaze, or tangible interactions. System output is spatially distributed across many displays that are simultaneously provided by smartphones, tablets, interactive tabletops, (mobile) projections, or large and even wall-sized interactive displays. As a consequence, the overall complexity of the design of such interactive spaces is far greater than that of typical systems in HCI research that follow the traditional “Windows, Icons, Menus, Pointer” (WIMP) paradigm and are mostly restricted to single person, single system use.

Despite this complexity, post-WIMP interactive spaces are of growing scientific and commercial importance as was expressed for example by Microsoft’s Chief Research and Strategy Officer Craig Mundie in 2011: “I believe the successor to the desktop is the room, that instead of thinking that the computer is just something on the desk that you go and sit in front of, in the future basically the whole room is the computer and you go in it” [2]. The promise of post-WIMP interactive spaces is to boost users’ creativity and productivity by augmenting the physical rooms, spaces, desks, and walls in our natural environment with digital tools. To achieve this boost, it is necessary to design them so that they feel “natural,” seamless, and usable and do not force users to perform what Weiser called “mental gymnastics” [1]. Rather than interfering with well-established non-digital practices, these new digital tools should coexist and benefit from the users’ preexisting physical and social skills. Regardless of whether our interactive space is intended to support data analysis (see “VisPorter: facilitating information sharing for collaborative sense making on multiple displays” in this issue) or to generate music from dancing movements (see “Collaborative creativity: The Music Room” in this issue), the necessary interactions should not conflict with our natural and familiar ways of interacting with our physical and social world—unless this conflict is a conscious decision made by the designer.

As a consequence, the design of interactive spaces requires us to rethink familiar designs of interactive technologies and our theories and conceptual frameworks of user interaction. We must provide novel theories, designs, technologies, and best practice examples of post-WIMP interactive spaces to HCI researchers and practitioners alike. Therefore, in May 2012, the editors of this issue organized the workshop “Designing Collaborative Interactive Spaces for e-Creativity, e-Science and e-Learning (DCIS 2012)” Footnote 1 at the AVI 2012 conference. Thanks to the great support by our co-organizers, the interest in the workshop was overwhelming, and to our own regret, the number of submissions greatly exceeded the number that we could accept. It became clear to us that a theme issue on this topic would not only be of interest to a few workshop participants but also for the quickly growing number of researchers working in this field.

Therefore, for this theme issue, we have published an open call in October 2012 to invite submissions from participants of the DCIS workshop as well as from the research community in general. We received 17 high-quality submissions from authors working in 13 different countries, reflecting the global interest in this topic. After a careful reviewing process with 38 external reviewers and several reviews per submission, we selected eight articles for publication in this theme issue with two articles from the United States, two articles from Germany, and one article from each of the following countries: France, Italy, Switzerland, and the United Kingdom. By covering a broad range of topics, challenges, and applications, the articles of this issue provide great insights into an emerging research field, addressing aspects such as theory, architectures, technical issues, frameworks, interaction techniques, collaborative applications, as well as user studies.

2 The papers of this theme issue

In the first article, Jetter, Reiterer, and Geyer introduce Blended Interaction as a conceptual framework and theoretical basis for understanding and designing interaction in collaborative interactive spaces. Their framework of Blended Interaction is based on embodied cognition and cognitive linguistics and uses conceptual blends to explain how users rely on familiar and real-world concepts whenever they learn to use new digital technologies. The authors structure the underlying problem and design space into design domains and exemplify their framework using related work and recent research projects.

The following three papers all provide fascinating accounts of the variety of application domains for collaborative interactive spaces. First, in the field of brainstorming and group creativity support, Jaco et al. introduce a tabletop interface “Trains of Thought” that allows users to rapidly accumulate and link semantically related ideas using graphical ropes as a central visual metaphor. To evaluate the effectiveness of this interface, it was tested against a condition without ropes. The Train-of-Thought condition produced more original ideas and improved users’ creativity. Second, the article by Chung et al. is devoted to the field of visual analytics using multiple and heterogeneous displays. VisPorter is introduced as a system to be used in a multiple display and device environment. Devices can be fluidly connected by lightweight touch interactions, thereby supporting sense-making activities. Third, Morreale et al.’s central topic in the following contribution is collaborative creativity in composing and playing music. The so-called Music Room is described as an interactive installation where couples can compose original music by moving in space. Proxemic cues and emotional dimensions of people’s movements are used as input for the music composition system.

How to actually develop collaborative systems by means of toolkits is the question addressed in the subsequent article by Schmid, Masson, and Hirsbrunner. They introduce TWICE as a toolkit for Web-based interactive collaborative environments. TWICE simplifies the development of primarily synchronous colocated collaborative systems with standard Web technologies. The resulting applications allow for device heterogeneity and true multi-user interaction.

In the following article, Spindler et al. focus on the exploration of complex information spaces by means of spatially aware handheld displays. By utilizing the spatial position and orientation of several mobile devices for interaction purposes, the physical three-dimensional spaces around larger interactive displays can be used effectively. The work reflects on the underlying design space, provides a requirement analysis regarding the technical realization, and describes a spatial tracking solution with a consumer depth camera.

The next article also addresses the usage of multiple displays in physical rooms, but uses robotics to make their physical configuration and layout adaptable to users’ needs. Houayek et al. introduce a programmable “animated work environment” (AWE) as a result of an iterative design process in a transdisciplinary team. In its core, it is a reconfigurable, foldable wall comprised of eight surfaces and three horizontal, mobile work surfaces. The active change of the physical work environment by means of robotic, programmable structures allows for fascinating new opportunities of supporting various collaborative work activities.

In the final article of this theme issue, Oleksik, Milic-Frayling, and Jones report on the actual long-term usage of a multi-device collaborative work setting for team meetings. In-situ observations of such meetings in an office with a tabletop, tablet PCs, and a vertical display were conducted revealing interesting usage and interaction patterns. With observations on binding gestures, previous gesture taxonomies and the notion of multi-touch on single displays could be extended beyond individual devices.

3 Outlook

Our aim of putting together this theme issue has been to report on the current state of the art in the research area of collaborative interactive spaces. With contributions from a wide range of perspectives, we hope to have documented at least a part of the current state of research and to inspire future work in this field. In fact, together with three international collaborators, the editors could organize a follow-up event at the ACM SIGCHI Conference on Human Factors in Computing Systems 2013. The full-day CHI workshop “Blended Interaction: Envisioning Future Collaborative Interactive Spaces” Footnote 2 attracted some 33 participants and was an extremely successful event with 19 presented contributions and three keynotes by highly acclaimed researchers. This workshop clearly demonstrated the rising interest in the research and development of collaborative interactive spaces. As a result of the discussions, several challenges and avenues for future research were identified. A selection of them shall conclude this editorial.

From a technological perspective, how can we sense, track, and fuse devices and other sensors to allow for their seamless combination? How can we benefit from the sensory richness of our real world? Which components of a collaborative space should be realized in the real versus the digital domain, and which novel expressive powers do digital technologies provide to us? It is without any doubt that future device ecologies will also require us to tackle considerable challenges in terms of tool support, interoperability, and standards.

With regard to interaction and collaboration, existing workflows, practices, and everyday activities need to be considered even more than with single device usage. What happens if we move from individual to heavily collective usage of interfaces? How can we design for cooperative usage, and how do we support learning in such collaborative spaces? How should natural interaction techniques be designed and evaluated, which facilitate bridging the real and the digital domain and connecting multiple devices and multiple users? How can theory, frameworks, and models support and inform design activities?

What are application domains which benefit most from digitally enhanced collaboration and blended interaction? Will certain standards and design guidelines eventually emerge? Which particular implications will such blended collaborative spaces have on our social coexistence, on privacy, and related issues? It is very obvious that this short list of questions is far from being complete and only illustrates how many research challenges are currently emerging, which are yet to be addressed.