Social Interaction Design Patterns for Urban Media Architecture

Media architecture is an emergent field in the intersection of HCI, design, architecture, urban planning, art and sociology. While the use of media technologies in architecture has a long history – exemplified by the neon signs at Times Square in New York dating back to the 1920s – the field we know as media architecture emerged when designers and architects began to embed digital technologies, in particular displays, into the built environment. Decreasing prices of important media architecture components, such as display and sensor technologies, has led to uptake in industry, and it is now a prominent feature in many cities in the world. Simultaneously, the interest in media architecture has grown in the HCI community, and in addition to a dedicated conference series [17], media architecture research is represented at many HCI conferences, including INTERACT. Typical of emergent disciplines related to HCI, the initial focus in many academic contributions was on technical aspects and the potentials of this novel form of interface. To the extent that social aspects of media architecture have been examined, it has primarily been in relation to specific installations. From these contributions, it is clear that social aspects play a very large role in how media architecture is perceived and used, and that interaction designers working in media architecture need an understanding of both technical and social dimensions to develop successful installations.

In this paper, we therefore offer an overview and analysis of social interaction in media architecture through a comparison of nine representative cases from the relatively brief history of the field. We combine the findings on social interaction from each original case study to offer two complementary contributions: firstly, a framework outlining six different modes of social interaction in relation to media architecture: appreciation, self-expression, playfulness, collective narratives, triangulation, and negotiation of space; secondly, a set of seven social interaction design patterns for media architecture, which represent different strategies for designing media architecture to achieve specific types of social interaction: shadow playing, remote control, smooth operator, soapbox, amusement park, swarm, and automatic gate.

The intended audience of the paper is HCI researchers working within the field of media architecture, who may employ the framework and design patterns, examine the relations between the technical and social aspects of media architecture and to categorise and analyse further installations. In addition, the social interaction design patterns may be of value for HCI practitioners, since they indicate specific strategies for developing media architecture installations.

Goffman’s seminal works in the dynamics of interaction in public spaces and the consequent impact on human social behaviour provide a suitable framework for analysis of urban interventions with media architecture. In Relations in Public [9], Goffman classifies agents in a social context as being either ‘singles’ (individuals by themselves) or in a ‘with’ (two or more individuals together). Such a configuration has direct implications on the relationship between the various individuals in a public space: for example, people generally feel more comfortable with approaching ‘singles’ than groups. Likewise, individuals in a ‘with’ relationship would behave differently than they would if they were singles in the same space.

The notion of design patterns, inspired by the work of Alexander [1], provides a frame for our analysis, which at the same time is directed towards applying insights in practice. Alexander’s patterns for architecture were developed from studies of architectural practice and history, describing recurring configurations of the built environment that over the course of time had proven to have specific effects, such as courtyards as places that facilitate gatherings and social exchanges. Likewise, in the mid-90s, Gamma et al. [8] – the so-called Gang Of Four (GoF) – inspired by Alexander’s work, adapted the concept of patterns to software engineering. By presenting a series of design patterns as reusable solutions for recurrent software development problems, their work underpinned the wide adoption of object-oriented programming and paved the way for greater reusability and scalability of computer systems. While the history of media architecture is much shorter, we can nevertheless begin to derive recurring patterns for social interaction. As in the works by Alexander and the GoF, these patterns can serve as strategies for designers of future media architecture installations.

Social aspects of media architecture have been addressed in many existing contributions; however, they have mainly addressed phenomena and design strategies in singular or few cases. For example, Dalsgaard and Halskov [6] propose that designers can work towards ‘situational interaction flexibility’ as a strategy for allowing people to ease in and out of specific social configurations around installations; however, this has not yet been replicated. Some social aspects discussed in the literature have started out as singular observations, and have since then been found to hold across different cases. The ‘honeypot effect’ [4], which proposes that the presence of people interacting with an installation will entice new users to start interacting to a higher degree than if nobody was using it, is one such aspect that has been observed in multiple cases. This is related to the observation by Mueller et al. [18] that many people notice the interactive potentials of a media architecture installation by observing other people interacting with it. Yet, an analysis of the recurring patterns of social interaction posed by media architecture design solutions is still lacking in the literature. Our approach in this paper has been to derive social interaction modes and patterns across a range of cases, with respect to common design variables such as layout of the public space and strategies for interaction and feedback. To the extent that existing literature corroborates our findings, we will discuss these in the relevant parts of the analysis.

Our approach to analysing social interaction design patterns is based on a study of nine prominent examples of media architecture. We based the selection of works on two main design factors common to any responsive media architecture installation in public space: (a) the type of interface; and (b) the public space layout. We describe below each of those factors, the selection process and the steps taken in our analysis.

3.1 Types of Interfaces

Hespanhol and Tomitsch [12] argue that interactive behaviour around urban media architecture may emerge intuitively as a product of its level of accessibility and the type of feedback it offers, therefore leading to particular modes of social interaction. Analysing a variety of responsive public spaces, they propose a classification of interfaces into three types: performative, allotted and responsive ambient. Performative interfaces are defined as those where the interactive zones are well delimited, yet restricted to a small number of participants, resulting in a natural division of the public into ‘performers’ (active participants) and ‘spectators’ (passive participants). Allotted interfaces share the same basic characteristics as performative ones, however are large enough to accommodate a population of participants, so that each no longer has full visibility of the interface. Instead, participants operate locally on their own section of the interface, with interaction therefore distributed across the environment. The third category, responsive ambient, refers to urban interfaces which track and react to the presence of people, however offer indirect and generic feedback rather than responding to specific individuals.

We employ this classification in our analysis since it offers a framework for understanding how the design of media architecture may influence the social behaviours of the local public. To that end, we include in our analysis a selection of works that represent a balance of the different types of interfaces (see Table 1).

3.2 Types of Spatial Layouts

In categorising and analysing the spatial layouts, we employ two categories: (1) plaza, and (2) thoroughfare. These are based on the concepts of “spatial nodes” and “links” (respectively), as described in urban planning works such as Hillier & Hanson’s The Social Logic of Space (1984) [13]. We define a plaza as a wide, open public space where a large number of citizens potentially congregate, facilitating social encounters as well as passive social practices as people watching or even loitering. A thoroughfare, by contrast, is a transit area connecting plazas, therefore characterized by the continuous flux of passers-by walking from one destination to another. In our selection, the plaza examples outweigh the thoroughfares; this is representative of the fact that media architecture is more often placed in such settings.

3.3 Selection of Case Studies

We adopted the large survey on media architecture by Haeusler et al. [11], as well as more recent literature, as departing points for our selection process. The former showcases 33 international contemporary media façades, while from the literature we singled out another 17 works, resulting on a total sample of 50 installations. From this sample, we made a selection based on the following criteria: the works should span from early examples of media architecture to newly launched works; they should range from small-scale installations to building-sized ones; they should represent a scope of uses from research experiments and artistic interventions over civic participation projects to commissioned works; they should employ a range of interaction technologies; and they should have different spatial layouts.

Within that framework, and to the extent possible, we gave preference to works we had the chance to visit in situ, either on their original setup (Aarhus By Light [6], Solstice LAMP [12], Interference [15], The Climate on The Wall [6] and Chromapollination [12]) or as a subsequent iteration (SCSD [2]), since those could offer us first hand access to evaluate the social interaction impact on their visitors. The selection of remaining works was largely determined by the availability in the literature of an in-depth account of their social interaction dynamics. Due to length constraints, we decided to include only three more representative works (MyPosition [22], Body Movies [16] and iRiS [3, 23]), limiting the number of case studies presented in this paper to nine. While it is important to acknowledge the limitations inherent to this selection process, we will argue that our analysis can provided the basis for further identification of social interaction patterns in urban media architecture. The balance between the different types of interfaces and spatial layouts adopted as selection criteria provides a representative sample of the works found both in the comprehensive field survey by Haeusler et al. [11] as well as in more recent literature.

3.4 Analytical Approach

We have examined the academic contributions describing each installation, the types of interfaces and spatial layouts that it represents and, to the extent that this is described in the literature, the designers’ intentions with regards to affecting and/or shaping social interaction. That analysis led us to define six recurring modes of social interaction. By further observing the ways those have been recurrently combined across the selected works, we then identified seven distinct strategies, here labelled design patterns, commonly implemented in the installations to elicit intended modes of social interaction.

The goal of this section is to present a sample of previous and current works in the field (9 in total), illustrating various approaches to augmenting the urban environment. We describe each in regards to (1) the types of interfaces, interactive strategies and spatial layouts; and (2) how social interactions unfolded around it.

4.1 Smart Citizen Sentiment Dashboard (SCSD)

The Smart Citizen Sentiment Dashboard (SCSD) [2] introduced the notion of media architectural interfaces (MAIs), a design approach where a tangible user interface (TUI), positioned on public space, functions as a mediator for people to interact with content in a media façade (the carrier). The TUI was an analogue console allowing users to select one of five discussion topics (environment, transport, safety, public space and housing), and use radio frequency identification (RFID) cards to select among three different moods: happy, indifferent or sad. RFID cards are used in many cities as digital tickets for the public transport system and building access, and thus constitute a familiar instrument for interaction with the urban realm; the design goal, therefore, was to opportunistically intervene in the social situation of people walking along a busy street for a brief civic poll.

SCSD was deployed in 2013 to the very large (3700 sqm.) media façade of a commercial building in the largest avenue of Sao Paulo, Brazil, running for three weeks. Due to the scale of the façade and the narrow available space in the sidewalk in front of it, the TUI was positioned across the avenue in a small square near the entrance to the local metro station. A variety of emerging behaviour by passers-by and participants could be observed. A large proportion (28 %) explored the interface playfully, expressing conflicting moods to the same topics. The majority (72 %) of participants, however, did express meaningful opinions. Awareness about the visualisations was also verified across the broader surrounding space, with a particular prevalence of people taking photos of the façade. According to the authors, however, given the small scale of the TUI compared to the high visibility of the façade, it was clear that most passers-by were unaware of the mechanisms of interaction or even the meaning of the façade graphics, rather enjoying them as visually appealing ambient art.

4.2 MyPosition

MyPosition [22] is a system designed for civic participation in public spaces. It consists of a polling interface back projected into a 5 × 2 m canvas, integrated to depth-cameras used to track passers-by. The interface displays a polling question at the top and graphics depicting a 4-point scale: ‘strongly agree’, ‘agree’, ‘disagree’ and ‘strongly disagree’. Each section consists in a visualisation of the votes it has received, each vote displayed as a colourful tile. When a participant is detected in the area in front of the screen, the section corresponding to the position they occupy is animated to indicate preference. They can then vote by positioning themselves along the display in front of a desired section and raising their arms for 2s, during which a dwelling animation is displayed as feedback. Only one participant can interact with the system at any given time. After a vote is cast, a new tile is added to the chosen option. Three modalities of visual feedback are available: (1) identical tiles for all participants; (2) each tile with the participant silhouette; (3) each tile with the participant image captured by the cameras.

The system was deployed as a field study for a week in 2013, at the cafeteria foyer in a large university in Berlin, Germany. It revealed that the playfulness of the interface was not a hindrance to participation, with people interviewed stating they actually meant the answers they gave. Notably, when participants were identified (i.e. their votes were displayed as tiles containing their images) interaction events decreased and were more evenly distributed across the four options, which might be due to an increased accountability regarding the votes cast. In a broader sense, however, the authors’ observed social interaction across the whole precinct, beyond the direct interaction zones: there was a considerable level of discussion, social learning and teaching as well as nudging among the public [22]. The display was not only noticed but also successfully promoted active discussion about the proposed topics.

4.3 Body Movies

Body Movies [16] is an interactive installation by artist Rafael Lozano-Hemmer. Large-scale portraits of people are projected onto an urban wall, blanked by strong white lights. As passers-by walk and position themselves in front of the lights, they cast their shadows onto the wall and reveal the images underneath.

Since its first inception in 2001 at the V2 Grounding, Rotterdam, The Netherlands, Body Movies has been exhibited multiple times worldwide. Social interaction generally emerges with ease, prompted by the familiarity of shadow playing – the work has famously attracted large crowds, with people readily congregating in front of the exhibition wall and moving around the space in order to cast shadows of different sizes. Notably, however, interaction through the shadow playing itself often becomes the greatest appeal of the work, as opposed to the unveiling of the underlying images, as per the artist’s original concept [5]. Rather than detracting from the experience, such a spontaneous manifestation actually enriches it, not only by allowing individuals and small groups of friends to express themselves playfully in public, but also reportedly bridging the social gap between non-acquaintances. As it is typical of performative scenarios, interaction unfolds mainly in the area directly in front of the projections, via triangulation [19]: people communicating through their representations on the façade. Conversely, the spots along the periphery of the space turn into comfort spaces [7] where passive appreciation and social commentary can take place.

4.4 Aarhus By Light

Aarhus By Light [6] was an interactive media façade created in 2008 with 180 square meters of semi-transparent LED screen, distributed in a non-rectangular pattern behind the glass front wall of the Musikhuset in Aarhus, Denmark. The façade faced a public park where interactive zones were clearly identified by colourful mats placed on pathways leading to the venue. Aarhus By Light was one of the first public interfaces designed to support a “walk-up-and-use” experience, enabling full body interaction with a media façade via computer vision. It was designed so that passers-by in the park or going to the Musikhuset could quickly figure out the mechanisms of interaction by providing them with a clear and direct feedback in the shape of their silhouettes magnified and integrated with the visual graphics in the façade. When no person interacted with the installation, the façade displayed small, animated social creatures moving through it. When individuals were detected, the creatures would become aware of their presence, come closer to their silhouettes and start playing with them.

Social interaction unfolded seamlessly, with both individuals and groups interacting via their representations on the allotted façade. The work also managed to successfully address the various regions of proxemics around the venue: people walking along the streets around the park would become aware of the Musikhuset by observing both the animated façade and the crowd gathering in front of it; people in the park would engage both in active and passive interaction with the work, and those inside the venue could watch the animations in the façade from behind, highlighting the building itself as destination. The work successfully fulfilled its goal of transforming a whole urban precinct and the social dynamics around it with interactive technology.

4.5 Solstice LAMP

Solstice LAMP [12] was a large-scale interactive installation developed by the Design Lab, University of Sydney, Australia, for the 2013 edition of the local Vivid Sydney festival. The work was designed to reactivate a normally underused square facing a 100 m tall skyscraper in the centre of the city. It consisted of two sections: (1) two interactive rectangular zones in the building forecourt, delimited by projections on the floor; and (2) laser projections on the entire building façade, derived from the interaction unfolding at the forecourt. Equipped with overhead depth-view cameras and data projectors, the installation tracked people within the interactive zones and projected halos around their bodies. If people got close enough, their halos would merge as a feedback to the fact they were then a single entity. During a period of one minute, each halo would become increasingly brighter and eventually pop, moving away from their hosts towards the building and then up towards the top of the building. At that point, a new interactive cycle would start at the forecourt. People could enter the interactive zones from all directions and the work would respond to as many participants as it could fit simultaneously, characterizing the interface as allotted [12].

Interaction with the work varied across demographics and different situations. The authors observed three main social interaction patterns: (1) children attracting adults; (2) tentative single ‘exploration’ versus expansive group ‘performance’; (3) “body gloss”. The first refer to the fact children would readily break into interaction upon reaching the environment, with many parents noticeably observing them ‘trying’ the environment before joining them in. The second points to the remarkable difference between the observed behaviour of individuals and that of groups engaging in interaction: when the space was already full of other people by the time they start interacting, participants tended to demonstrate restrain and self-consciousness. However, when engaging with the work as part of a larger group of friends, they would try expansive movements like dancing, jumping or even running around. Finally, the third pattern was the amplification of what Goffman termed ‘body gloss’ [9], i.e. a visual reinforcement of personal space, here prompted by the projected graphics rather than emphatic body language: people not acquainted to each other would often apologise when their halos merged, even without any physical touch actually happening.

4.6 iRiS

iRiS [3, 23] is a system for remote interaction with a media façade via an application running on a smartphone, which allows participants to visualize the façade on their devices, superimposed with a user interface for drawing over the image. It identifies the colour selected by the user and the region of the image they are drawing over and turns the corresponding pixels in the façade accordingly. Multiple users can interact simultaneously with the façade, which is shared on a first-come, first-served basis.

The work was trialled at the Ars Electronica Festival 2010, in Linz, Austria, running at the exterior walls of the Ars Electronica Center. Evaluation conducted by the authors found that participants perceived the façade as (1) easy-to-learn, but also that (2) it might leave users unaware of the actions of others [3]. Concurrent users could see multiple actions unfolding on the screen, but were not always able to see who was responsible for them. Likewise, access to the interface was also sometimes denied to new users. The lack of social protocol for such an unfamiliar public space would often lead to user frustration when participants were not acquainted with each other.

4.7 Interference

Interference [15] is a permanent interactive installation by Danish company Kollision in a tunnel for pedestrians and bicycles under an avenue in Kolding, Denmark. Opened in 2014, it consists of colourful light panels installed along the walls of the underpass, equipped with a series of trip-wire infrared sensors. Passers-by are detected as they move, with the light panels in front of them turning on in response. The light effects follow the direction of their movement, progressively illuminating the way ahead. If many people occupy the tunnel at the same time, the same interaction pattern is applied simultaneously to each of them, resulting in a fleeting social interaction that amplifies their awareness of each other: if people are moving in the same direction, whoever is in front is “warned” about the presence of those coming behind; if they move in opposite directions, their light beams collide in anticipation of their physical encounter. The social experience of sharing the public space is thus highlighted by the mutual interference of the light patterns created by each individual.

4.8 The Climate on the Wall

The Climate on the Wall [6] was an installation conceived to be in operation during the climate conference Beyond Kyoto, in Aarhus, Denmark, in 2009. It consisted in a large projection on a wall of the Ridehuset, a prominent historical building in a busy intersection in the centre of the city. The projection would show falling speech bubbles containing terms employed in the ongoing climate debate. Dedicated software used cameras installed along the façade to detect people going through the sidewalk. They could then ‘grab’ a word, carry it along and reposition it on the wall, thus contributing to the climate debate by forming phrases expressing their views. The interface was perceived as playful and occasionally passers-by would stop by and interact with the projections. More often then not, however, implicit or inadvertent interaction [18] would take place: people would unwillingly engage in interaction just by walking passed the wall, with some noticing they had carried a speech bubble along with them only after it had already happened. The lack of proper grammar seemed to hinder the formation of coherent sentences but, most importantly, the best spot to observe the evolution of graphics on the screen was from the opposite side of the street – i.e., by people watching the façade, not those actually interacting with it. Despite the general awareness about climate debate sparked by the installation, the social conversation via the media façade seemed to have felt short of expected [6].

4.9 Chromapollination

Chromapollination [12] was a responsive ambient sculpture developed as an urban activation study by the Design Lab, University of Sydney, Australia, for the 2012 edition of the local Vivid Sydney festival. The public space was an underutilised thoroughfare beneath an elevated highway in the centre of the city. The installation consisted of three large-scale sculptures of beds of dandelion flowers, embedded with fibre optics and ultrasound motion detectors. Above, a triangular LED ceiling seamlessly connected the sculptures. As people walk through and around them, their movement was detected, creating ‘digital wind’ – a stream of light flowing overhead from one flower bed to another, carrying the colour of the source dandelion to be mixed with that of the target, which would then be ‘pollinated’ by changing colour. Responses to the detected movement were, by design, subdued and delayed, in order to avoid people interacting for too long and cluttering the public thoroughfare. Most commonly, couples or small groups of friends and families would stop for close appreciation and photos of themselves near the work, while ‘singles’ would usually observe and take photos or videos from a relative distance.

From the juxtaposition of the key features of the nine installations summarised in Table 1, certain recurrent design strategies and resulting forms of social interaction can be identified. For example, most urban interventions designed for performance (e.g. Body Movies, Aarhus By Light or Solstice LAMP) are placed in urban spaces that can be characterized as plazas, while responsive ambient media architecture (such as Interference or Chromapollination) has been often used as a way to enhance the experience of thoroughfares. Likewise, full body interactions are clearly more effective to entice explicit and inadvertent interaction as a way to facilitate collective participation, leveraging from established norms of social behaviour in public spaces [18]. Allotted interfaces making use of full body interaction, in particular, reduce the social pressure and risk of embarrassment by allowing a larger number of people to perform simultaneously. In that sense, they can be considered as a less disrupting intervention on the dynamics of the social space, while still supporting both direct interaction among people and indirect interaction via triangulation [19] through the interfaces. Consoles (as used in SCSD) and mobile devices (as in iRiS) both constitute TUIs functioning as mediators to the interaction with the media architecture.

5.1 Social Interaction Modes

By comparing the installations and analysing how the social interaction unfolded around them, we propose that six modes of social interaction are present in the cases: (1) appreciation, (2) self-expression; (3) playfulness; (4) collective narratives; (5) triangulation; and (6) negotiation of space. These modes can be further grouped into three broader categories: appreciation and self-expression into spectacle; playfulness and collective narratives into creativity; triangulation and negotiation of space into conversation. Table 2 displays how each mode maps to the media architecture installations considered in our analysis. We discuss each of these categories below, and will later reference them to derive social interaction design patterns.

Spectacle.

Creating spectacle is inherent to media architecture, especially when it is interactive. Just like street performers, media architecture disrupts the normal flow of urban activities, turning passers-by into spectators and, depending on its nature, active performers. Accordingly, we can consider two aspects of the spectacle offered by media architecture: appreciation (i.e. the passive spectatorship of the urban intervention) and self-expression (active interaction via performance or casting of votes, for example). As Table 2 indicates, although all works analysed prompted public appreciation, not all of them led to self-expression. Noticeability of the interaction zones is critical for active participation, particularly in regards to the spatial ratio between those zones and the media architecture. As pointed out by Fischer and Hornecker [7], for media façades the display space – i.e. the space around the façade where people can see it – is often much larger than the interactive zones, making it difficult for distant observers to perceive the interactive nature of the work. Installations such as Body Movies, Aarhus By Light and MyPosition, for example, successfully addressed this problem by positioning the interactive zones in prominent, wide and therefore very visible areas right in front of their façade. The interactive console used in SCSD, however, was much harder to be spotted amidst the busy urban environment of the large avenue in Sao Paulo. Equally important to making the interactive zone visible to potential participants, however, is to position it so that the feedback provided by the architecture can clearly communicate its intention: direct and uniquely addressed to an individual, if to convey agency and identity; or delayed and defused across the space if to communicate ambient response. While MyPosition, for example, was highly successful in its use of feedback to communicate interactivity to casual passers-by, guiding them through the various steps required for voting via the interface, The Climate on the Wall’s feedback was more visible to spectators on the other side of the street than to the actual (and often unaware) participants near the façade.

Embodied allotted interfaces, such as Solstice LAMP and Interference, offer a neat solution for both problems: they are wide enough to be noticed from a distance and, for having the interactive zone largely coinciding with the media architecture itself, they make the effects of participation obvious to participants. Moreover, at a closer distance, visibility is equally given to the environment and the people within it, highlighting their roles as agents over the immediate urban space. Such increased tangibility results in an additional benefit: learnability by demonstration. As pointed out by Mueller et al. [18], many people notice interactivity by observing other people interacting. Allotted interfaces, being able to accommodate a large number of simultaneous participants, are therefore highly effective in promoting social interaction, creating a positive feedback loop whereby many people interacting make even more people not only aware of the interaction but also able to readily join in. The shared interface also helps to reduce the fear of public embarrassment [21] that can potentially hinder participation by people less comfortable to perform in public.

Creativity.

While media architecture can be subject to appreciation and lead to individual self-expression, it is not necessarily playful or able to accommodate simultaneous interaction by multiple agents. SCSD and MyPosition are examples of works intentionally designed to be predominantly functional (albeit enjoyable) and restricted to individual interaction. Interactive public art and urban activation projects (such as Aarhus By Light, Solstice LAMP, Interference and Body Movies), on the other hand, often rely on playfulness to subvert ordinary street practices. As Huizinga [14] pointed out, play is a cultural construct perceived as not “real” or “ordinary” life; it refers to activities that are intrinsically motivated, situated outside of everyday life and with no direct benefit or goal. By enabling a playful situation, interactive media architecture can achieve a relaxation of the established social norms, easing the constraints for social encounters to emerge. When the interface also allows for simultaneous interaction by multiple people, collective narratives may emerge out of the constraints and suspension of disbelief posed by the responsive environment, akin to what is observed in alternative or mixed reality games [20].

Conversation.

As Goffman [10] pointed out, when faced with an unfamiliar social setting, people scan each other for clues about what the appropriate behaviour is. In the case of interactive public environments, unless the rules are explicitly stated via call to actions [22] they often end up emerging on the spot and collectively as people try to make sense of an otherwise unusual situation. This process of sensemaking often emerges among strangers as a consequence of triangulation [19], a social situation where conversation is started out of mutual interest in the content itself displayed by the interface. In that sense, allotted interfaces may have different results depending on how much visible participants are to each other: iRiS, although playful and enabling self-expression, was occasionally perceived as frustrating since participants could not always see physically who they were interacting with on the screen. In Solstice LAMP and Interference, on the other hand, participants were always on sight of each other and therefore negotiation of space unfolded according to established social norms. Technology, in those cases, supported and promoted social interaction through the mutual awareness of other people in the space.

5.2 Social Interaction Design Patterns

When compared with the interaction strategies and spatial layouts of the selected media architecture installations, the social interaction modes discussed above indicate a series of recurring strategies employed by the designers. We propose that those strategies can be characterised as social interaction design patterns for eliciting specific forms of crowd behaviour and, as a consequence, the social identity of the public space. In that sense, our proposed patterns are conceptually closely related to Alexander’s pattern language for architecture [1] than to the software design patterns proposed by the GoF [8], consisting in reusable strategies to use digital media for designing the interaction among people and between them and their surrounding built environment. From Table 2, we derived the patterns by grouping either recurring layout factors (i.e. spatial layout, interaction strategy and type of feedback) or social interaction modes. We then named each pattern by using metaphors to familiar concepts that operate in a similar fashion, with corresponding social reactions.

Grouping recurring layout factors (Table 3, left) produced three patterns: (a) Shadow Playing; (b) Remote Control; and (c) Smooth Operator. Grouping the social interaction modes (Table 3, right) produced further four patterns: (a) Soapbox; (b) Amusement Park; (c) Swarm; and (d) Automatic Gate. Figure 1 summarises the proposed patterns, which can be construed as design strategies for prompting specific forms of social interaction. Below, we discuss each pattern and its effects on social interaction.

Social interaction design patterns.

Full size image

Shadow Playing.

This pattern captures the recurrent combination of full-body interaction and direct feedback (Table 3, left). It prescribes the use of full body interaction and immediate visual feedback to create a 1-to-1 relationship between individuals and their graphical representation in the media architecture (as in Body Movies, Aarhus By Light and Solstice LAMP). Previous studies have found that shadow playing is an effective way to communicate interactivity to passers-by via inadvertent interaction, with mirror images, in particular, being more effective than silhouettes and avatars [18]. By minimizing noise and latency on the communication, the interface enables a high sense of agency and identity among participants, who feel like they are controlling the media content directly. Social interaction therefore unfolds both in the physical and digital environments simultaneously, via triangulation (and possibly negotiation of space) between individuals and their “projected selves”. Apart from spectacle, this pattern often entails playfulness and enables the emergence of collective narratives, with multiple rows of participants tending to form around the interactive zones [18]. For that reason, this pattern typically requires a plaza spatial layout, so that multiple participants can be accommodated, allowed to move freely and have good visibility of their digital representations on the media architecture.

Remote Control.

In this pattern, a tangible user interface (TUI) is employed as a mediator between the public and the media architecture [2], usually in a plaza spatial layout (for the same reasons as Shadow Playing). Two opposite design strategies have traditionally been adopted: centralised remote controls (such as the console used in SCSD) or distributed portable devices (often mobile devices, as in iRiS). In both cases, discoverability about the interactive aspects of the architecture becomes a recurrent challenge: how to easily communicate to the public they can use smartphones to interact with the content on the façade, or indicate the existence and location of a mediating console? Both scenarios also pose trade-offs for the design. Interaction via mobile devices allows simultaneous participation, but negotiating time on the interface may be problematic; moreover, it is usually pre-empted by the system itself rather than a result of a tacit social agreement between the participants – who, by the way, often cannot see each other. Interaction via a centralised TUI, on the other hand, solves such an accessibility problem, however eliminates the possibility of more elaborate social interaction to unfold via triangulation with the media architecture.

Smooth Operator.

This pattern describes the delayed feedback to full-body tracking unfolding in a thoroughfare, responding to movements of passers-by in a covert way (Table 3, left). While affected by the presence of passers-by, a smooth operator does not respond direct to them, and therefore is perceived as an independent entity, not controlled by the public (or at least not entirely). Typically, its responses to inputs from the public is delayed and, often, not in their field of view (e.g. the landing effect [18] verified in Chromapollination and The Climate on the Wall). Smooth operators are normally designed for thoroughfares and regions of intense pedestrian traffic, raising awareness about the public urban space yet without compromising the flow of people through it.

Soapbox.

As shown in Table 3, right, this pattern represents installations enticing predominantly individual expression and its appreciation, but with little playfulness, collective narratives or direct social conversation. Its goal is to offer an opportunistic encounter whereby citizens can informally express their opinions about a local topic without too much disruption to their daily activities. Public consultation can therefore happen in a lightweight yet regular basis, providing local government with invaluable feedback without excessive interference on the social dynamics of the community. As such, soapboxes tend to be designed for quick but reliable individual interaction, and seem to be more effective when ludic elements are downplayed – in other words, they benefit from a lack of playfulness, increasing the focus on their actual functionality. Yet, as in any interface, user experience should be taken into account. For example, according to the designers of MyPosition the interface was deliberately designed to create a good balance between playful interaction and meaningful participation. The goal was to enable incidental interaction leading to an enjoyable voting experience, yet avoiding the inadvertent interaction observed in The Climate on the Wall [22].

Amusement Park.

The amusement park is an urban media architecture installation that promotes a wide array of social interaction, reaching out to all three proposed domains: spectacle, creativity and conversation (Table 3, right). On those settings, there is not only individual self-expression and its appreciation by bystanders, but also collective play leading to co-created narratives, mediated via triangulation with the media architecture and, occasionally (as it is the case of embodied allotted interfaces, such as Solstice LAMP), negotiation of the physical space during the experience. It is characterized for being a disruptive urban intervention that captures the attention of passers-by and turns some of them into performers, while others participate passively as spectators. As a playful environment, it entices suspension of disbelief and, consequently, subversion of pre-established social norms, with new ones defined on the spot as a result of the social interaction itself. Performative and allotted interfaces are the most effective choices for designing amusement parks.

Swarm.

This pattern refers to environments where people interact via triangulation with the architecture, negotiating the physical space in the process (Table 3, right). A swarm is an allotted interface built upon full body interaction. Generally deployed to plazas, it consists in an interface large enough to accommodate many simultaneous participants, each involved in embodied interaction with their own section of the interface. Feedback is such that participants are continuously aware of both their own representation on the interface and those of the peers in their immediate vicinity. Consequently, local actions of one individual may imply in reactions by their immediate neighbours, which will then somehow impact their own neighbours, in a process resembling a swarm of insects or school of fishes. Works like Solstice LAMP and Interference are good examples of swarms, successfully assisting the autonomous negotiation of space by participants by making use of technology to create ‘body gloss’ [9] (i.e. the emphasis and exaggeration of bodily presence). In addition, the social relationships among participants may also be highlighted by the feedback they receive. For instance, Solstice LAMP’s feedback mechanism of merging halos when people got close enough to each other indicated the transition between a state where people were perceived by the environment as separate individuals (or ‘singles’, as defined by Goffman [9]) to a scenario where they could be socially characterized as a group (in a ‘with’). As observed, people would often adjust their behaviour as a result of the new social configuration.

Automatic Gate.

As indicated in Table 3, right, this pattern describes environments with varying levels of playful feedback and appreciation yet no room for self-expression or collective narratives. An automatic gate is a media architecture installation implemented in thoroughfares and areas of high pedestrian activity, which people attend to with peripheral attention. It is characterised by being strongly blended into the built environment, to the point of being almost integrated to it. It responds to pedestrians in a subtle, seamless and non-disruptive way, changing the environment appearance either in front or behind them. Interference and Chromapollination – enhancing pedestrianised areas without structurally altering them – constitute good examples. In other cases, however, such a pattern emerges unintentionally: for example, when interaction fails to be understood or noticed, the work may be inadvertently interpreted as ambient, as was the case with The Climate on the Wall.

Successful media architecture combines insights into technical, architectural and social issues. While the field is still relatively new, patterns in how these three aspects can be combined to elicit specific types of behaviour and use are emerging. Our focus has been to understand how different technical and architectural configurations – here labelled types of interfaces and spatial layouts – can lead to specific modes of social interaction, and whether there are recurring strategies for bringing about these social modalities. We find that the field, albeit still in the making, is mature enough to identify recurring design strategies for designing media architecture and analyse the effect they have had on the social aspects in the locations in which they were introduced. Through an analysis of nine media architecture installations that are representative of the field, we have thus identified six distinct modes of interaction, ranging from appreciation to creative expression and dialogue. In continuation, we have identified seven recurring design patterns, which have been employed as strategies to bring about specific modes of social interaction. While these contributions build on analyses of works selected to represent the scope and diversity of media architecture thus far, we do not consider the list of modes and design patterns of social interaction to be exhaustive. A clear limitation lies in our selection of works, which we have limited in order to provide enough details to make the function and purpose of the installations clear; it is plausible that a different or more expansive selection of works could lead to further examples of modes and patterns. Indeed, we would welcome additions to this work, and we expect new modes and patterns to emerge as media architecture continues to evolve. Another limitation in our methodology is that since we have strived to include installations that show the diversity of media architecture, the sample size is not large enough to indicate if specific design patterns are statistically more prevalent or successful. However, this initial identification of social modes and design patterns may lay the ground for more detailed analyses of strategies for influencing and developing modes of social interaction. In addition, we hope that the identification of design patterns can be valuable for interaction designers of future media architecture installations, who strive to bring about specific modes of social interaction.