What About Document Folding? User Impressions and a Design Approach

There are many possibilities for creating documents with folds in the age of desktop publishing, and a large selection of software is available for this purpose. However, most software focuses on organizing content into a conventional sequence of pages rather than produce foldable documents composed by several smaller panels joined in arbitrary positions. This hampers the creation of documents that are meant to be folded, and compels the user to design the document before editing its content by, for example, measuring each part bounded by the folds and adding up the total area of the document. After the document size is set, the content is edited as a single page, even though it has logical divisions (cover, back, middle, back-cover, etc.). Systems like Scribus^{Footnote 1} offer features to add guides that will not appear on the final document separating the content inside the document. This may help the user but adds nothing to the ease of changing the content of a single part later on. Moreover, due to the two dimensional nature of the screen it may be hard for the designers to understand the dynamics of the document and how the folds will behave, causing them to resort to techniques such as paper prototyping [5].

To understand the process of creating brochures, we performed a user study with design experts. They reported the routine of art creation related to foldable documents. Designer experts need to go through a variety of steps during the brochure creation process to achieve a good visualization and interaction before printing the final version of the document.

In this paper, we propose an interactive visualization approach for helping designers and eventual brochure creators to build, visualize and interact with foldable documents in a 3D environment. Users can create the document in two distinct stages: first, each brochure panel is designed separately; after this panels are assembled into one document. It gives users freedom to change the form of the document without worrying about its content and also they can validate the design by interacting with the document in a three dimensional view. A prototype was developed from the proposed approach to present the results regarding the visualization and interaction possibilities.

This paper is organized as follows: we start presenting the information gathered from the interviews with design experts and the existing solutions related to foldable documents. Next, we present details of the approach proposed to help users interact and visualize documents as brochures. Thereafter, we describe the prototype created and the results related to the possibilities of visualization and interaction with foldable documents.

There are many approaches related to document engineering centered in the document layout and content organization rather than the document shape and dynamics. From existing document design software such as InDesign^{Footnote 2}, Photoshop^{Footnote 3} and CorelDraw^{Footnote 4}, we see that they provide few features to support the design of foldable documents.

Generally, design experts adopt techniques such as paper prototyping to visualize the results of brochure creation, to see how it will be folded and also to interact with the document front-and-back sides, as the visualization that many tools offer is not enough to represent it. Paper prototyping [5] has been used to model a wide variety of design ideas and to help designers and users to interact with elements from an initial project.

Thus, to learn more about brochure creation and the designer’s methods, we performed a user study as described in the next section.

2.1 User Study

To gather information about designer needs during the brochure creation, we performed a set of interviews with 12 designers. The interviews follow a set of 12 questions from designer profile to experience with brochure creation.

From the participants, 7 were male and 5 female. The average time of experience in brochure creation reported by each participant is about 7 years, and 10 of them have a degree related to either Arts & Design or Marketing as we can see on Table 1.

We asked each participant about the tools they use to design brochures and the most mentioned software were Illustrator (cited by 8), InDesign (cited by 7), followed by Photoshop and Corel Draw (used by 6 and 7 respectively). They clarified that they need a combination of software like Photoshop and others to create the art of panels and thereafter merge it using Illustrator, InDesign, etc. According to them, there are no standards for brochure creation and it depends essentially on designer habits or needs. We asked participants to described the steps to create a foldable document, focusing on the layout process:

P6: “First we need to define the layout and the amount of pages it will have. We create the idea of the layout and we interact with it through a paper prototype to imagine the document composition and also to start thinking about the panels organization”.

Designers reported that they need to define the sequence of the foldable document to organize the pages and the content:

P4: “We need to distribute the content along the panels and give it a logical sequence according to the layout we planed to create”.

P12: “If we define the number of creases and there is not enough content to fill the pages, we need to create the document all over again”.

Most participants (10) comment that the brochure creation process is a hard and time consuming task. Also, participants said that one of the critical steps during brochure creation is related to the visualization of the layout structure and pagination of the front-and-back sides. To understand and visualize the sequences of the panels and how the pages will connect to each other, they usually print all brochure’s faces and build a mockup, also known as paper prototype:

P6: “For presenting to customers an idea or the work in progress, we usually take a photo from a mockup to show them the creases it will have and the document front-and-back sides”.

They commented about the connection among pages that some brochure layouts demands:

P2: “The connection among the pages can be a critical stage in the brochure creation, because it demands time”.

P6: “Sometimes the creases we create simply do not work because the amount of content we need to distribute among the pages is not enough. Also, there is no mirroring to visualize the back of each page”.

Yet, regarding the pages connection, participant P12 add that: “Once the connections among the pages change, it will increase the final product cost.

Designers use paper prototyping as mockups for evaluating the final look of a brochure, its dynamics, to visualize its panels, pagination and the connection between its folds. This is a practice cited by all designers, because the tools they use do not have a simple approach to accomplish this task. The lack of visualization of the brochure front-and-back sides was the major complaint regarding the tools designer are used.

Paper prototypes are known by their low-fidelity to the final project and it can be frustrating for customers to imagine how a product will be when finalized. Furthermore, the project cost may increase quickly as the prototypes are printed over and over.

Therefore, the design of brochure documents, could benefit from 3D digital prototyping that can represent a document in high-fidelity [7] and also be fully interactive with panels and faces.

The data collected from the interviews helped us to understand the challenges faced by the designers during the brochure creation process. They also contributed to motivate the development of the proposed approach.

2.2 Existing Solutions

Little has been said about foldable documents in the literature: Chiu [2] proposed an approach to fold digital documents on multi-touch devices based on a focus-plus-context principle, in which the user folds the document to focus on a region they want to read. However, this approach is concerned only with document visualization and not with the design of foldable documents.

Also, Khalilbeigi et al. [4] perform experiments to explore interaction techniques for manipulating digital content with folds through a novel device concept that features double-sided displays. They found that mobile handheld devices have great potential to be explored in this field. However, they did not explore folding along different axes and they also used predefined hinges, limiting the paper configuration.

Straightforward solution is the use of templates. The design experts we interviewed do not usually take advantage of it, but it might be a fast way to create a document for the casual designer. Otherwise, the fact that the template is ready, waiting for content, may turn into a disadvantage if the user wants a personalized piece.

In the next section, we present our approach to build, visualize and interact with document with folds based on the results of the interviews reported in this section and the lack of related works with this goal.

The main ideas explained in this section may be divided in two parts. The first involves the concept of panels and explains how designers are used to build documents with folds. The second corresponds to a method to visualize and interact with the built document making full use of three dimensions. These two parts are described in the next subsections.

3.1 Working with Panels

In our approach, panels are page divisions that hold content in a document. When a sheet is folded in half, four panels are created, two at each side of the sheet, with the fold between them. When the paper is folded multiple times, some of the panels may overlap. Figure 1 illustrates panel overlapping in a tri-fold brochure. The dashed lines represent each fold in the paper and the middle panel is created from both left and right folds.

A tri-fold brochure: dashed lines represent folds and the same document in perspective.

Full size image

Thus, the first part of our approach is an interactive method to shape the document. We assume that the panels are already designed and we provide an environment in which the document arrangement is a matter of dragging the panels around and connecting them.

Users select panels from a set of existing ones and drag them to an editing area. The creation of a fold corresponds to the action of joining two panels by their edges. As Fig. 2 illustrates, if the bottom of panel 1 is joined with the top of panel 2, they form a larger element and their common border is said to be a fold.

Example of a horizontal fold.

Full size image

Either the front or the back faces of the panels will be facing each other when the new element is folded. The same idea applies if two panels are joined by their sides, but instead of a fold on the horizontal direction, a fold on the vertical direction will be formed. When two panels are joined they form a new panel that may be joined to other panels and to which other panels may be joined to. If panels have different sizes, the one that was dragged is resized to match the edge of the other panel, preserving its aspect ratio as shown in Fig. 3.

Panels matching edges with different sizes.

Full size image

Fig. 4 shows step by step how Fig. 3(b) is created. In Fig. 4(a), there are three single panels, represented internally as three single nodes. On Fig. 4(b) panels 1 and 2 are joined, creating a horizontal fold. Their nodes, thus, share the same parent node (H) that represents the horizontal fold. Lastly, on Fig. 4(c), panel 3 is joined to the composed panel to create a vertical fold, thus, node 3 shares the same parent (V) as the horizontal fold node, which is a vertical fold node.

Steps to build a composed panel. On the left, how the composition is seen by the user, on the right how the composition is represented internally.

Full size image

During the process, a second view shows the back of the document as if the document was held with its back facing a mirror, as in Figure 5. That view provides information on how the document will look when flipped and gives a better sense of the overall result. It is also possible to change the content of any panel (either front or back sides), even if that panel is grouped with other panels or groups.

3.2 3D Interaction and Visualization

Once panels are joined together, we provide a way to interact with the document for previewing without the need to print it. We use a 2D view of the document with a crease pattern on it and a 3D view that will show all folds that are performed. The crease pattern is obtained from the panel adjacencies and is used as a map to show the places where the document may be folded. In this context, a panel adjacency is turned into a crease only when its horizontal length is equal to the document width or its vertical length is equal to the document height. Figure 6 depicts an example in which three panels are joined together. Panel 1 is vertically adjacent to panel 2 and both panel 1 and 2 are horizontally adjacent to panel 3. Because the adjacency A between panels 1 and 2 does not have the same length as the document height, it is not turned into a crease whereas B is turned into a crease because it has the same length as the width of the document.

First view (a) and second view (b) showing the front and the back of a document respectively.

Full size image

The creases are subject to restrictions depending on the current configuration of the 3D copy (that is how the document is currently folded). We define a crease as available when it is possible to fold the document through it and as unavailable when the action is impossible. Figure 7 illustrates a document being folded in half two times and how the crease pattern responds to the document configuration. First the vertical crease is folded, making the horizontal crease unavailable until its completion (frames 1 through 3). Then the horizontal crease is folded, making the vertical crease unavailable (frame 4). In this last case the vertical fold would still be unavailable even when the horizontal fold is completed because the former is blocked by the latter. A crease is said to be available only if all of its parts are collinear and it is not blocked by any other folded crease.

The document may be folded through B but not through A.

Full size image

A crease pattern and the document (1). When the vertical crease is half folded (2), the horizontal crease becomes unavailable. Once it is completed, the horizontal crease becomes available (3), so the paper may be folded horizontally, but when doing it, the vertical crease is blocked and turns unavailable (4). Crease status is shown as green when available and red when unavailable.

Full size image

This document model resembles the work of Balkcom [1] on origami folding. The author models paper such that its creases are hinges and uncreased parts are rigid bodies. Comparing to our approach, panels adjacencies may be thought of as the hinges and the panels as the rigid bodies. Since our model simulates paper, it should also respect its properties such as upon on folding a paper, its distances should be preserved, and it should not intersect, although it is allowed to touch [3, 6]. Due to the hard task of modeling paper (it remains an open problem [6]), our implementation is restricted to simple configurations on the folded document. Folding the document multiple times, specially where the document is already folded, is a limitation because it would require collision detection and resolution, techniques that would require extra computational complexity.

We implemented a prototype in Java, using Processing^{Footnote 5} as the rendering engine. As shown in the Fig. 8 the prototype consists of a screen divided in two menus, one at the top, the other at the bottom, and a working area at the center. The top menu in Fig. 8 (a) has a button to export the final work to a PDF file, and the bottom menu Fig. 8 (b) is used to perform actions that aid the document editing.

Screenshot of the prototype.

Full size image

The working area displays the panels used to build the final document Fig. 8 (c) and two other areas labeled as front Fig. 8 (d) and back Fig. 8 (e). The front area is where the document is built and is called the editing area. The back area is called the mirror area and shows how the content in the editing area would look like if it was held with its back facing a mirror. A cursor is shown over a page in this area if the mouse pointer is over the same page in the editing area. This gives a reference for users to quickly remember the back of the page in context. There is also a button (Fig. 8(f)) to flip the areas. When pressed, this button will rotate the editing and the mirror area revealing their backs, with the same effect of flipping a paper over a table. This allows the users to edit the document looking directly at its back while the front is shown in the mirror area. The 3D button (Fig. 8(g)) changes the view to the 3D perspective, where the users can visualize how the document will be and may interact with the document folding and unfolding it as they wish.

To start editing the document, users drag one panel from the list of panels (Figure 8(c)) to the editing area. Once there, panels are dragged around, joined by the edges or selected for flipping or removing. A fold is formed by dragging a panel close to another panel’s edge, and once the edges are close enough, the panel being dragged snaps to the other one, automatically aligning and resizing if necessary and preserving its aspect ratio, becoming a compound element that can also be dragged, deleted, flipped and snapped. Users are also allowed to ungroup panels by selecting a group and clicking on the break button and to change the content of a panel at any moment – even if it is inside of a group – by releasing a panel with the new content over the panel with the content to be changed. The panels are never out of sight, if the user drags one panel too far from another, the editing area automatically zooms out to fit every panel within its bounds.

Once users want to manipulate the final product, they may select a group of panels and switch to the 3D mode by pressing the 3D button (Fig. 8 (g)). The users may interact with only one group of panels at a time, and the program will switch to the 3D mode only if a group is selected. The 3D mode is composed by two copies of the document, one is a flat version that holds the crease pattern as explained in Sect. 3 and the other is the 3D version that shows the folds the users perform on the document. The interaction is exclusively via the crease pattern, thus, every fold the users want to perform on the 3D copy must be issued from the crease pattern. As an example, Fig. 9 shows a configuration where every internal edge intersects with at least another edge. This means that the document can be folded one time horizontally, as the figure shows, and many times vertically. As one can expect from either figures, it should be possible to fold the document horizontally and then vertically or vice versa, however this is still a limitation of the prototype. This is, an implementation issue due to the problem of simulating paper digitally and not in the approach of joining panels to represent folds.

Complex configuration folded vertically many times and horizontally.

Full size image

In this paper we presented a new interactive visualization approach for building documents with folds (a subject seldom discussed in the literature) that consists in dragging panels adjacent to each other to compose the document and then interact with it in a three dimensional view. The main difference between the works cited here and our approach is that we allow users to interact with the document by changing its content and exploring distinct types of folds avoiding the need of paper prototyping. This is a new method to interact with documents that are being designed and enhances its visualization during the development.

We conducted interviews with design experts and observed that paper prototyping is a technique widely used by them to visualize and interact with documents. However, this technique is labor intensive and time consuming when it is necessary to generate too many document versions. Thus, designers require approaches to ease the interaction and visualization of brochures. Based on their answers reported in such interviews, we proposed an approach to work with panels during the creation of foldable documents and a method to visualize and interact with the built document. Also, we developed a prototype that allows for users the panels visualization during the content organization process and interact with the document creases.

As next steps, we intend to present the results from a second phase of interviews with designer experts. We have presented to designers the developed prototype as a video to show the possibilities of interaction with the folds. They have agreed that it is helpful for visualization purposes and the 3D approach does can decrease the manual work, save time and resources, avoiding the creation of paper prototypes which are time consuming and low-fidelity for costumers visualization.

We plan to embed this approach as a plug-in into existing document design systems, such as Illustrator and InDesign to evaluate its usability. Also, more investigation is needed to solve the problem of folding the paper over itself many times respecting the physical restrictions a sheet of paper is subject to, which is challenging due to the nature of paper.