Storytelling with Animated Interactive Objects in Real-time 3D Maps

Download

Full text (PDF, 23.82Mb)

Open access

Author

Schnürer, Raimund

Date

2023

Type

Doctoral Thesis

ETH Bibliography

yes

Altmetrics

Download

Full text (PDF, 23.82Mb)

Rights / license

In Copyright - Non-Commercial Use Permitted

Abstract

Digital story maps present geographic information embedded in a narrative structure and often supplemented by multimedia elements. The currently predominant extrinsic approach, however, impedes following the storyline since the reader's attention is split between maps and additional illustrative content in the user interface. Incoherent representations between abstract map elements and realistic multimedia elements can be seen as another shortcoming. To close these gaps, an intrinsically oriented approach, which is inspired by historical and contemporary pictorial maps, is proposed in this dissertation: Figurative objects, which act as storytellers, are introduced into the map. By spatially anchoring the complementary entities within 3D maps in particular, the connection between the cartographic model and geographic reality will be strengthened. The overall goal of this work is to automatically turn static objects from prevalent 2D pictorial maps into animated objects for interactive 3D story maps. Artificial neural networks, primarily convolutional neural networks (CNNs), are applied in a sequence of discriminative and generative tasks to achieve the goal. For each task, data is prepared to train the networks in a supervised manner. Firstly, pictorial maps are identified from publicly available images on the internet by CNNs for classification since metadata of the images is not always present or reliable. Different strategies are investigated to input the images into the CNNs. Secondly, bounding boxes of objects on pictorial maps are detected using the example of sailing ships. Although map descriptions may include the occurrences of objects, their positions and sizes are usually unknown. To determine these two measures, CNNs for object detection are examined while modifying their hyperparameters. Thirdly, silhouettes of pictorial objects are recognised, exemplified by human figures. As the manual preparation of training data would be too labour-intensive, combinations of figurative and realistic entities are evaluated. Following, body parts and pose points are extracted from the silhouettes of the figures, which is a prerequisite for skeletal animations and the insertion of speech bubbles at head positions. CNNs with varying numbers of skip connections are compared for this task. Lastly, 3D figures are derived from their 2D counterparts based on the outputs of the previous step by a series of networks. This significantly facilitates and accelerates the 3D modelling process. The figures are represented by implicit surfaces, which are advantageous for curved surfaces, and rendered in real time by a ray tracing algorithm. Quantitative metrics, such as accuracy and rendering speed, and qualitative results are reported for each task. The inferred 3D pictorial objects may guide readers through the map while providing background information and offering interaction possibilities like quizzes. This is especially suitable for atlases, touristic or educative applications, even in augmented and virtual reality. Animated interactive objects are intended to engage map readers through an immersive storytelling approach, increase their map literacy and frequency of use, and create long-lasting memories. Show more

Permanent link

https://doi.org/10.3929/ethz-b-000663385

Publication status

published

External links

Search print copy at ETH Library

Contributors

Examiner: Hurni, Lorenz

Examiner: Bandrova, Temenoujka
Examiner: Öztireli, A. Cengiz

Publisher

ETH Zurich

Subject

Pictorial maps; Machine learning; 3D cartography; Storytelling; Computer graphics

Organisational unit

03466 - Hurni, Lorenz / Hurni, Lorenz

Funding

ETH-11 17-1 - Storytelling with Animated Interactive Objects in Real-time 3D Maps (ETHZ)

Related publications and datasets

Has part: http://hdl.handle.net/20.500.11850/444044

Has part: https://doi.org/10.3929/ethz-b-000501723

Has part: https://doi.org/10.3929/ethz-b-000621884

Show all metadata

ETH Bibliography

yes

Altmetrics

Research Collection

Search

Storytelling with Animated Interactive Objects in Real-time 3D Maps Mendeley CSV RIS BibTeX

Storytelling with Animated Interactive Objects in Real-time 3D Maps

Mendeley

CSV

RIS

BibTeX