Multimedia: from information source to components of transformational games

This summary overviews a keynote talk that the author is giving at the WebMedia conference. Christel will discuss his journey with multimedia research over the past six years, taking him from Carnegie Mellon University's Computer Science Department into the Entertainment Technology Center (ETC). The story begins with the use of speech recognition, image processing, and language technologies to automatically process large video corpora. Such processing facilitates more efficient retrieval. As demonstrated by top scores in the international TRECVID benchmarking forum, Christel's CMU Informedia research group has experienced success in finding relevant video shots quickly from large masses of material. The emphasis is on leveraging the intelligence of a human user in the interactive retrieval loop, with lessons shared in visual analytics papers. That work drew the attention of oral historians, who amassed large quantities of video stories that were not easily accessible. The value of synchronized metadata to open up these collections for the web audience illustrates the power of multimedia processing to help organize and present cultural repositories. As the user community appreciated the layers of meaning within these often riveting stories, Christel was drawn to the power of multimedia elements in entertainment technologies, including games. Transformational games are designed to change the player in some way, such as improving health habits, changing attitudes, or providing education. The ETC provides multidisciplinary teams of graduate students with skills in visual arts, design, sound, programming, and production the opportunity to create such games in the course of a semester-long project. Christel will briefly overview this development process, and then demonstrate some of the ETC-produced games, highlighting the role of multimedia elements within them and the promise of the work to positively affect the game player.