DSHR's Blog

Arguably, video games have become the most important entertainment medium. 2022 US video game revenue topped $85B, compared with global movie industry revenue of $76.7B. Game history is thus an essential resource for scholars of culture, but the industry's copyright paranoia means they have little access to it.

Salvador Table 1

In 87% Missing: the Disappearance of Classic Video Games Kelsey Lewin of the Video Game History Foundation describes their recent study in cooperation with the Software Preservation Network, published by Phil Salvador as Survey of the Video Game Reissue Market in the United States. The report's abstract doesn't mince words:

Only 13 percent of classic video games published in the United States are currently in release (n = 1500, ±2.5%, 95% CI). These low numbers are consistent across platform ecosystems and time periods. Troublingly, the reissue rate drops below 3 percent for games released prior to 1985—the foundational era of video games—indicating that the interests of the marketplace may not align with the needs of video game researchers. Our experiences gathering data for this study suggest that these problems will intensify over time due to a low diversity of reissue sources and the long-term volatility of digital game storefronts.

Below the fold I discuss some of the details.

Two days before Christmas Ilya Kreymer posted Announcing the New OldWeb.today. The old oldweb.today was released five years ago, and Ilya described the details in a guest post here. It was an important step forward in replaying preserved Web content because users could view the old Web content as it would have been rendered at the time it was published, not as rendered in a modern browser. I showed an example of the difference this made in The Internet is for Cats.

Below the fold, I look at why the new oldweb.today is an improvement on the old version, which is still available at classic.oldweb.today

Jeff Rothenberg's seminal 1995 Ensuring the Longevity of Digital Documents focused on the threat of the format in which the documents were encoded becoming obsolete, and rendering its content inaccessible. This was understandable, it was a common experience in the preceeding decades. Rothenberg described two different approaches to the problem, migrating the document's content from the doomed format to a less doomed one, and emulating the software that accessed the document in a current environment.

The Web has dominated digital content since 1995, and in the Web world formats go obsolete very slowly, if at all, because they are in effect network protocols. The example of IPv6 shows how hard it is to evolve network protocols. But now we are facing the obsolescence of a Web format that was very widey used as the long effort to kill off Adobe's Flash comes to fruition. Fortunately, Jason Scott's Flash Animations Live Forever at the Internet Archive shows that we were right all along. Below the fold, I go into the details.

Unless you're a member of Usenix (why aren't you?) you'll have to wait a year to read two of three interesting preservation-related articles in the Fall 2019 issue of ;login:. Below the fold is a little taste of each of them, with links to the full papers if you don't want to wait a year:

Source

David Gerard posts Wine on Windows 10. It works.

Windows 10 introduced Windows Subsystem for Linux — and the convenience of Ubuntu downloadable from the Microsoft Store. This makes this dumb idea pretty much Just Work out of the box, apart from having to set your DISPLAY environment variable by hand.

So far, it's mindbogglingly useless. It can only run 64-bit Windows apps, which doesn't even include all the apps that come with Windows 10 itself.

But I want to stress again: this now works trivially. I'm not some sort of mad genius to do this thing — I only appear to be the first person to admit to having done it publicly.

Gerard recounts the history of this "interesting" idea. Although he treats this as a "geek gotta do what a geek gotta do" thing, the interest for Emulation & Virtualization as Preservation Strategies is in the tail of the post:

TO DO: 32-bit support. This will have to wait for Microsoft to release WSL 2. I wonder if ancient Win16 programs will work then — they should do in Wine, even if they don't in Windows any more.

Of course, if they run in Wine on Ubuntu on Windows 10 on an x86, they should run on Wine on Ubuntu on an x86. But being able to run Wine in an official Microsoft environment might make deployment of preserved Win16 programs easier to get past an institution's risk-averse lawyers.

I've written before about the valuable work of the Software Preservation Network (SPN). Now they have released their EaaSI Sandbox, in which you can explore the capabilities of "Emulation as a Service" (EaaS), a topic I discussed in my report Emulation and Virtualization as Preservation Strategies. Below the fold I try EaaSi for the first time.

Oya Rieger of Ithaka S+R has published a report entitled The State of Digital Preservation in 2018: A Snapshot of Challenges and Gaps. In June and July Rieger:

talked with 21 experts and thought leaders to hear their perspectives on the state of digital preservation. The purpose of this report is to share a number of common themes that permeated through the conversations and provide an opportunity for broader community reaction and engagement, which will over time contribute to the development of an Ithaka S+R research agenda in these areas.

Below the fold, a critique.

Three years ago in Emulation and Virtualization as Preservation Strategies I wrote about Controlled Digital Lending (CDL):

One idea that might be worth exploring as a way to mitigate the legal issues is lending. The Internet Archive has successfully implemented a lending system for their collection of digitized books; readers can check a book out for a limited period, and each book can be checked out to at most one reader at a time. This has not encountered much opposition from copyright holders.

A similar system for emulation would be feasible; readers would check out an emulation for a limited period, and each emulation could be checked out to at most one reader at a time. One issue would be dependencies. An archive might have, say, 10,000 emulations based on Windows 3.1. If checking out one blocked access to all 10,000 that might be too restrictive to be useful.

Now, Controlled Digital Lending by Libraries offers libraries the opportunity to:

• better understand the legal framework underpinning CDL,
• communicate their support for CDL, and
• build a community of expertise around the practice of CDL.

Below the fold, some details.

There was lots of great stuff at the Internet Archive's Annual Bash. But for those of us who can remember the days before PCs played music, the highlight was right at the end of the presentations when the awesome Jason Scott introduced the port of 1997's WinAmp to the Web. Two years earlier:

WinPlay3 was the first real-time MP3 audio player for PCs running Windows, both 16-bit (Windows 3.1) and 32-bit (Windows 95). Prior to this, audio compressed with MP3 had to be decompressed prior to listening.

Source

WinPlay3 was the first, but it was bare-bones.It was WinAmp that really got people to realize that the PC was a media device. But the best part was that WinAmp was mod-able. It unleashed a wave of creativity (Debbie does WinAmp, anyone?), now preserved in the Archive's collection of over 5,000 WinAmp skins!

Jason has the details in his blog post Don't Click on the Llama:

Thanks to Jordan Eldredge and the Webamp programming community for this new and strange periscope into the 1990s internet past.

When I first clicked on the llama on The Swiss Family Robinson on my Ubuntu desktop the sound ceased. It turns out that the codec selection mechanism is different between the regular player and WinAmp, and it needed a codec I didn't have installed. The fix was:

sudo apt install ubuntu-restricted-extras

Source

I should also note that the Archive's amazing collection of emulations now includes the Commodore 64 (Jason's introduction is here), and 1,100 additional arcade machines.

In my report Emulation and Virtualization as Preservation Strategies I discussed the legal issues around emulating obsolete software, the basis for the burgeoning retro-gaming industry. These issues have attracted attention recently, as Kyle Orland reports:

In the wake of Nintendo's recent lawsuits against other ROM distribution sites, major ROM repository EmuParadise has announced it will preemptively cease providing downloadable versions of copyrighted classic games.

Below the fold, some comments on this threat to our cultural history.

Jason Fagone at the San Francisco Chronicle has a fascinating story of heroic, successful (and timely) emulation in The Silicon Valley quest to preserve Stephen Hawking’s voice. It's the story of a small team which started work in 2009 trying to replace Hawking's voice synthesizer with more modern technology. Below the fold, some details to get you to read the whole article

Science Friday's Lauren Young has a three-part series on digital preservation:

1. Ghosts In The Reels is about magnetic tape.
2. The Librarians Saving The Internet is about Web archiving.
3. Data Reawakening is about the search for a quasi-immortal medium.

Clearly, increasing public attention to the problem of preserving digital information is a good thing, but I have reservations about these posts. Below the fold, I lay them out.

The Digital Preservation Coalition's International Digital Preservation Day was marked by a wide-ranging collection of blog posts. Below the fold, some links to and comments on, a few of them.

Jonathan Farbowitz's NYU MA thesis More Than Digital Dirt: Preserving Malware in Archives, Museums, and Libraries is well worth a more leisurely reading than I've given it so far. He expands greatly on the argument I've made that preserving malware is important, and attempting to ensure archives are malware-free is harmful:

At ingest time, the archive doesn't know what it is about the content future scholars will be interested in. In particular, they don't know that the scholars aren't studying the history of malware. By modifying the content during ingest they may be destroying its usefulness to future scholars.

For example, Farbowitz introduces his third chapter A​ ​Series​ ​of​ ​Inaccurate​ ​Analogies thus:

In my research, I encountered several criticisms of both the intentional collection of malware by cultural heritage institutions and the preservation of malware-infected versions of digital artefacts. These critics have attempted to draw analogies between malware infection and issues that are already well-understood in the treatment and care of archival collections. I will examine each of these analogies to help clarify the debate and elucidate how malware fits within the collecting mandate of archives, museums, and libraries

He goes on to to demolish the ideas that malware is like dirt or mold. He provides several interesting real-world examples of archival workflows encountering malware. His eighth chapter Risk​ ​Assessment​ ​Considerations​ ​for​ ​Storage​ ​and​ ​Access is especially valuable in addressing the reasons why malware preservation is so controversial.

Overall, a very valuable contribution.

Kyoto Railway Museum

Much as I love Kyoto, now that I'm retired with daily grandparent duties (and no-one to subsidize my travel) I couldn't attend iPRES 2017.

I have now managed to scan both the papers, and the very useful "collaborative notes" compiled by Micky Lindlar, Joshua Ng, William Kilbride, Euan Cochrane, Jaye Weatherburn and Rachel Tropea (thanks!). Below the fold I have some notes on the papers that caught my eye.

At last December's WinHEC conference, Qualcomm and Microsoft made an announcement to which I should have paid more attention:

Qualcomm ... announced that they are collaborating with Microsoft Corp. to enable Windows 10 on mobile computing devices powered by next-generation Qualcomm® Snapdragon™ processors, enabling mobile, power efficient, always-connected cellular PC devices. Supporting full compatibility with the Windows 10 ecosystem, the Snapdragon processor is designed to enable Windows hardware developers to create next generation device form factors, providing mobility to cloud computing.

The part I didn't think about was:

New Windows 10 PCs powered by Snapdragon can be designed to support x86 Win32 and universal Windows apps, including Adobe Photoshop, Microsoft Office and Windows 10 gaming titles.

How do they do that? The answer is obvious: emulation! Below the fold, some thoughts.

The Internet Archive's massive collection of software now includes an in-browser emulation in the Emularity framework of the original Mac with MacOS from 1984 to 1989, and a Mac Plus with MacOS 7.0.1 from 1991. Shaun Nichols at The Register reports that:

The emulator itself is powered by a version of Hampa Hug's PCE Apple emulator ported to run in browsers via JavaScript by James Friend. PCE and PCE.js have been around for a number of years; now that tech has been married to the Internet Archive's vault of software.

Congratulations to Jason Scott and the software archiving team!

This post is an edited extract from my talk at last year's IIPC meeting. This part was the message I was trying to get across, but I buried the lede at the tail end. So I'm repeating it here to try and make the message clear.

Emulation technology will evolve through time. The way we expose emulations on the Web right now means that this evolution will break them. We're supposed to be preserving stuff, but the way we're doing it isn't preservable. We need to expose emulations to the Web in a future-proof way, a way whereby they can be collected, preserved and reanimated using future emulation technologies. Below the fold, I explain what is needed using the analogy of PDFs.

Apart from a short paper describing a heroic effort of Web archaeology, recreating Amsterdam's De Digitale Stadt, the whole second morning of iPRES2016 was devoted to the preservation of software and Internet-based art. It featured a keynote by Sabine Himmelsbach of the House of Electronic Arts (HeK) in Basel, and three papers using the bwFLA emulation technology to present preserved software art (proceedings in one PDF):

• A Case Study on Emulation-based Preservation in the Museum: Flusser Hypertext, Padberg et al.
• Towards a Risk Model for Emulation-based Preservation Strategies: A Case Study from the Software-based Art Domain, Rechert et al.
• Exhibiting Digital Art via Emulation – Boot-to-Emulator with the EMiL Kiosk System, Espenschied et al.

Preserving software art is an important edge case of software preservation. Each art piece is likely to have many more dependencies on specific hardware components, software environments and network services than mainstream software. Focus on techniques for addressing these dependencies in an emulated environment is useful in highlighting them. But it may be somewhat misleading, by giving an exaggerated idea of how hard emulating more representative software would be. Below the fold, I discuss these issues.

I was one of five members of a panel on Software Preservation at iPRES 2016, moderated by Maureen Pennock. We each had three minutes to answer the question "what have you contributed towards software preservation in the past year?" Follow me below the fold for my answer.