Capture: a desktop display-centric text recorder

As more and more information is designed for human visual consumption through computer displays, the need to capture and process display-centric content is becoming increasingly important, especially for visually impaired users. We present Capture, a novel display-centric text recorder that facilitates real-time access to onscreen text and its structure and contextual information, including data associated with both foreground and background windows. Capture provides an intelligent caching architecture that integrates with the standard accessibility framework available on modern operating systems to continuously track onscreen text and metadata. This enables fast, semantic information recording without any modifications to applications, window systems, or operating system kernels. The recorded data is useful for a variety of problem domains, including assistive technologies, desktop search, auditing, and predictive graphical user interfaces. We have implemented a Capture prototype on Linux with the GNOME Accessibility Toolkit. Our results on real desktop applications demonstrate that Capture provides low runtime overhead and much more complete recording of onscreen text than modern desktop screen readers used for visually impaired users.