Query by Image and Video Content: The QBIC System

Advances in scanning, networking, compression and video technology--and the proliferation of multimedia computers--have led to the generation of large on-line collections of images and videos. These collections have created a need for new methods to locate specific images or video clips. The Query by Image Content (QBIC) project is studying methods to extend and complement text-based retrievals by querying and retrieving images and videos by content. Queries can be performed using attributes such as colors, textures, shapes, and object position. Video-specific queries include those on camera motion parameters like zoom, pan, and object motion. The project has resulted in a prototype system with two major steps: database population and query. In population, methods identify objects in still images, segment videos into short sequences called shots, and compute features describing color, texture, shape, position, or motion information. In database query, images and shots can be retrieved by example ("Show me images similar to this image") or by selecting properties from pickers such as a color wheel, a sketched shape, a list of camera motions, or a combination of these. Key QBIC technical issues include a visual query language and a graphical user interface that lets users form a query by painting, sketching, or selecting graphical elements. Key also are indexing techniques for high-dimensional features describing image and video content, automatic segmentation techniques for images (to identify interesting objects), and videos (to identify shots and interesting moving and static objects), and similarity retrieval (to match human perception). QBIC technology has moved into the Ultimedia and Digital Library commercial IBM products.