Desktop search

This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages) This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (October 2014) (Learn how and when to remove this message) ‹The template Manual is being considered for merging.›  This is written like a manual or guide. Please help rewrite this and remove advice or instruction. (October 2016) (Learn how and when to remove this message)

Desktop search tools search within a user's own computer files as opposed to searching the Internet. These tools are designed to find information on the user's PC, including web browser history, e-mail archives, text documents, sound files, images, and video. A variety of desktop search programs are now available; see this list for examples. Most desktop search programs are standalone applications. Desktop search products are software alternatives to the search software included in the operating system, helping users sift through desktop files, emails, attachments, and more.^[1][2][3]

Desktop search emerged as a concern for large firms for two main reasons: untapped productivity and security. According to analyst firm Gartner, up to 80% of some companies' data is locked up inside unstructured data — the information stored on a user's PC, the directories (folders) and files they've created on a network, documents stored in repositories such as corporate intranets and a multitude of other locations.^[4] Moreover, many companies have structured or unstructured information stored in older file formats to which they don't have ready access.

The sector attracted considerable attention in the late 2004 to early 2005 period from the struggle between Microsoft and Google.^[5][6][7] According to market analysts, both companies were attempting to leverage their monopolies (of web browsers and search engines, respectively) to strengthen their dominance. Due to Google's complaint that users of Windows Vista cannot choose any competitor's desktop search program over the built-in one, an agreement was reached between US Justice Department and Microsoft that Windows Vista Service Pack 1 would enable users to choose between the built-in and other desktop search programs, and select which one is to be the default.^[8] As of September 2011, Google ended life for Google Desktop.

Most desktop search engines build and maintain an index database to improve performance when searching large amounts of data. Indexing usually takes place when the computer is idle and most search applications can be set to suspend indexing if a portable computer is running on batteries, in order to save power. There are notable exceptions, however: Voidtools' Everything Search Engine,^[9] which performs searches over only file names, not contents, is able to build its index from scratch in just a few seconds. Another exception is Vegnos Desktop Search Engine,^[10] which performs searches over filenames and files' contents without building any indices. An index may also not be up-to-date, when a query is performed. In this case, results returned will not be accurate (that is, a hit may be shown when it is no longer there, and a file may not be shown, when in fact it is a hit). Some products have sought to remedy this disadvantage by building a real-time indexing function into the software. There are disadvantages to not indexing. Namely, the time to complete a query can be significant, and the issued query can also be resource-intensive.

Desktop search tools typically collect three types of information about files:

• file and folder names
• metadata, such as titles, authors, comments in file types such as MP3, PDF and JPEG
• file content, for the types of documents supported by the tool

Long-term goals for desktop search include the ability to search the contents of image files, sound files and video by context.^[11][12]

Indexing Service, a "base service that extracts content from files and constructs an indexed catalog to facilitate efficient and rapid searching",^[13] was originally released in August 1996. It was built in order to speed up manually searching for files on Personal Desktops and Corporate Computer Network. Indexing service helped by using Microsoft web servers to index files on the desired hard drives. Indexing was done by file format. By using terms that users provided, a search was conducted that matched terms to the data within the file formats. The largest issue that Indexing service faced was the fact that every time a file was added, it had to be indexed. This coupled with the fact that the indexing cached the entire index in RAM, made the hardware a huge limitation.^[14] This made indexing large amounts of files require extremely powerful hardware and very long wait times.

In 2003, Windows Desktop Search (WDS) replaced Microsoft Indexing Service. Instead of only matching terms to the details of the file format and file names, WDS brings in content indexing to all Microsoft files and text-based formats such as e-mail and text files. This means, that WDS looked into the files and indexed the content. Thus, when a user searched a term, WDS no longer matched just information such as file format types and file names, but terms, and values stored within those files. WDS also brought "Instant searching" meaning the user could type a character and the query would instantly start searching and updating the query as the user typed in more characters.^[15] Windows Search apparently used up a lot of processing power, as Windows Desktop Search would only run if it was directly queried or while the PC was idle. Even only running while directly queried or while the computer was idled, indexing the entire hard drive still took hours. The index would be around 10% of the size of all the files that it indexed, e.g. if the indexed files amounted to around 100GB, the index size would be 10GB.

With the release of Windows Vista came Windows Search 3.1. Unlike its predecessors WDS and Windows Search 3.0, 3.1 could search through both indexed and non indexed locations seamlessly. Also, the RAM and CPU requirements were greatly reduced, cutting back indexing times immensely. Windows Search 4.0 is currently running on all PCs with Windows 7 and up.

In 1994 the AppleSearch search engine was introduced, allowing users to fully search all documents within their Macintosh computer, including file format types, meta-data on those files, and content within the files. AppleSearch was a client/server application, and as such required a server separate from the main device in order to function. The biggest issue with AppleSearch were its large resource requirements: "AppleSearch requires at least a 68040 processor and 5MB of RAM."^[16] At the time, a Macintosh computer with these specifications was priced at approximately $1400; equivalent to $2050 in 2015.^[17] On top of this, the software itself cost an additional $1400 for a single license.

In 1997, Sherlock was released alongside Mac OS 8.5. Sherlock (named after the famous fictional detective Sherlock Holmes) was integrated into Mac OS's file browser – Finder. Sherlock extended the desktop search function to the World Wide Web, allowing users to search both locally and externally. Adding additional functions—such as internet access—to Sherlock was relatively simple, as this was done through plugins written as plain text files. Sherlock was included in every release of Mac OS from Mac OS 8, before being deprecated and replaced by Spotlight and Dashboard in Mac OS X 10.4 Tiger. It was officially removed in Mac OS X 10.5 Leopard

Spotlight was released in 2005 as part of Mac OS X 10.4 Tiger. It is a Selection-based search tool, which means the user invokes a query using only the mouse. Spotlight allows the user to search the Internet for more information about any keyword or phrase contained within a document or webpage, and uses a built-in calculator and Oxford American Dictionary to offer quick access to small calculations and word definitions.^[18] While Spotlight initially has a long startup time, this decreases as the hard disk is indexed. As files are added by the user, the index is constantly updated in the background using minimal CPU & RAM resources.

There are a wide range of desktop search options for Linux users, depending upon the skill level of the user, their preference to use desktop tools which tightly integrate into their desktop environment, command-shell functionality (often with advanced scripting options), or browser-based users interfaces to locally running software. In addition, many users create their own indexing from a variety of indexing packages (e.g. one which does extraction and indexing of PDF/DOC/DOCX/ODT documents well, another search engine which works ith/ vcard, LDAP, and other directory/contact databases, as well as the conventional find and locate commands.

Ubuntu Linux didn't have desktop search until release Feisty Fawn 7.04. Using Tracker^[19] desktop search, the desktop search feature was very similar to Mac OS's AppleSearch and Sherlock. It not only featured the basic features of file format sorting and meta-data matching, but support for searching through emails and instant messages was added. In 2014 Recoll^[20] was added to Linux distributions, working with other search programs such as Tracker and Beagle to provide efficient full text search. This greatly increased the types of queries and file types that Linux desktop searches could handle. A major advantage of Recoll is that it allows for greater customization of what is indexed; Recoll will index the entire hard disk by default, but can be made to index only selected directories, omitting directories that will never need to be searched.^[21]

Starting with KDE4, the NEPOMUK was introduced. It provided the ability to index a wide range of desktop content, email, and use semantic web technologies (e.g. RDF) to annotate the database. The introduction faced a few glitches, much of which seemed to be based on the triplestore. Performance improved (at least for queries) by switching the backend to a stripped-down version of the Virtuoso Open Source Edition, however indexing remained a common user complaint. Based on user feedback, the Nepomuk indexing and search has been replaced with the Baloo framework^[22] based on Xapian.^[23]

• List of desktop search engines