Nothing Special   »   [go: up one dir, main page]

The un-queried life is not worth living
— Socrates (Plato, The Apology, 38a)

W3C INK

W3C XML Query (XQuery)

30,000 Foot View

XQuery: 30,000 foot view (CIO, CTO, Journalist)

XQuery is a standardized language for combining documents, databases, Web pages and almost anything else. It is very widely implemented. It is powerful and easy to learn.

XQuery is replacing proprietary middleware languages and Web Application development languages. XQuery is replacing complex Java or C++ programs with a few lines of code. XQuery is simpler to work with and easier to maintain than many other alternatives.

Do more with less.

Contents

Architects

XQuery for the systems analyst or architect

The W3C XML Query Working Group worked with the W3C XML Schema Working Group and the W3C XSL Working Group to make a set of specifications that all work together.

Use XQuery to take data from multiple databases, from XML files, from remote Web documents, even from CGI scripts, and to produce XML results that you can process with XSLT.

Use XQuery on the back-end of a Web server, or to generate Enterprise-wide executive reports.

Examples of XML Query In Action

Submit your entry by sending mail to liam at w3.org, with [XQuery site] in the subject. Remember to give the full URL, and remember that it must be public.

Oxford African American Studies Center is a site published by Oxford University Press using an XQuery system. See the Site Credits link there for more details.

Alberta learning Center for education in the Canadian province of Alberta; e.g. see the Search link, which does not require registration.

AuthorMapper shows authors of scientific papers and articles broken down by geography and subject. The site is by Springer.

Business Week Business School Comparator is said to use XQuery to let users compare universities and business schools; it didn't seem to work for me, so maybe IE only? Or maybe I'm not cut out for business.

Business Exchange, another Business Week site, uses XQuery to drive a site where, it says, usiness professionals can collaborate and network around business topics.

CQ Legislative Impact is a tool to explore how pending US legislation might affect existing laws.

Data Request Broker, DRB, at Gael Consultant, is an open source Java API for processing heterogeneous data. It includes XQuery and Scema support. License is LGPL.

The Dolley Madison Digital Edition by the University of Virginia Press.

fromoldbooks.org has an image search engine powered by XML Query; you can see the text of the queries (follow the About link on the Search page). This is also used by Liam Quin's Photograph search page.

MarkMail is an XQuery-based application for searching and visualising mailing lists.

The New England Journal of Medicine uses XQuery to search and retrieve comprehensive case summaries and graphical icons that identify available supplemental content such as lab reports, radiological scans, histopathology slides, and photos associated with a particular case record.

O'Reilly Labs use XQuery to power code search, image search, statistics and more.

Pop Culture Universe is a Web site about American popular culture, including movies, books and music, with over 300 publications indexed and searchable.

Springer Images uses XQuery to search and retrieve scientific images, photos, tables, charts etc. for researchers.

Springer Exemplar provides a full-text interface to large numbers of scientific and technical journals, together with tools for narrowing down search results.

Wiley Custom Select is a Web site for creating custom course books.

Worldcolor has a custom publishing system using XQuery; they have a Flash demo.

Have you got a Web site that's powered by XML Query? A success story to share? Contact liam at w3.org.

Users

XQuery: choosing an implementation

There are over 40 different software packages that support XML Query in some way. Things to look for include availability of support, platforms, price, performance, all the usual issues, but you should also ask whether the software supports the final syntax from the W3C Recommendation or implements an earlier draft. Another XML Query specific feature is support for XML files, for fetching documents via HTTP, and for connecting to relational (or other) data sources: that is, whether the package lives up to the XML Query promise of unifying access to many different forms of information.

List of XQuery Implementations

Learning

There are some books listed; there are also people offering training and tutorials. If there is anything you found particular helpful, let us know!

There are also some mailing lists devoted to XML and to XML Query. You should look at the archives of each list before posting; you'll also need to subscribe to the list before you can post to it in most cases.

www-ql

This is the W3C public mailing list on query languages, including (but not limited to) discussion on the XML Query project. Do not use this to send comments on the specification, such as errata or feature requests; see the Status section in each specification for instructions on how to send comments to the Working Group.

xquery-talk

A mailing list hosted at x-query.com, especially for discussing XQuery.

xml-dev

Probably the most widely-known list for discussing XML.

Reading the Specs

W3C Specifications are aimed first and foremost at programmers writing implementations of them. We also try to make them readable for people trying to learn the language—but given a choice between making a standard precise and making it easy to read, we have to make it precise.

If you are fairly technical, you could start by reading the XML Query specification, and the XQuery Use Cases document has some examples. Many people would rather look for a book or tutorial.

Implementers

XQuery for the implementer: hard core query

Implementers: what would you most like to see here? What would have helped you the most?

The XQuery Test Suite

W3C test suites exist to show that specifications can be implemented. They are testing the specs, not the code!

Static Typing and Formal Semantics

XPath 2 has typed values; that is, the language associates a value type with each expression, variable or function. The set of possible types is that defined by W3C XML Schema, augmented by user-defined types derived from those basic Schema types using an external schema. The way in which an XPath or XQuery system derives and checks the type of an expression is defined formally, using a mathematical notation, in the XQuery 1.0 and XPath 2.0 Formal Semantics Recommendation.

Note that both external W3C Schema support and static typing are optional features, so not all implementations support them.

Conformance Statements

You will often see things in the specifications marked as being implementation defined. You must document what your implementation does for each of these.

Documents

Specifications and Working Group Notes

The W3C XML Query Working Group has published a lot of documents. Many of these were done together with the XSL Working Group and are marked Joint.

Both of these Working Groups also met with the W3C XML Schema Working Group, to make sure our specifications all work together.

First, the main XML Query documents:

XQuery 3.1: An XML Query Language (W3C Recommendation)

XML Syntax for XQuery 3.1 (XQueryX) (W3C Recommendation)

XML Query (XQuery) 3.1 Requirements (W3C Working Group Note)

XML Query 3.0 Use Cases (W3C Working Group Note)

Building a Tokenizer for XPath or XQuery (Joint Note)

XSLT and XQuery share a lot of the same functionality:

XSL Transformations (XSLT) Version 2.0 (by the XSL Working Group)

XSL Transformations (XSLT) Version 3.0 (by the XSL Working Group)

XQuery and XSLT both use XPath:

XML Path Language (XPath) 3.1 (W3C Recommendation; Joint)

XPath in turn is built on a number of Joint specifications :

XQuery and XPath Data Model (W3C Recommendation; Joint)

XSLT and XQuery Serialization (W3C Recommendation; Joint)

XQuery and XPath Functions and Operators (W3C Recommendation; Joint)

The XML Query and XSL Working Groups developed Full-Text Search for XPath and XQuery:

XML Query and XPath Full-Text Requirements (Joint)

XML Query and XPath Full-Text Use Cases (Joint)

XQuery 1.0 and XPath 2.0 Full-Text (Joint)

The XML Query Working Group developed an update facility for XQuery; this lets you write Query expressions that change documents and perhaps save the result.

XQuery Update Facility

XQuery Update Facility Requirements

XQuery Update Facility Use Cases

The XML Query Working Group is no longer working on Scripting Extensions for XQuery. The goal was to investigate whether adding imperative (procedural) features such as variable assigment and explicit sequencing to XQuery makes the language significantly more powerful or easier to use.

XQuery Scripting Extension 1.0

XQuery Scripting Extension 1.0 Requirements

XQuery Scripting Extension 1.0 Use Cases

News

Recently added...

Send your XQuery-related news item to liam@w3.org, with [XQuery] in the Subject. Note, I am sorry that sometimes I miss announcements. I get hundreds, sometimes thousands, of messages in a day. If your announcement does not appear within one week, send it again and please accept my apologies!

Subscribe to the RSS feed.

XQuery 3.0, XPath 3.0 and supporting specs now W3C Recommendations

Fri, 11 Apr 2014

XQuery 3.0 is a W3C Recommendation along with XQueryX, XPath 3, XDM 3, Serialization 3 and of course functions and Operators 3.

BaseX 7.8 Released

Wed, 12 Feb 2014

BaseX 7.8 was released today; it adds a project view, enhancements to the built-in editor, speedups to insert, delete, XQuery functions and more, and includes enhancements to the the XQuery add-on modules such as JSON , Full-Texxt and ExPath File. The software is also now available in Russian and Spanish.

XMLSpy 2014

Sat, 14 Dec 2013

Altova has released XML Spy 2014 with support for XQuery 3.0, XPath 3.0, XSLT 3.0 draft and also XML Schema 1.1 using the RaptorXML engine.

Apache Lux

Wed, 06 Nov 2013

Lux release 0.11; includes moving to Saxon 9.5, Solr 4.4, experimental SolrCloud support.

BaseX 7.7 Released

Wed, 07 Aug 2013

BaseX 7.7 was released today; it includes support for the XQuery 3.0 Candidate Recommendation, many new features and performance enhancements, improved support for large databases, and a birthday cake for Leo.

Zorba 2.9 released

Mon, 24 Jun 2013

Zorba 2.9 was released today; Zorba is an open source XQuery implementation in C++ that also includes a JSONiq implementation. This release increases XQuery 3 support (including higher order functions) and also makes changes to the JSONiq implementation.

Xidel HTML/XML data extraction tool 0.7

Fri, 29 Mar 2013

Xidel is a tool for extracting data from Web pages using a combination of CSS selectors, XQuery and JSONiq. [since renamed to Xidel]

W3C Java Applets updated

Tue, 29 Jan 2013

W3C Java Applets have been updated to incorporate the latest versions of the specifications for the XPath and XQuery 3.0 Candidate Recommendations as well as for the 1.0 and 2.0 documents.

W3C Recommendations!

Tue, 23 Jan 2007

W3C Recommendation Status for XQuery 1.0: An XML Query Language, XML Syntax for XQuery 1.0 (XQueryX) and XML Path Language (XPath) 2.0, as well as the supporting specifications, XQuery 1.0 and XPath 2.0 Functions and Operators, XQuery 1.0 and XPath 2.0 Data Model (XDM), XSLT 2.0 and XQuery 1.0 Serialization and of course XQuery 1.0 and XPath 2.0 Formal Semantics. In addition, XSL Transformations (XSLT) Version 2.0 by the XSL Working Group is also a W3C Recommendation.

Sun, 03 Dec 2006

Micro XQuery Engine 0.1

Micro XQuery Engine 0.1 was released.

Mon, 20 Nov 2006

oXygen Editor 8.0

oXygen editor 8.0 includes debugging support for both XML Query and XSLT, with support for Berkeley XML DB, IBM DB2 Pure XML, eXist XML Database, MarkLogic, Microsoft SQLServer 2005, Oracle 10g R2, RainingData TigerLogic XDMS, SoftwareAG Tamino and XHive XML Database.

[older] *-->

Patent Disclosures

The XML Query Working Group operates under the Royalty Free terms of the W3C Patent policy. Patent disclosures relevant to the specifications produced by the XML Query working group can be found in the Implementation of the W3C Patent Policy (IPP) XML Query IPP status page and, for XSL and joint specifications the XSL WG IPP status page. Older disclosures are on the XML Query Working Group's patent disclosure page at http://www.w3.org/2002/08/xmlquery-IPR-statements.

Specifications that are joint work with the XSL working group have also the additional patent disclosures provided by the XSL wg at http://www.w3.org/Style/XSL/Disclosures.

Discussion/Feedback:

Implementations

XML Query Implementations

If your implementation is not here, or if you know of an implementation that is not listed, send liam@w3.org the details!

Software that implements the XQuery and XPath Full Text Facility is so marked in this list. There is a separate list of XPath 2 implementations.

  1. The W3C XPath and XQuery Grammar Test Page features Java applets that read expressions and show the resulting parse tree.
  2. Abacus Systems' Relational XQuery supports both relational data (via JDBC) and other sources including XML files, and also claims XQJ (XQuery for Java API) conformance. Includes a GUI for creating and editing queries. 30 day evaluation.
    [2011-03-25: Last update July 2009; project homepage is gone.
  3. Altova GmbH XMLSpy 2006 includes an XQuery Debugger, a code generator for mapping between Schemas, and AltovaXML Query Processor which handles both XSLT 2 and XML Query 1.0 [30-day free trial]
  4. Apple's Sherlock for Mac OS X; see also their XML Query Extension functions.
  5. BEA's Oracle Data Services Platform [90-day free trial]. BEA was an active participant in the XML Query Working Group; the company was bought by Oracle, also an active participant.
  6. Berkeley Lab's Nux, an open source Java in-memory toolkit for XML, XQuery, XPath, schema validation, fuzzy fulltext similarity search and related technologies using Saxon, XOM, Xerces and JAXB [open source under a BSD-style license]. Implements the XQuery Update Facility; Full-Text Support; latest release seems to be June 2006.
  7. Bluestream Database Software Corp.'s XStreamDB, a native XML database server and full text support, aimed primarily at DITA. [commercial with trial download]
  8. David Carlisle's xq2xml converts XQuery to XML, to XQueryX and to XSLT.
  9. Cerebra Inc.'s Cerebra Server supports XQuery, OWL-DL and RDF, and can connect to external databases, but their Web server no longer responds.
  10. Cognetic Systems's XQuantum implements XML Query 1.0 in an XML-native indexed data store. They have a Web page demonstrating the XQuery Use Cases, and support static typing and modules as well as some full-text extensions. [Windows and Linux; 30-day evaluation] Full-Text Support
  11. DataDirect's DataDirect XQuery (tm), an embeddable component for XQuery that implements the XQuery for Java(tm) API (XQJ) [Java; 15-day trial download]. DataDirect participates in the XML Query Working Group.
  12. DataDirect's Stylus Studio 5.0 (XQuery, XML Schema and XSLT IDE). DataDirect participates in the XML Query Working Group.
  13. EMC's xDB; this was formerly X-Hive, and EMC also owns Documentum. Their xDB product claims to be a native XML database in Java, with full XQuery support. [commercial, free evaluation download]
  14. eXist has a Java-based native XML database with an XQuery interface. [Open source, GNU LGPL.] Implements the XQuery Update Facility
  15. The open source GCX, a streaming in-memory XQuery engine with static and dynamic buffer minimzation developed originally at Saarland University [open source]
  16. MXQuery from ETH, a research project; the sourceforge page says, The Micro XQuery Engine is a low-footprint, extensible implementation of XQuery 1.0 including extensions like the XQuery Update and XQueryP. It supports streaming execution and runs on all devices support CLDC 1.0 upwards. [Open source, BSD/Apache license]. ETH is an active participant in the XML Query Working Group. Implements the XQuery Update Facility
  17. Fatdog Software's XQEngine Java. [Open source: GPL or as negotiated]. Full-Text Support [last update 2009-07-18]
  18. GAEL's Derby provides a Java API via their Data Request Broker. There is extensive support for data analysis, including plotting graphs and making tables.
  19. Galax. Open-source (in OCAML), with a Galatex full text search implementation. The authors of Galax include a number of active participants in the XML Query Working Group, both psat and present. Full-Text Support
  20. GNU's Qexo (Kawa-Query) by Per Bothner. Compiles XQuery on-the-fly to Java bytecodes. Based on and part of the Kawa framework. Qexo implements the optional XQuery static typing feature. [Open-source under the GPL-like Kawa License].
  21. HXQ, a compiler from XQuery to Haskell; appears to be an imcomplete research project, but said to be already useful. Open source, license terms unclear from the Web page.
  22. Ipedo's XIP includes a "dual core" SQL + XML Query engine (XMLDB).
  23. IBM's WebSphere Application Server Feature Pack for XML supports XPath 2.0, XSLT 2.0, and XQuery 1.0, with a Java API that unifies all three languages. [Free download; requires WebSphere Application Server, which is commercial]
  24. IBM's xqnsta: XQuery Normalizer and Static Analyzer (XQNSTA) is a Java API and GUI for normalizing and computing the static type of XQuery expressions. IBM is an active participant in the XML Query Working Group.
  25. IBM's DB2 9 stores XML in its native format and provides support for XQuery.
  26. IPSI's IPSI-XQ [java; free download]; this seems to have moved to sourceforge. Last update 2001-11-29.
  27. Ispras Modis' Sedna. Native XML DBMS in C/C++ and Scheme; partial support for XML Query. Includes an Apache HTTP module, and APIs for .NET, Python and Chicken Scheme. There is also a Firefox extension, XqUSEme, and a special build of firefox, XDIB (XQuery In a Browser), for unning client-side XQuery scripts. [Open source under the Apache License].
  28. Liquid Technologies' Liquid XML Studio features an XQuery Debugger [30-day trial for Microsoft platforms].
  29. Apache Lux combines the Lucene/Solr search index with the Saxon XQuery/XSLT processor, adding XQuery support to Solr and adding persistence to Saxon. [Open source.]
  30. MarkLogic's MarkLogic Server 4.0 (formerly known as Content Interaction Server). There is also a technical overview document. Commercial, with free download restricted to 100 Megaybytes of data. A limited duration trial license is also available, limited to 1G of content. Full-Text Support. Implements the XQuery Update Facility. MarkLogic is an active participant in the XML Query Working Group.
  31. Microsoft's SQL Server 2005 Express, with XML Schema, XPath 2, and XML Query support. Later versions of SQL Server continue to support XQuery. Microsoft is an active participant in the XML Query Working Group.
  32. CWI's MonetDB/XQuery is an XQuery system that also supports XQUF updates. It is based on the Pathfinder compiler developed at TU Munich, and aims at achieving high performance. Open Source (adapting the Mozilla Public License). Implements the XQuery Update Facility.
  33. OpenLink Software's Virtuoso Universal Server claims to support XSLT 1.2 (?! their link points to the XSLT 1.1 draft), XQuery and SQLX.
  34. Oracle Berkeley DB XML 2.0, formerly Sleepycat's, an embeddable native XML database with support for XQuery 1.0 (July 2004 draft), implemented in C++, with interfaces for Java, Python, Perl and PHP. Open source. Oracle is an active participant in the XML Query Working Group. Full-Text Support
  35. Oracle'sOracle XQuery implementation is part of the Oracle Database product [multi-platform; seems to be a free binary download]. Oracle is an active participant in the XML Query Working Group.
  36. Patternist, an XQuery 1.0, XSL-T 2.0 and XPath 2.0 implementation that provices a C++ API (open source under GPL, uses TrollTech's Qt library)
  37. PHP XML Classes includes XqueryLite, a PHP implementation from 2002. [open source]
  38. Politecnico di Milano's XQBE and other XQuery products
  39. QuiLogic's SQL/XML-IMDB supports a mixure of SQL statements and XQuery expressions. [Free trial requires a restart every hour]
  40. RainingData's TigerLogic XDMS XML Data Management Server for Sun Solaris and Microsoft Windows [free trial].
  41. Renmin University of China's OrientX, a native XML database system in C/C++ developed under Renmin University of China. [open source]
  42. Saarland University Database Group's FluXQuery [no longer maintained], an extension of the XQuery language, FluX, that supports event-based query processing and the conscious handling of main memory buffers. Obsoleted by GCX, but the web page is still mirrored at Cornell.
  43. Saxonica's Saxon implements both XML Query and XSLT 2.0. Available in a schema-aware version as a commercial product, and without schema support as open source. Saxonica is an active participant in the XML Query Working Group.
  44. Software AG's
  45. Sonic Software's Sonic XML Server [30-day trial]
  46. The Universität Konstanz's Database and Information Systems Group's BaseX (open source/GPL) Implements the XQuery Update Facility; Full-Text Support
  47. videlibri, an open source XPath 2 implementation in Free Pascal by Benito van der Zander. Further work continues under the name Xidel (q.v.).
  48. The Apache Software Foundation's VXQuery says, The focus is on the evaluation of queries on large amounts of XML data. Specifically the goal is to evaluate queries on large collections of relatively small XML documents. To achieve this queries will be evaluated on a cluster of shared nothing machines.
  49. The University of Texas at Arlington Computer Science Department has people working on XQP: XQuery Processing on a P2P System. [Java; open source]
  50. The Univerisity of Antwerp's Blixem LiXQuery engine implements a subset of XQuery intended for teaching (the download link no longer works, as of September 2009).
  51. Worcester Polytechnic Institute's RainbowCore. [Java. available at no charge and without warranty].
  52. XBird, a light-weight embeddable XQuery processor and database system written in Java, with a distributed XQuery processor. [open source]
  53. Xidel by Benito van der Zander, is an open source implementation (in Pascal) of XPath 2, XQuery 1 with some XPath 3 and XQuery 3 features, JSONiq, CSS selectors and templates, with the aim of making a system to scrape Web sites easily.
  54. XMLmind's Qizx comes in three versions: (1) an open source one (Qizx/open); (2) a commercial implementation, Qizx/db, with an indexed native XML database and full-text support, and (3) Qizx/db Free Engine, a freely downloadable version of Qizx/db but that has a database size limit of approximately one gigabyte of XML. Implements the XQuery Update Facility; Full-Text Support
    55. *-->
  55. Xpriori's XMS native XML database, with XPath2.0/XQuery access language support [.Net on Linux and MS Windows; free unlimited download for development purposes.]
  56. XQuare Group and Universite' de Versailles Saint-Quentin's: XQuare Fusion and XQuare Bridge, open-source, used to be called xQuark (see also the Xquare home page)
  57. XQIB, XQuery In the Browser, an XQuery plugin for Microsoft Internet Explorer [open source]
  58. XQilla, C++ implementation based on pathan and Xerces-C. Open source (BSD/Sleepycat license). Sleepycat (Oracle) is an active participant in the XML Query Working Group.
  59. XQSharp, XQuery for the .NET framework, from CBCL. Includes Schema support and static typing. XQSharp was previously known as Anglo. [commercial; free for non-commercial use] CBCL is an active participant in the XML Query Working Group.
  60. Xyleme's Xyleme LCMS [commercial]
  61. Zorba, an open source portable embeddable C++ implementation of XQuery. There are also PHP, Python and Ruby APIs. [open source, Apache licence]

Unconfirmed Implementations

Please send liam@w3.org any information about these; I have tried to contact people where possible.

XPath 2 Implementations

Software that implements XPath 2.0, but not XML Query or XSLT 2

Related Products and Resources

The following is a (non-comprehensive) list of announcements of products that will include some support for XQuery, or that are of related nature:

Publications

Books

I have tried to indicate where authors participated in the XML Query Working Group, sent comments on the specifications, or have written implementations themselves. This does not necessarily make them good writers, but it may help you to understand their point of view and their connection with XML Query.

Books that I have received more recently are at the top of the list.

I have not seen the following books:

Tutorials

A Google search for "(XQuery tutorial"|"XML Query tutorial") produced over 150 matches (Google actually said about 30,500 but this turned out to mean exactly 153).

Other Pointers

To suggest a new pointer, send an email to member-query-feedback@w3.org.

Contacting Us

To contact the XML Query and XSL Working Groups, you can send email to public-qt-comments at w3.org

To report errors in the specification please use bugzilla, as described in the Status section at the start of each specification.

To comment on this page, send mail to liam at w3 dot org

Valid XHTML 1.0! First created by Massimo Marchiori on April 2000.
Page redesigned in 2007 by Liam; images from www.fromoldbooks.org used by permission.
Page maintained by Liam Quin.
Current: $Revision: 1.221 $ $Date: 2017/04/22 18:49:04 $

Copyright © 2000-2006W3C (MIT, ERCIM, Keio ), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply. Your interactions with this site are in accordance with our public and Member privacy statements.