No spec changes needed. The spec allows (but does not force) the implementation to put limits on the value space of types like xs:integer. It is your implementation that does (or does not) restrict to a 64 bit signed integer, and it's your implementation that does not detect the minus unary operator over an integer literal and special case it.

The thing is that I believe that the spec does *not* allow to have less than 64 bit as supported range. This means that if you support the minimum, you cannot express -2^63 using literal syntax using the current production rules.

So don't use the minimum then.

This might be a problem if the spec recognized 2^64 and -2^64 as boundary cases for xs:integer literals, but it doesn't. The range of xs:integer literals is implementation-defined.

The spec requires support for at least 18 digits - and an implementation that took that literally would not allow 9223372036854775808 anyway (I make it 19 digits).

The only place where 2^64 comes into play is for types such as xs:long; but we don't have xs:long literals in the language, you can only construct them from strings or xs:integer values, where the issue of "-" being an operator rather than part of the value doesn't arise.

So you may have an implementation issue here, but I don't think there is a spec issue.

Michael Kay
Saxonica

(In reply to Michael Kay from comment #4)
> The spec requires support for at least 18 digits - and an implementation
> that took that literally would not allow 9223372036854775808 anyway (I make
> it 19 digits).
I had some trouble finding this, but see it now in XSD 1.0:

<quote>
Note:  All ·minimally conforming· processors ·must· support decimal numbers with a minimum of 18 decimal digits (i.e., with a ·totalDigits· of 18). However, ·minimally conforming· processors ·may· set an application-defined limit on the maximum number of decimal digits they are prepared to support, in which case that application-defined maximum number ·must· be clearly documented. 
<quote>

Interestingly, this is a "Note" that contains a "MUST", which seems in contradiction with itself. Nevertheless, this then has the effect that xs:long, xs:positiveInteger, xs:unsignedLong and the like *MAY* be limited by the IntegerLiteral production in XPath, even though those very same types have, in the same XSD 1.0, normative limits that extend to 19 digits.

I feel as though this is a rather strange conflicting limit:

1) xs:decimal *MAY* be limited to 18 digits
2) xs:unsignedLong etc *MUST* be allowed a range to 2^64

I understand that these limits are imposed from the dependency on XSD 1.0 and not so much by XPath itself.

By extension, since we say in XPath that an IntegerLiteral must be treated as an xs:integer (as opposed to an xs:decimal), this inherits its restriction. xs:integer itself has no (extra) restriction. But further derived types like xs:unsignedLong have a restriction, are xs:integer and therefore xs:decimal but require more digits than a *MINIMALLY CONFORMING PROCESSOR* is required to support.

I don't know if this is a spec issue or not, but it seems that different definitions are in conflict with one another.

> The only place where 2^64 comes into play is for types such as xs:long; but
> we don't have xs:long literals in the language, you can only construct them
> from strings or xs:integer values, where the issue of "-" being an operator
> rather than part of the value doesn't arise.
True, with the caveat that xs:unsignedLong(18446744073709551615) (20 digits) is parsed as xs:integer from the literal, then cast to unsignedLong. But the first step may fail because of the limits-restriction. Obviously, this can be circumvented by writing xs:unsignedLong("18446744073709551615").

> So you may have an implementation issue here, but I don't think there is a
> spec issue.
I'm not sure, you may be right. Actually, given the current spec and this discussion, I think an implementation (minimally conforming) *MAY* throw FOAR0002 on xs:unsignedLong(18446744073709551615) and *MUST NOT* throw on xs:unsignedLong("18446744073709551615").

> those very same types have, in the same XSD 1.0, normative limits that extend 
> to 19 digits.
My list contained xs:positiveInteger, but this does not have an additional range imposed. And the ranges are in the value space, but can nevertheless not be represented once xs:decimal itself has an implementation-defined limitation.

From last weeks telcon, I was tasked with the request to either close this issue or to make a proposal.

I would like the WG to consider a non-mandatory Note along the following lines, probably best placed under description of Literal.

Note:
[XSD] poses a minimal requirement to support 18 decimal digits, which is too limited to express certain literals in limited range types such as xs:long. Implementations that impose such limits are encouraged to support integer literals up to +/- 18446744073709551616 (2^64) to allow parsing of expressions such as xs:long(123456789012345678). If parsing such literals is not possible due to said restrictions, processors MUST raise error FORG002 (or error ...), as explained in [link].

Here's a suggested redrafting of the proposed addition, to go beneath the paragraph starting "The value of a numeric literal..."

Note

The effect of the above rule is that in the case of an integer or decimal literal, a dynamic error FOAR0002 will generally be raised if the literal is outside the range of values supported by the implementation (other options are available: see [F+O section 4.2] for details.)

The XSD specification allows implementations to impose a limit (which must not be less than 18 digits) on the size of integer and decimal values. It is RECOMMENDED that any limit should be sufficient to accommodate the full range of values of built-in subtypes of xs:integer, such as xs:long and xs:unsignedLong: this requires 20 digits. Negative numbers such as the minimum value of xs:long (-9223372036854775808) are technically unary expressions rather than literals, but they SHOULD still be expressible.

At the meeting on 2016-03-22, the WG agreed to adopt the wording proposed in comment 8

Just a note: while this is marked "Fixed", the current internal WD of XP31 does *not* yet contain the proposed text, at least I couldn't find it.

Liam: I propose to resolve bug 29419 by removing “which must not be
less than 18 digits” and to allow editor’s discretion to add a note
about the host language defining a limit.

Action A-637-02 is tracking the implementation of the decision, and references comments #8 and #11.
Changing to resolved-fixed.

I have now made changes to the text, and hope that they fix the problem, but they are a little different from what was originally proposed, so I am reopening so that we will discuss it.

I was told to implement comment #8 and comment #11, which contradict each other somewhat.  I also noticed that this kind of limit is specified in the Conformance section of XQuery, not under literals.  So I added one sentence to the existing statement in Conformance:


<quote>
For the xs:decimal type, the maximum number of decimal digits (totalDigits facet) (MUST be at least 18). <add>This limit SHOULD be at least 20 digits in order to accommodate the full range of values of built-in subtypes of xs:integer, such as xs:long and xs:unsignedLong.</add>
</quote>

I think that fixes the problem.  Am I missing anything?

(In reply to Jonathan Robie from comment #13)
> <quote>
> For the xs:decimal type, the maximum number of decimal digits (totalDigits
> facet) (MUST be at least 18). <add>This limit SHOULD be at least 20 digits
> in order to accommodate the full range of values of built-in subtypes of
> xs:integer, such as xs:long and xs:unsignedLong.</add>
> </quote>
> 
> I think that fixes the problem.  Am I missing anything?
Are you sure about the "MUST be at least 18"? I thought we had established in an earlier discussion that it must be at least 18 if you support XSD 1.0, and at least 16 if you support XSD 1.1 (see https://www.w3.org/TR/xmlschema11-2/#partial-implementation, first bullet).

> I was told to implement comment #8 and comment #11, which contradict
I believe the contradiction came from comment #8 being accepted with the provision of comment #11 to remove the line on the actual limit (i.e., to *not* mention the 16 or 18 digit limit). 

I think we should either choose to mention both, or neither, but not one limit.

> So I added one sentence to the existing statement in Conformance:
I can't find your changes in the current internal draft. Do you mean to say you made only this change, or did you also incorporate MK's proposal of comment #8, mentioning the error?

Do we keep a change-log? I mean, is XP31 supposed to link and mention every bug that results in changes to the spec? I searched for 29419, but didn't see it.

(In reply to Abel Braaksma from comment #14)
> Do we keep a change-log? I mean, is XP31 supposed to link and mention every
> bug that results in changes to the spec? I searched for 29419, but didn't
> see it.
Looks like it was only applied to the XQuery 3.1 internal WD, not to the XPath 3.1 internal WD.

I couldn't find the mention of FOAR0002 in XQ31 either though. Only the line that was added to the conformance section 5.3, item 4.a.

Today, we decided to resolve this as follows.

For XPath, add the following note to the section on literals:

<add>
The XSD specification allows implementations to impose a limit (which
must not be less than 18 digits) on the size of integer and decimal
values. The full range of values of built-in subtypes of xs:integer,
such as xs:long and xs:unsignedLong, can be supported only if the
limit is 20 digits or higher. Negative numbers such as the minimum
value of xs:long (-9223372036854775808) are technically unary
expressions rather than literals, but implementations may prefer to
ensure that they are expressible.
</add>

For XQuery, add the following sentence:

<quote>
For the xs:decimal type, the maximum number of decimal digits (totalDigits facet) (MUST be at least 18). <add>This limit SHOULD be at least 20 digits in order to accommodate the full range of values of built-in subtypes of xs:integer, such as xs:long and xs:unsignedLong.</add>
</quote>

(In reply to Jonathan Robie from comment #16)
> (which must not be less than 18 digits) 
Please note (for XPath, not XQuery) that we also discussed *removing* this minimum limit as XSD 1.0 forces minimum of 18 digits and XSD 1.1 forces minimum of 16 digits.

Alternatively, we can reword something like as follows in the Note:

    (which must not be less than 18 digits for XPath processors supporting 
    [XSD 1.0] and 16 digits when supporting [XSD 1.1])

Note also, that the rewording of MSMcQ was on the second paragraph only. The first paragraph (without mandatory SHOULD etc) should be kept as is, from comment #8:

    "The effect of the above rule is that in the case of an integer or decimal
    literal, a dynamic error FOAR0002 will generally be raised if the literal is 
    outside the range of values supported by the implementation (other options 
    are available: see [F+O section 4.2] for details.)"

Minutes from the meeting are here. We kept the 18 digits recommendation - and it is in a NOTE, so it is not normative. I neglected to add the first paragraph, I will do so now.

https://lists.w3.org/Archives/Public/public-xsl-query/2016Jun/0031.html

CMSMcQ: Here is a version of the note in comment 8 without RECOMMENDED and
SHOULD:

     The effect of the above rule is that in the case of an integer or
     decimal literal, a dynamic error FOAR0002 will generally be raised if
     the literal is outside the range of values supported by the
     implementation (other options are available: see [F+O section 4.2] for
     details.)

     The XSD specification allows implementations to impose a limit (which
     must not be less than 18 digits) on the size of integer and decimal
     values. The full range of values of built-in subtypes of xs:integer,
     such as xs:long and xs:unsignedLong, can be supported only if the
     limit is 20 digits or higher. Negative numbers such as the minimum
     value of xs:long (-9223372036854775808) are technically unary
     expressions rather than literals, but implementations may prefer to
     ensure that they are expressible.

DECISION: add CMSMcQ's Note to the XPath spec.

I think it was missed in the minutes, it was a decision from a while back, captured in Bug 29552, comment #2, which says:

"Since XPath does not define these limits, the text that resolves Bug 29419 (which mentions the 18 digits) will be slightly and editorially adjusted. This will be tracked in that bug itself."

Its intent was to remove the mention of 18 digits.

But like you say, it is a Note anyway, and you could also argue, who in his right mind would implement XSD 1.1 after XSD 1.0 with less support (going from 18 to 16 digits). It seems ludicrous ;).

So let's leave it as is.

Thanks for adding the additional paragraph, the mention of the error condition was the main reason that caused this issue.

(In reply to Abel Braaksma from comment #19)
> I think it was missed in the minutes, it was a decision from a while back,
> captured in Bug 29552, comment #2, which says:
> 
> "Since XPath does not define these limits, the text that resolves Bug 29419
> (which mentions the 18 digits) will be slightly and editorially adjusted.
> This will be tracked in that bug itself."
> 
> Its intent was to remove the mention of 18 digits.

I asked this very question during the meeting, because I had conflicting instructions about whether it should say 18 digits or not. We discussed text that said 18 digits and accepted it after I asked that question.  I think we did decide to include it.

> But like you say, it is a Note anyway, and you could also argue, who in his
> right mind would implement XSD 1.1 after XSD 1.0 with less support (going
> from 18 to 16 digits). It seems ludicrous ;).

Yes, it does sound ludicrous.

> So let's leave it as is.

OK. Thanks.