Thank you for the pull request, @javagl!

✅ We can confirm we have a CLA on file for you.

It's never as easy as it looks...

The last point may make it necessary to store the "effective" offset/scale inside the PickedMetadataInfo.

This is not possible. At the point where the actual value is obtained (namely, inside the shader), it is not longer possible to know where any offset/scale came from to begin with. So it's possible to "undo" the value transform in the shader (with some quirks). And it is possible to write that value in [0,1] into the color (ending up as [0, 255] in the frame buffer). But it is not possible to convert this back to the right value after reading it from the frame buffer, because at that point, we don't know whether the "value transform" that was applied came from a 'class property' or from the 'property texture/attribute property'.

There might be solutions to that, but all approaches that I can think of right now are ... complicated, to say the least. We certainly don't want to replicate the whole logic of value transforms (and things like KHR_texture_transform) inside the metadata picking shader function ... 🤔

The solution to the previous point turned out to be not toooo complicated.

When the picking starts, it will obtain the StructuralMetadata from the object, and then either obtain the PropertyTextureProperty or the PropertyAttributeProperty that corresponds to the picked metadata. (This may require some more thought - e.g. could there be an attribute and a texture that describe the same property...? Just thinking about corner cases and limitations here...).

This object always offers the offset/scale, in its final form - i.e. even when no offset/scale have been defined, this will be filled with default values, but also handle possible overrides from the class property. This object is then put into the PickedMetadataInfo, and used

1. for creating the shader code that maps the metadata values to [0, 1] for the picking
2. for converting the value in [0,255] from the frame buffer back into its original range

I started adding the corresponding tests. In that process I stumbled over an issue where the shader compilation bails out for normalized UINT8 array properties in property textures with offset/scale being defined in the class property, and it looks like this is independent of the picking (and therefore may have to be moved into a dedicated issue), but I have to verify that.

I'd also like to do more tests with property attributes (because these are currently not covered at all). Since the picking only relies on the metadata, and this does not even "know where it comes from" (attribute or texture), this might just work transparently. But now there is some code that explicitly has to handle both textures and attributes, so that should be covered by specs as well.

So, I did try out some of this with property attributes. Or rather tried to try it. There are more issues with that, and see the handling of property attributes for metadata picking as a part of #12225 , likely to be spun out into a dedicated issue. (Handling of arbitrary types being moved through the frame buffer does raise quite a few questions).

So I'll consider this as ready for review, given that it contains several spec test cases, and I tried it out with the data from #12232 (where this originally came up), and there, the current state seems to work as expected.

I did have to re-run some CI steps a few times. We do have some flaky tests there...

A small update: As mentioned above, pricking arbitrary property attributes is not supported until now. It is not entirely clear what the expected behavior should be when someone tries to pick property attributes. But in the previous state, it could happen that this caused an error to be thrown (specifically, when the property attribute type was one of the types that could not trivially be transported through the picking frame buffer, like a FLOAT32 or ENUM).

The last commit handles this: The function that determines the "picked metadata property" now only looks for property texture properties (and no longer for property attribute properties), and bails out early if no property texture property is found. A // Note... was added to the function, pointing to #12225 , which keeps track of this.