Open Geospatial Consortium |
Submission Date: 2022-06-29 |
Approval Date: 2023-05-08 |
Publication Date: 2023-07-14 |
External identifier of this OGC® document: http://www.opengis.net/doc/is/COG/1.0 |
Internal reference number of this OGC® document: 21-026 |
Version: 1.0 |
Category: OGC® Implementation Standard |
Editor: Joan Maso |
OGC Cloud Optimized GeoTIFF Standard |
Copyright notice |
Copyright © 2023 Open Geospatial Consortium |
To obtain additional rights of use, visit http://www.opengeospatial.org/legal/ |
Warning |
This document is an OGC Member approved international standard. This document is available on a royalty free, non-discriminatory basis. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation.
Document type: OGC® Standard |
Document subtype: |
Document stage: Approved for public release |
Document language: English |
License Agreement
Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.
If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.
THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD.
THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.
This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.
Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications. This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.
- 1. Scope
- 2. Conformance
- 3. References
- 4. Terms and Definitions
- 5. Conventions
- 6. Overview
- 7. GeoTIFF format requirements
- 8. HTTP range support requirements
- 9. Media Types for any data encoding(s)
- Annex A: Conformance Class Abstract Test Suite (Normative)
- Annex B: Revision History
- Annex C: Bibliography
i. Abstract
The Cloud Optimized GeoTIFF (COG) relies on two characteristics of the TIFF v6 format (tiles and reduced resolution subfiles), GeoTIFF keys for georeference, and the HTTP range, which allows for efficient downloading of parts of imagery and grid coverage data on the web and to make fast data visualization of TIFF or BigTIFF files and fast geospatial processing workflows possible. COG-aware applications can download only the information they need to visualize or process the data on the web. Numerous remote sensing datasets are available in cloud storage facilities that can benefit from optimized visualization and processing. This standard formalizes the requirements for a TIFF file to become a COG file and for the HTTP server to make COG files available in a fast fashion on the web.
The key work for crafting this OGC Standard was undertaken in the Open-Earth-Monitor Cyberinfrastructure (OEMC) project, which received funding from the European Union’s Horizon Europe research and innovation program under grant agreement number 101059548 and in the All Data 4 Green Deal - An Integrated, FAIR Approach for the Common European Data Space (AD4GD) project, which received funding from the European Union’s Horizon Europe research and innovation program under grant agreement number 101061001.
ii. Keywords
The following are keywords to be used by search engines and document catalogues.
ogcdoc, OGC document, Cloud, GeoTIFF, Imagery, Web, COG
iii. Preface
COG was previously defined by a community of users with the intention of improving the distribution of remote sensing images on the cloud. This document respects the decisions taken and conventions adopted by the community and formalizes them in standard requirements and recommendations. The Standard name still reflects this original purpose: a format optimized for the cloud. Similar principles are applied to other emerging formats such as Zarr (https://zarr.readthedocs.io/en/stable/) and Cloud Optimized Point Cloud (https://copc.io/).
This document depends on the TIFF specification and the OGC GeoTIFF Standard. For big files, it may depend on the BigTIFF specification. The Standard takes advantage of some existing characteristics of the TIFF specification and the existing HTTP range specification and does not modify them in anyway.
iv. Security Considerations
COG mainly describes a data format. Security considerations for implementations utilizing COG are in the domain of the implementing application, deployment platform, operating system, networking environment, and web browsers.
In general, this Standard does not place constraints on application, platform, operating system level, network, or web browser security. However, this Standard has a requirements class that requires "HTTP range." Attention has been made to also allow secure variants of HTTP such as HTTPS and still be able to mix data from different origins in web browsers using HTTP range. In that respect, considerations to support Cross Origin Resource Sharing (CORS) are done in the document. In particular, servers are encouraged to advertise the support for HTTP ranges in the Control-Allow-Headers header to allow for HTTPS CORS.
|
Note
|
The security measures discussed in the following sections only allow various kinds of authentication under HTTPS. HTTPS only provides confidentiality or encryption at the protocol level. Confidentiality or encryption at the data level is not discussed by this Standard. |
v. Submitting organizations
The following organizations submitted this Document to the Open Geospatial Consortium (OGC):
-
UAB-CREAF
-
Planet Labs PBC
-
Spatialys
-
Development Seed
vi. Submitters
All questions regarding this submission should be directed to the editor or the submitters:
Name |
Affiliation |
Joan Maso |
UAB-CREAF |
Quinn Scripter |
Planet Labs PBC |
Even Rouault |
Spatialys |
Vincent Sarago |
Development Seed |
1. Scope
This document is an OGC Standard that specifies how TIFF files can be organized in a way that favors the extraction of convenient parts of the data at the needed resolution for visualization or analysis purposes. It also specifies how to use HTTP (or HTTPS) to communicate only the part of information needed without downloading the complete file.
This document depends on the TIFF specification and the OGC GeoTIFF Standard. For big files, it also may depend on the BigTIFF specification. The Standard takes advantage of some existing characteristics of the TIFF specification and the existing HTTP range RFC and does not modify them in anyway.
2. Conformance
This Standard defines 5 conformance classes.
The following are the conformance classes that target TIFF Encoders.
This class defines what characteristics of the TIFF (or BigTIFF) format should be used. Used alone responds to use cases doing analysis on images that can be processed by tiles. This images can be geospatial or other (e.g. medical scans)
This class defines the overview characteristics of the TIFF (or BigTIFF) format that should be used. Used alone responds to use cases doing data visualization on images. This images can be geospatial or other (e.g., medical scans).
This class defines which GeoKeys to georeference the data should be used. It restricts the applicability to geospatial images.
This class defines more rules on how the images should be built to accelerate visualization in the cloud and web environments in general. This approach is what has made COG format popular.
The following conformance classes target web servers.
This class defines how to use COG in the cloud and the web to allow downloading only a fragment of the data.
Conformance with this Standard shall be checked using all the relevant tests specified in Annex A (normative) of this document. The framework, concepts, and methodology for testing, and the criteria to be achieved to claim conformance are specified in the OGC Compliance Testing Policies and Procedures and the OGC Compliance Testing web site.
In order to conform to this OGC Standard, a software implementation shall choose to implement: * Any one or more of the conformance levels specified in Annex A (normative).
All requirements-classes and conformance-classes described in this document are owned by the Standard(s) identified.
|
Note
|
Currently there is no way to list the conformance classes that a GeoTIFF file conforms to, so the COG is not specifying how to do so. |
3. References
The following normative documents contain provisions that, through reference in this text, constitute provisions of this document. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest edition of the normative document referred to applies.
Adobe Development Association. TIFF Revision 6.0 Final — June 3, (1992) https://www.itu.int/itudoc/itu-t/com16/tiff-fx/docs/tiff6.pdf
Fielding, R., Nottingham, M. and Reschke, J.: HTTP Semantics RFC 9110, Section 14. Range Requests. Internet Engineering Task Force (IETF). ISSN: 2070-1721 (June 2022) https://www.rfc-editor.org/rfc/rfc9110#name-range-requests
OGC: OGC GeoTIFF Standard. OGC 19-008r4. (2019) http://docs.opengeospatial.org/is/19-008r4/19-008r4.html
van Damme J.: The BigTIFF File Format. https://www.awaresystems.be/imaging/tiff/bigtiff.html (No normative reference)
4. Terms and Definitions
This document used the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this standard and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the 'ModSpec'. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
For the purposes of this document, the following additional terms and definitions apply.
4.1. cloud computing
on-demand availability of computer system resources, especially data storage (cloud storage) and computing power, without direct active management by the user [Wikipedia]
4.2. geokey
key in a GeoTIFF equivalent in function to a TIFF tag, but using a different offset mechanism
4.3. GeoTIFF
standard for storing georeference and geocoding information in a TIFF 6.0 compliant raster file or in a BigTIFF [GeoTIFF Format Specification 1.0, modified]
4.4. overview
downsampled versions of the same image.
|
Note
|
There is no mention of the term overview in the TIFF v6 specification. This standard favors the use of the expression Reduced-Resolution Subfile to talk about an overview |
4.5. tag
packet of numerical or ASCII values, which have a numerical TIFF "Tag" identifier indicating their information content
4.6. tile
geometric shape with known properties that may or may not be the result of a tiling (tessellation) process. A tile consists of a single connected "piece" (topological disc) without "holes" or "lines" [OGC 19-014r3]+
In the context of a 2D tile matrix, a tile is one of the rectangular regions of space, which can be uniquely identified by an integer row and column, making up the tile matrix.
In the context of a geospatial data tile set, a tile contains data for such a partition of space as part of an overall set of tiles for that tiled geospatial data. [OGC 2DTMS]
5. Conventions
This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, use of XML schema, or special notes regarding how to read the document.
5.1. Abbreviations
| BigTIFF |
Big Tagged Image File Format |
| COG |
Cloud Optimized GeoTIFF |
| CORS |
Cross-Origin Resource Sharing |
| EPSG |
Registry of CRS descriptions maintained by the International Association of Oil and Gas Producers (IOGP) formerly known as the "European Petroleum Survey Group (EPSG)" and currently used only as an acronym |
| HTTP |
Hypertext Transfer Protocol |
| HTTPS |
Hypertext Transfer Protocol Secure |
| IFD |
Image File Directory |
| TIFF |
Tagged Image File Format |
6. Overview
Traditionally, high resolution imagery over the web requires that big files have to be entirely downloaded to the client before they can be analyzed or visualized. In the past, these files were provided by servers in the provider premises. Depending on the circumstances, this process can require considerable download time thus preventing the creation of responsive real time applications. Nowadays, cloud services offer almost unlimited storage, but the cost of the service depends on the intensity of use. The solution to the problems specified here is commonly called Cloud Optimized GeoTIFF (COG) that is based on a particular usage of one of the most popular formats in the world: the TIFF format. While the solution is popular in the cloud, the solution is equally effective in the web in general, and it is some times named as "web optimized."
The TIFF format was created by the Aldus Corporation for use in desktop publishing. Aldus released the last version of the TIFF specification in 1992 (v. 6.0), subsequently only updated with an Adobe Systems copyright after the Adobe acquired Aldus in 1994. Several Aldus or Adobe technical notes have been published with minor extensions to the format, and several specifications have been based on TIFF 6.0, including TIFF/EP (ISO 12234-2), TIFF/IT (ISO 12639), TIFF-F (RFC 2306) and TIFF-FX (RFC 3949), GeoTIFF (OGC 19-008r4, v1.1) and BigTIFF.
TIFF is a flexible, adaptable file format for handling images and data within a single file, by including the header tags (size, definition, image-data arrangement, applied image compression) that provide metadata about the images. The ability to store image data in a lossless format makes a TIFF file a useful image archive. TIFF can be used to store grey scale, color or RGB images as well as integer of floating point data, making it ideal as a support for storing the rangeset of a 2D grid coverage data.
To improve TIFF performance over the web, COG relies on two characteristics of the TIFF v6 format, the georeference GeoTIFF keys and a relatively unused HTTP property. This way, COG allows for efficient downloading of parts of imagery and grid coverage data on the web, enables fast data visualization, and facilitates faster geospatial processing workflows. This particular type of TIFF has been recently used to set up a large series of remote sensing images on cloud providers repositories (e.g., Amazon Web Services), enabling cloud processing at lower traffic. In fact, COG-aware software can request just the portions of data that it needs, improving access time and bandwidth.
COG is based on the GeoTIFF standard and does not introduce additional capabilities to the TIFF v6. As such, legacy software should be able to read COG files with no additional modifications. However, legacy software may not be able to take advantage of the streaming (understood here as partial download) capabilities, but still can easily download the whole file and read it.
The amount of data available for geospatial analytics has increased considerably in recent years. Therefore, downloading the data into a single computer is often not feasible. Data producers that provide data in the COG format can help decrease how much data is downloaded and copied. This is because online software systems do not need to keep their own copy of the data for efficient access. New online software can access the content efficiently, while old versions can still download the data completely. This avoids the need to have two copies of the file: one for fast access and another for download purposes.
COG relies on two complementary approaches already available in the existing standards to achieve its goal:
-
The first is the ability of GeoTIFF to store the raw pixels of the image organized in an efficient way using tiles and overviews; and
-
The second is HTTP GET Range request, that let web clients request just the portions of a file that they need.
Using the first approach, COG organizes the GeoTIFF so the latter requests can easily select and get the parts of the file that are useful for processing.
6.1. Efficient organization of data in a TIFF file
The Tiling and Reduced-Resolution Subfiles (sometimes called overviews) in the GeoTIFF format supports the necessary structure for COG files so that the HTTP GET Range queries can request just the part of the file that is relevant.
Reduced-Resolution Subfiles come into play when the client wants to render a quick image of the whole or a big part of the area represented in the file. Instead of downloading every pixel, the software can just request a smaller, already created, lower resolution version. The structure of the COG file on an HTTP Range supporting web server enables client software to easily find and download just the part of the whole file that is needed.
Tiles come into play when some small area of the overall extent of the COG file needs to be processed or visualized. This could be part of a reduced-resolution subfile, or it could be at full resolution. Tile organization makes all the relevant bytes of an area (a tile) to be in the same part of the file, so the software can use HTTP GET Range request to get only the tiles it needs.
The number of requirements classes can be confusing and respond to different use cases. The Efficient organization of TIFF into COG is handled by the http://www.opengis.net/spec/COG/1.0/req/optimized_geotiff conformance class and its dependencies.
6.2. Relation to OGC Tile Set Standards
The combined use of tiles and resolution levels is not new in OGC Standards. In fact the OGC Two-dimensional Tile Matrix Set Standard (and the older OGC WMTS 1.0) use the same approach. However, the OGC API - Tiles Standard and the older WMTS 1.0 Standard require either a service to be installed in the web server provider or thousands of pre-generated independent files (one for each tile) to be created. None of this is necessary in the COG approach as most of the modern web services natively support HTTP range.
7. GeoTIFF format requirements
A COG file is a file that conforms to a subset of the TIFF specification (or the BigTIFF specification) and adheres to the GeoTIFF tag requirements. A COG uses two main organization techniques available in the TIFF version 6.0 specification: Tiling and Reduced-Resolution Subfiles (sometimes called Overviews). The tiled data can also be compressed for more efficient passage online. In addition, a COG file uses the GeoTIFF conventions for storing the relevant geospatial metadata. For convenience, Tiling and Reduced-Resolution Subfiles are presented in two independent conformance classes.
7.1. Requirement Class GeoTIFF Tiles
Requirements Class |
|
Target type |
TIFF Encoder |
Dependency |
https://www.itu.int/itudoc/itu-t/com16/tiff-fx/docs/tiff6.pdf |
Dependency |
|
7.1.1. Basic format
A COG file is a TIFF or a BigTiFF file.
Requirement 1 |
/req/geotiff-format/use-geotiff |
A |
If the resulting COG file is bigger than 4 GByte, the COG file SHALL follow the BigTIFF standard that modifies the TIFF version 6.0 standard |
Recommendation 1 |
/rec/geotiff-format/use-geotiff |
A |
If the resulting COG file is smaller than 4 GByte a COG file SHOULD follow the TIFF version 6.0 standard |
7.1.2. Tiles
In the context for a TIFF file, Tiling is a strategy for dividing the content in the TIFF file differently than using the classical Strips. In the Strips approach the data are organized into sequences of lines (rows) while tiling creates a number of internal rectangular tiles stored in the actual image. Strips divide the content of an image vertically (rows) but not horizontally (columns). With Tiling, a much quicker access to a certain area or two dimensional bounding box is possible as the relevant data is closer in the file and the portion of bytes that needs to be read is smaller than in the strips approach.
Requirement 2 |
/req/geotiff-format/tiling |
A |
All Image File Directories (IFD) in a COG file SHALL be organized into tiles (instead of strips) as described in section 15 of the TIFF version 6.0 specification |
|
Note
|
The ability to have multiple Image File Directories (IFD) becomes important when the COG has also overviews. See the next requirements class for more details. |
Tiles, as defined in the TIFF version 6.0 specification, can be mapped to the ones defined in the OGC Two Dimensional Tile Matrix Set Standard (2D-TMS). For example in 2D-TMS, the TIFF 6.0 forces all tiles to be of the same size. This is possible with the introduction of the concept of padding: if necessary extra blank rows or columns are added to the right-most and bottom-most tile to make them the same shape as other tiles. However, the naming of the TIFF tags used version 6.0 and the property names used in the 2D-TMS differ. The following table provides a mapping between the two standards.
| OGC 2D-TMS | TIFF v. 6.0 | Definition |
|---|---|---|
TileWidth |
TileWidth |
The tile width in pixels. The number of columns in each tile |
TileHeight |
TileLength |
The tile height in pixels. The number of rows in each tile |
MatrixWidth |
TilesAcross |
Number of tiles in the width direction |
MatrixHeight |
TilesDown |
Number of tiles in the height direction |
Please note that the TIFF 6.0 specification imposes the requirement that TileWidth and TileLength must be a multiple of 16. The specification argues that this restriction improves performance in some graphics environments and enhances compatibility with compression schemes such as JPEG. There is no such restriction in the OGC 2D-TMS.
7.1.3. Compression
Compression of the bytes is a general best practice for enabling software to quickly access data, in particular over the network. The combination of compression with the HTTP GET Range requests maximizes efficiency. HTTP GET Range will be standardized in another requirements class.
Recommendation 2 |
/rec/geotiff-format/rec-compression |
A |
In a COG file, tiled data should be compressed efficiently using the original compression algorithm indicated in TIFF v6 (LZW or JPEG) or a more modern image compression algorithm. Be aware that certain compressions may not be supported by specific TIFF readers.. |
|
Note
|
While LZW and JPEG are the two image compressions mentioned in TIFF v6, there are many other image-based compressions that have become popular since the TIFF v6 original release in 1992. Other possible compressions are: ZSTD, DEFLATE, WEBP, LERC, LERC_ZSTD, etc. In particular, modern lossless compressions are better than the initially proposed LZW. New compressions are constantly being released so this recommendation cannot limit compression to a fixed list. See https://www.awaresystems.be/imaging/tiff/tifftags/compression.html for more details. |
7.1.4. Planar Configuration considerations
When more than one component is encoded in a TIFF file, the TIFF provides two possibilities.
-
The component values for each pixel are stored contiguously. This is marked in the file as PlanarConfiguration=Contig (a.k.a. Chunky format, value 1). This is a common arrangement for RGB combinations of bands. The data is stored as RGBRGBRGB… (this arrangement is also known as Band Interleaved by Pixel, BIP).
-
The components are stored in separate component planes. This is marked in the file as PlanarConfiguration=Separate (a.k.a. Planar format, value 2). This is the common arrangement for the bands of a multispectral image (this arrangement is also known as Band Sequential, BSQ).
|
Note
|
(extracted from the TIFF 6.0 specification) PlanarConfiguration=2 is not currently [written in 1992] in widespread use and it is not recommended for general interchange. |
Following this note, most common implementations of the COG encoders (e.g., GDAL) use PlanarConfiguration=1 (see https://gdal.org/drivers/raster/cog.html#high-level).
7.2. Requirement Class GeoTIFF Overviews
Requirements Class |
|
Target type |
TIFF encoder |
Dependency |
|
7.2.1. Requirement Reduced-Resolution Subfiles
Reduced-Resolution Subfiles (a.k.a Overviews) are down-sampled versions of the same image included in the same TIFF file. This means that an overview is a zoomed out version from the original image. It has less detail but is also smaller. For visualization purposes or for analytical processes that do not require full resolution, a COG can provide Reduced-Resolution Subfiles that match different scale denominators or cell sizes required by clients. Reduced-Resolution Subfiles increase the size of the file but also increase performance.
|
Note
|
The general description of the COG use the concept of overviews. Actually, there is nothing called overviews in the TIFF version 6.0 specification. Instead, the TIFF version 6.0 describes how a TIFF file can be composed of one or more images, each one stored in an Image File Directories (IFD). Some IFDs may contain reduced-resolution subfiles. This is the reason why this standard favors the use of the expression Reduced-Resolution Subfiles to talk about overviews. |
There may be more than one IFD in a TIFF file and each IFD indicates the offset of the next IFD sits in the file (or 0 if no other IFD is available). Each IFD defines a subfile. Having images as subfiles has several applications. The NewSubfileType entry specifies several types of subfiles, among which we distinguish: - IFD with a NewSubfileType tag whose bit 0 is not set, or without a NewSubfileType tag, to mean a full-resolution image data; and - IFD with a NewSubfileType tag whose bit 0 is set, to mean a reduced-resolution image data of the full resolution one.
Requirement 3 |
/req/geotiff-overviews/overviews |
A |
A COG file SHALL have at least one IFD with a NewSubfileType tag whose bit 0 is not set, or without a NewSubfileType tag (full-resolution image data). |
B |
An IFD with a NewSubfileType tag whose bit 0 is not set, or without a NewSubfileType tag SHALL have an offset to the next IFD that will point to an IFD with a NewSubfileType tag whose bit 0 is set (reduced-resolution image data) if there is one, or an offset to the next full-resolution image data (e.g., another band or a mask) or an offset 0 if there is no other image. |
C |
This IFD with a NewSubfileType tag whose bit 0 is set SHALL have an Image Width an Image Height inferior than the previous IFD. |
D |
If other reduced resolution images are needed, a IFD with a NewSubfileType tag whose bit 0 is set SHALL have an offset to the next IFD that will point to an IFD with a NewSubfileType tag whose bit 0 is set (reduced-resolution image data). |
E |
This additional IFD with a NewSubfileType tag whose bit 0 is set SHALL have an Image Width an Image Height inferior than the previous IFD. |
F |
When no additional reduced-resolution image data is available for this image, the IFD SHALL have an offset to the next IFD pointing to the next full-resolution image data (e.g. another band or a mask) or an offset 0 if there is no other image. |
|
Note
|
The wording of this requirement is complex due to the possible presence of several images (e.g. bands or masks) in the file. In essence, it says that for a single image, the full resolution image IFD should be immediately linked to its overview IFDs. If there is more than one image in the file, the last overview IFD of the previous image will linked to the full resolution next image IFD. |
|
Note
|
The presence of reduced-resolution subfiles in a COG file is optional. Only the COG files that conform to this conformance class are forced to have reduced-resolution subfiles. |
The TIFF specification is very flexible with how IFDs are positioned in the file; the only requirement is that the last 4 bytes of the IFD point to the byte offset of the next. So a totally valid TIFF could contain an optimized image where the full resolution image IFD is at the start of the file, the IFD of the first overview is at the end of the file, and the IFD of the second overview is at the middle of the file. Reading the header of an TIFF like this would require making three consecutive requests.
To increase performance, the IFD should be arranged in a particular order expressed in the following recommendation. In the following recommendation, the "order" implies both the order of the IFD links as well as the position in the file implied by the offset.
The advantage of this IFD order is allowing readers to fetch the entire TIFF header (including all IFDs for all subfiles) in a single request upon opening the file for the first time. This reduces the number of file accesses needed to read it and improves cache-ability of the TIFF header for repeat requests to the same image.
For example, when the file is accessed by HTTP range, available from a S3 backend, the latency per HTTP request is about ~20-30ms that will be added other latencies due to data locality and internet transfer (~150ms is a common value). Fetching the header of a COG over 3 consecutive HTTP requests adds ~540ms of latency which is large enough to be perceived by the user, giving a feeling of slow/clunky display especially if rendered in a web browser. Minimizing the number of requests needed to read a COG file is more important over slow internet connections where latencies are exacerbated.
Recommendation 3 |
/rec/geotiff-overviews/ifd-order |
A |
In a COG file, and after the TIFF/BigTIFF header/signature and pointer to first IFD (and an eventual ghost area) sections SHOULD follow the following order: 1. IFD of the full resolution image, followed by TIFF tags values, excluding the TileOffsets and TileByteCounts arrays 2. IFD of the mask of the full resolution image, if present, followed by TIFF tags values, excluding the TileOffsets and TileByteCounts arrays 3. IFD of the first (largest in dimensions) Reduced-Resolution Subfile, if present 4. …(there may be several sections for IFD of intermediate dimensions) 5. IFD of the last (smallest) Reduced-Resolution Subfile, if present 6. IFD of the first (largest in dimensions) Reduced-Resolution Subfile of the mask, if present 7. …(there may be several sections or IFD of intermediate dimensions for the mask) 8. IFD of the last (smallest) Reduced-Resolution Subfile of the mask, if present TileOffsets and TileByteCounts arrays of the above IFDs 9. tile data of the smallest Reduced-Resolution Subfile, if present (with each tile followed by the corresponding tile of mask data, if present), with leader and trailer bytes 10. …(there may be several tile data sections of intermediate resolutions) 11. tile data of the largest Reduced-Resolution Subfile, if present (interleaved with mask data if present) 12. tile data of the full resolution image, if present (interleaved with corresponding mask data if present) |
|
Note
|
There can be more than one full-resolution image data in the file. In this case, each full-resolution image data will be contiguous to its reduced-resolution subfiles and their IFDs. |
7.3. Requirement Class GeoTIFF Keys
The GeoTIFF standard defines a mechanism to add GeoTIFF keys to a TIFF file. The main purpose of these keys is to add a geospatial reference to the data stored in the TIFF file. In a COG, GeoTIFF keys are a fundamental part of the standard as most of the COG files are geospatially-referenced TIFFs.
Requirements Class |
|
Target type |
GeoTIFF Encoder |
Dependency |
|
Dependency |
|
7.3.1. Requirement GeoTIFF
A COG uses GeoTIFF to document the metadata about the TIFF file, including the georeference of the full resolution images IFDs. The georeference of the Reduced-Resolution Subfiles is derived from the georeference of the full resolution images IFDs.
Requirement 4 |
/req/geotiff-keys/basic-metadata-format |
A |
A COG file SHALL include and encode geospatial metadata in the GeoTIFF format following the GeoTIFF standard |
The OGC Naming Authority (OGC-NA) maintains a register of TIFF tags relevant for the geospatial community (most of them defined in the GeoTIFF standard) with the URL http://www.opengis.net/def/geotiff-tag. The COG standard does neither defines COG specific TIFF tags nor GeoTIFF keys. In the future it may be necessary for the communities that use COG to register tags for their specialist use cases. It is also possible that they reuse the existing GeoTIFF tags to define new GeoTIFF keys. Such tags or keys could then be organized into groups that deal with the needs of a particular domain.
7.3.2. Requirement Georeference Keys
There is a geometrical relation between the reduced-resolution subfiles and the corresponding full-resolution subfile. Only full-resolution subfiles are required to have GeoTIFF keys.
Requirement 5 |
/req/geotiff-keys/georeference |
A |
In a COG file, the IFD Subfiles of full resolution (a.k.a IFDs whose bit 0 of the value of the NewSubfileType tag is not set, or NewSubfileType absent) SHALL contain georeference data using ModelTiepointTag, ModelPixelScaleTag and GeoKeyDirectoryTag tags. |
All linked reduced-resolution subfiles (a.k.a IFDs whose bit 0 of the value of the NewSubfileType tag is set) that are linked to a full resolution IFD (i.e., an IFD whose bit 0 of the value of the NewSubfileType tag is not set, or NewSubfileType absent) share the set of georeference keys from the full resolution IFD. In practice this means that they have a common CRS and extent information, and differ only by their resolution. This means that there is the assumption that all related subfiles share the same point of origin for the top left corner or the image.
Requirement 6 |
/req/geotiff-keys/point-of-origin |
A |
In a COG file all reduced-resolution subfiles SHALL not contain georeference data and use the one in the corresponding full resolution subfile. As a consequence, COG readers SHALL consider that all reduced-resolution subfiles share the same point of origin for the top left corner of the image as the full resolution subfile |
The pixel size in the first dimension of a reduced-resolution subfile can be calculated by multiplying the full-resolution subfile pixel size by the ImageWidth ratio between the full-resolution and the reduced-resolution. The pixel size in the second dimension of a reduced-resolution subfile can be calculated by multiplying the full-resolution subfile pixel size by the ImageHeight ratio between the full-resolution and the reduced-resolution.
For the IFD reduced-resolution subfiles linked to IFD0, IFDn_PixelScaleX and IFDn_PixelScaleY can be calculated like this:
IFDn_PixelScaleX=IFD0_PixelScaleX*IFD0_ImageWidth/IFDn_ImageWidth IFDn_PixelScaleY=IFD0_PixelScaleY*IFD0_ImageHeight/IFDn_ImageHeight
For a georeferenced file with a projected CRS that is fully defined in the EPSG database there is no need to define the base Geographic CRS, geodetic datum, etc. in the TIFF file. In these cases, the following keys in the IFD0 are sufficient.
ImageWidth = 20111 ImageHeight = 16882 ModelTiepointTag = (0 0 0 187334 3255440 0) ModelPixelScaleTag = (30 30 0) GeoKeyDirectoryTag: - GTModelTypeGeoKey = 1 (ModelTypeProjected 2D) - GTRasterTypeGeoKey = 1 (RasterPixelIsArea) - ProjectedCRSGeoKey = 32628 (Projected 2D CRS WGS 84 / UTM zone 28N)
This example corresponds to imagery over the Canary Islands with a resolution of 30 meters per pixel. There is a grid intersection line in the image at pixel location (0, 0).
The 2 first numbers in ModelTiepointTag correspond to the projected coordinate reference system easting/northing of (X:187334, Y:3255440) (the numbers 4th and 5th in ModelTiepointTag) in the EPSG:32628 (WGS 84 / UTM zone 28N).
The georeference of the reduced-resolution subfiles (a.k.a. IFDs whose bit 0 of the value of the NewSubfileType tag is set)
that are linked to this IFD0 are supposed to be the same except where ModelPixelScaleTag
represents meters per pixel (pixel size).
In this example, IFD0 has ImageWidth=20111 and ImageHeigth=16882 while IFD1 has ImageWidth=10056 and ImageHeigth=8441, so
IFD1_PixelScaleX and IFD1_PixelScaleY can be calculated as:
30*20111/10056=59.997 and 30*16882/8441=60.
As we can see in the example, tiles can easily result in non-square-pixels; a situation that should be handled by clients combining the data with other imagery that have square pixels.
7.4. Requirement Class Optimized GeoTIFF
Requirements Class |
|
Target type |
TIFF encoder |
Dependency |
|
Dependency |
|
In the Requirement Class GeoTIFF Overviews requirement class, there is no guidance on how many reduced resolution subfiles a given COG should have. In a TIFF file with a big image, a COG with zero or one overview, can be rendered on a web client but it could be too slow to process and show in a client app. This requirement class provides more restrictions on how the COG should be build to accelerate visualization in web clients. It also adds the need for GeoTIFF as a dependency to Requirement Class GeoTIFF Keys. The implementation of this conformance class and its dependencies generates what has been commonly called COG.
7.4.1. Requirement Small Tiles
For optimized visualization, tiles should be small in their number of pixels, ideally a number smaller than the common number of pixels of a screen view port.
Requirement 7 |
/req/optimized_geotiff/small-sizes |
A |
A COG file SHALL have square tiles (TileWidth=TileLength) |
B |
A COG file SHALL define TileWidth and TileLength as a number smaller than or equal to the common screen viewport. |
|
Note
|
While years ago 256x256 was a recommended size, today common sizes are 512x512 or 1024x1024. |
7.4.2. Requirement Reduced-Resolution Subfiles Number
The number of reduced resolution subfiles is defined as a function of the number of pixels of the original image by creating progressive reduced resolution subfiles with less tiles until the number of tiles very small.
Requirement 8 |
/req/optimized_geotiff/number |
A |
A COG file SHALL contain reduced resolution subfiles each one reducing the resolution by a minimum factor of 2 and a maximum factor of 10 from the previous one. |
B |
In a COG file, the last reduced resolution subfile (the one with bigger pixel size) SHALL have TilesAcross or TilesDown equal to one. |
It is a common practice for visualization purposes that pixel sizes are generated as a power of 2. By selecting the number columns and rows of the original image also as a power of 2, each reduced resolution subfile can be generated from the previous smaller resolution by averaging the values of 4 contiguous pixels in one of the new subfile.
Recommendation 4 |
/rec/optimized-geotiff/size-and-numbers |
A |
A COG file SHOULD define TileWidth and TileLength as a power of two. |
B |
All tiles in the file SHUOLD have the same TileWidth and TileLength. Recomended values are 256, 512 or 1024 |
C |
The size of a pixel (a.k.a resolution) r of the n reduced resolution subfile SHOULD be generated by this formula: rn=2n r0 (where r0 is the size of a pixel of the original image). |
8. HTTP range support requirements
HTTP Version 1.1 introduced a range header in the GET requests that supports requesting only a fragment of a resource. If the server advertises "Accept-Ranges: bytes" in its response headers of a HEAD or GET request, the server is telling the client that bytes of data can be requested in parts, in separated requests. The client can request just the bytes that it needs from the server at any time. In a web environment, this is very useful for serving files such as video. By using range requests, clients do not need to download the entire file to begin playing it. In the case of COG, HTTP range is useful to get only the tiles needed to be processed or shown. This is done by getting the headers and IFDs of the TIFF file first and using this information to determine the conversion between tile indices to byte ranges containing the needed tiles. A client trying to show a COG file on the screen can request the resolutions needed and only the tiles needed to cover the screen. Once the user moves or pans, other GET range requests will get the new needed resolutions and tiles.
8.1. Requirement Class HTTP range
Requirements Class |
|
Target type |
Web server |
Dependency |
|
Dependency |
|
8.1.1. Requirement range
Requirement 10 |
/req/http-range/range |
A |
A HTTP or HTTPS server SHALL support serving TIFF files |
B |
A HTTP or HTTPS server SHALL support HTTP range |
C |
A HTTP or HTTPS server shall support bytes ranges. |
|
Note
|
Some web services are only capable to serve TIFF files if the MIME type (image/tiff) is registered |
|
Note
|
HTTP servers serving COG files advertise the support to bytes ranges by adding this line in the headers: Access-Ranges: bytes of the response
|
The use of range in web browsers using HTTPS is restricted by the Cross-Origin Resource Sharing (CORS) rules (read more about this here: https://support.streamroot.io/hc/en-us/articles/115003168773-Range-request-basics). If the client and the server are in different domains (e.g., by using a generic COG-enabled web map client requesting COG files in a Amazon Cloud bucket) servers have to explicitly declare that they allow the "range" header from a client that is not in the same domain. Since this is a very common use case in geospatial data sharing the decision was to make this a requirement.
Requirement 11 |
/req/http-range/https-headers |
A |
A HTTPS server serving COG files SHALL support Cross-Origin Resource Sharing (CORS). |
A |
A HTTPS server serving COG files SHALL advertise that it allows the presence of a header requesting access control for ranges by adding this line in the headers of the response: |
|
Note
|
Support for range in HTTPS is necessary but not sufficient. It is not the purpose of this standard to specify how CORS is supported and that may change as web browsers and servers become more concerned about security.
However, it is worth notice that a server that wants to authorize a client application from another domain
(under the CORS rules) should also use the Access-Control-Allow-Origin: header
to list what domains are authorized to read the COG file
(or use `* to indicate that all domains are authorized).
Services that want to comply about the open data sharing principles should be willing to allow all clients reading the data even if they are in other domains.
However, this is up to the server implementers.
|
9. Media Types for any data encoding(s)
A GeoTIFF file is a TIFF file. It is common to use the tiff MIME type, image/tiff according to [RFC3302] (see https://www.iana.org/assignments/media-types/image/tiff). The registration of the MIME type also allows for a application with no format restriction for default value.
Recommendation 5 |
/rec/media-type/media-type |
A |
When required, a COG file should use the MIME type image/tiff with the application parameter value geotiff and the profile parameters value cloud-optimized, therefore as follows: image/tiff; application=geotiff; profile=cloud-optimized |
B |
Alternatively in an HTTP content negotiation a combination of Content-Type and Content-Profile can be used as follows:
|
Annex A: Conformance Class Abstract Test Suite (Normative)
A implementation of this standard must satisfy the following system characteristics to be conformant with this specification.
A.1. Conformance Class GeoTIFF Tiles
Conformance Class |
|
Target type |
TIFF Encoder |
Requirements class |
|
A.1.1. Basic format
Abstract Test 1 |
/conf/geotiff-format/use-geotiff |
|---|---|
Test Purpose |
Validate that a COG file bigger than 4 GByte follows the BigTIFF standard |
Requirement |
/req/geotiff-format/use-geotiff |
Test method |
Validate the requirements of the BiGTIFF Test passes if a COG file bigger than 4 GByte follows the BigTIFF standard |
A.1.2. Tiles
Abstract Test 2 |
/conf/geotiff-format/tiling |
|---|---|
Test Purpose |
Validate that all Image File Directories (IFD) in a COG file are organized into tiles |
Requirement |
/req/geotiff-format/tiling |
Test method |
Validate the requirements of tiled IFDs Test passes if all Image File Directories (IFD) in a COG file are organized into tiles |
A.2. Conformance Class GeoTIFF Overviews
Conformance Class |
|
Target type |
TIFF Encoder |
Requirements class |
|
A.2.1. Overviews
Abstract Test 3 |
/conf/geotiff-overviews/overviews |
|---|---|
Test Purpose |
Validate that a COG file contains the full resolution image as well as the reduced resolution images |
Requirement |
/req/geotiff-overviews/overviews |
Test method |
Validate the requirements of overviews Test passes if at least one IFD whose bit 0 of the value of the NewSubfileType tag is not set, or NewSubfileType absent (full-resolution image data) is present in the COG file and an the IFD with SubfileType 1 has an offset to an IFD with a NewSubfileType tag whose bit 0 is set (reduced-resolution image data), or 0 if there is no reduced-resolution image and all the IFD with a NewSubfileType tag whose bit 0 is set have an Image Width an Image Height inferior than the previous IFD. |
A.3. Conformance Class GeoTIFF Keys
Conformance Class |
|
Target type |
GeoTIFF Encoder |
Requirements class |
|
A.3.1. GeoTIFF
Abstract Test 4 |
/conf/geotiff-format/use-geotiff |
|---|---|
Test Purpose |
Validate that a COG file includes and encodes geospatial metadata following the GeoTIFF standard |
Requirement |
/req/geotiff-format/use-geotiff |
Test method |
Validate the requirements of GeoTIFF metadata Test passes if geospatial metadata following the GeoTIFF standard is found on the COG file |
A.3.2. Georeference Keys
Abstract Test 5 |
/conf/geotiff-keys/georeference |
|---|---|
Test Purpose |
Validate that a COG file contains georeference data using ModelTiepointTag, ModelPixelScaleTag and GeoKeyDirectoryTag tags in the IFD Subfiles of full resolution |
Requirement |
/req/geotiff-keys/georeference |
Test method |
Validate the requirements of georeference Test passes if georeference data using ModelTiepointTag, ModelPixelScaleTag and GeoKeyDirectoryTag tags is found in the IFD Subfiles of full resolution of the COG file (a.k.a IFDs whose bit 0 of the value of the NewSubfileType tag is not set, or NewSubfileType absent) |
Abstract Test 6 |
/conf/geotiff-keys/point-of-origin |
|---|---|
Test Purpose |
Validate that all reduced-resolution subfiles do not contain georeference data. |
Requirement |
/req/geotiff-keys/point-of-origin |
Test method |
Validate the requirements of no metadata in subfiles Test passes if all reduced-resolution subfiles do not contain georeference data in the COG file |
A.4. Conformance Class HTTP range
Conformance Class |
|
Target type |
Web server |
Requirements class |
|
A.4.1. Range
Abstract Test 7 |
/conf/http-range/use-range |
|---|---|
Test Purpose |
Validate that a server supports HTTP range |
Requirement |
/req/http-range/use-range |
Test method |
Validate the requirements of the HTTP range Test passes if an HTTP or HTTPS server supports HTTP range and uses byte ranges. |
Abstract Test 8 |
/conf/http-range/https-headers |
|---|---|
Test Purpose |
Validate that the header Access-Control-Allow-Headers: range is provided |
Requirement |
/req/http-range/https-headers |
Test method |
Validate the requirements of HTTPS allow headers Test passes if an HTTPS server advertises that allows access control for ranges by adding this line in the headers Access-Control-Allow-Headers: range |
Annex B: Revision History
| Date | Release | Editor | Primary clauses modified | Description |
|---|---|---|---|---|
2021-10-26 |
0.1 |
Joan Masó |
all |
initial version from parts of OGC 21-025 |
2022-06-01 |
0.2 |
Joan Masó |
all |
First complete version including ATS |
2023-03-31 |
1.0 |
Joan Masó |
all |
Revision from the naming authority |
Annex C: Bibliography
[1] Cloud Optimized GeoTIFF. An imagery format for cloud-native geospatial processing. In Internet: https://www.cogeo.org/
[2] GDAL COG - Cloud Optimized GeoTIFF generator. In Internet: https://gdal.org/drivers/raster/cog.html
[3] Maso J. OGC Testbed-17: Cloud Optimized GeoTIFF specification Engineering Report. OGC 21-025 (2022)