Normalize CRS handling (#2116)

* Normalize CRS handling

* Normalize storage_crs

* Update crs.py

* Cleanup

* Use CrsTransformSpec for all CRS conversion

* Create `get_transform_from_spec`

* Add additional tests

* Reduce diff

* Revert transform_bbox changes

* Update documentation

* Update CRS docs

* Update plugins.rst

* Fix comment spacing in configuration docs

* Incorporate PR review feedback

* Small change to verbiage

Author: webb-ben

docs/source/configuration.rst

@@ -48,37 +48,43 @@ For more information related to API design rules (the ``api_rules`` property in
     encoding: utf-8  # default server encoding
     language: en-US  # default server language
     locale_dir: /path/to/translations
-    gzip: false # default server config to gzip/compress responses to requests with gzip in the Accept-Encoding header
+    ogc_schemas_location: /opt/schemas.opengis.net  # optional local copy of https://schemas.opengis.net
+    gzip: false  # default server config to gzip/compress responses to requests with gzip in the Accept-Encoding header
     cors: true  # boolean on whether server should support CORS
     pretty_print: true  # whether JSON responses should be pretty-printed
+    admin: false  # whether to enable the Admin API
 
-    limits:  # server limits on number of items to return.  This property can also be defined at the resource level to override global server settings
+    # server limits on number of items to return.
+    # overridable when redefined in resource level configuration
+    limits:
         default_items: 50
         max_items: 1000
         max_distance_x: 25
         max_distance_y: 25
         max_distance_units: m
         on_exceed: throttle  # throttle or error (default=throttle)
 
-    admin: false  # whether to enable the Admin API
-
-    # optional configuration to specify a different set of templates for HTML pages. Recommend using absolute paths. Omit this to use the default provided templates
-    # This property can also be defined at the resource level to override global server settings for specific datasets
-    templates: # optional configuration to specify a different set of templates for HTML pages. Recommend using absolute paths. Omit this to use the default provided templates
+    # configuration to specify directory tree for HTML page templates
+    # omit this to use the default pygeoapi templates
+    # overridable when redefined in resource level configuration
+    templates:
+      # recommended to use absolute paths
       path: /path/to/jinja2/templates/folder # path to templates folder containing the Jinja2 template HTML files
       static: /path/to/static/folder # path to static folder containing css, js, images and other static files referenced by the template
 
-    map:  # leaflet map setup for HTML pages
+    # leaflet map setup for HTML template rendering
+    map:
         url: https://tile.openstreetmap.org/{z}/{x}/{y}.png
         attribution: '&copy; <a href="https://openstreetmap.org/copyright">OpenStreetMap contributors</a>'
-    ogc_schemas_location: /opt/schemas.opengis.net  # local copy of https://schemas.opengis.net
 
-    manager:  # optional OGC API - Processes asynchronous job management
+    # optional OGC API - Processes asynchronous job management configuration
+    manager:
         name: TinyDB  # plugin name (see pygeoapi.plugin for supported process_manager's)
         connection: /tmp/pygeoapi-process-manager.db  # connection info to store jobs (e.g. filepath)
         output_dir: /tmp/  # temporary file area for storing job results (files)
 
-    api_rules:  # optional API design rules to which pygeoapi should adhere
+    # optional API design rules to which pygeoapi should adhere
+    api_rules:
         api_version: 1.2.3  # omit to use pygeoapi's software version
         strict_slashes: true  # trailing slashes will not be allowed and result in a 404
         url_prefix: 'v{api_major}'  # adds a /v1 prefix to all URL paths
@@ -95,8 +101,8 @@ The ``logging`` section provides directives for logging messages which are usefu
   logging:
       level: ERROR  # the logging level (see https://docs.python.org/3/library/logging.html#logging-levels)
       logfile: /path/to/pygeoapi.log  # the full file path to the logfile
-      logformat: # example for milliseconds:'[%(asctime)s.%(msecs)03d] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s'
-      dateformat: # example for milliseconds:'%Y-%m-%dT%H:%M:%S'
+      logformat:  # example for milliseconds:'[%(asctime)s.%(msecs)03d] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s'
+      dateformat:  # example for milliseconds:'%Y-%m-%dT%H:%M:%S'
 
 .. note::
    If ``level`` is defined and ``logfile`` is undefined, logging messages are output to the server's ``stdout``.
@@ -112,11 +118,12 @@ The ``rotation`` supports rotation of disk log files. The ``logfile`` file is op
   logging:
       logfile: /path/to/pygeoapi.log  # the full file path to the logfile
       rotation:
-          mode: # [time|size]
-          when: # [s|m|h|d|w0-w6|midnight]
+          mode:  # [time|size]
+          when:  # [s|m|h|d|w0-w6|midnight]
           interval: 
           max_bytes: 
           backup_count: 
+
 .. note::
   Rotation block is not mandatory and defined only when needed. The ``mode`` can be defined by size or time.
   For RotatingFileHandler_ set mode size and parameters max_bytes and backup_count.
@@ -198,13 +205,13 @@ default.
           keywords:  # list of related keywords
               - observations
               - monitoring
-          linked-data: # linked data configuration (see Linked Data section)
+          linked-data:  # linked data configuration (see Linked Data section)
               context:
                   - datetime: https://schema.org/DateTime
                   - vocab: https://example.com/vocab#
                     stn_id: "vocab:stn_id"
                     value: "vocab:value"
-          links:  # list of 1..n related links
+          links: # list of 1..n related links
               - type: text/csv  # MIME type
                 rel: canonical  # link relations per https://www.iana.org/assignments/link-relations/link-relations.xhtml
                 title: data  # title
@@ -219,38 +226,37 @@ default.
                   end: 2007-10-30T08:57:29Z  # end datetime in RFC3339
                   trs: http://www.opengis.net/def/uom/ISO-8601/0/Gregorian  # TRS
           providers:  # list of 1..n required connections information
-              # provider name
-              # see pygeoapi.plugin for supported providers
-              # for custom built plugins, use the import path (e.g. mypackage.provider.MyProvider)
-              # see Plugins section for more information
-              - type: feature # underlying data geospatial type: (allowed values are: feature, coverage, record, tile, edr)
-                default: true  # optional: if not specified, the first provider definition is considered the default
-                name: CSV
+              - type: feature  # underlying data geospatial type. Allowed values are: feature, coverage, record, tile, edr
+                name: CSV  # required: plugin name or import path. See Plugins section for more information.
                 data: tests/data/obs.csv  # required: the data filesystem path or URL, depending on plugin setup
                 id_field: id  # required for vector data, the field corresponding to the ID
-                uri_field: uri # optional field corresponding to the Uniform Resource Identifier (see Linked Data section)
-                time_field: datetimestamp  # optional field corresponding to the temporal property of the dataset
-                title_field: foo # optional field of which property to display as title/label on HTML pages
-                properties:  # optional: only return the following properties, in order
+
+                # optional fields
+                uri_field: uri  # field corresponding to the Uniform Resource Identifier (see Linked Data section)
+                time_field: datetimestamp  # field corresponding to the temporal property of the dataset
+                title_field: foo  # field of which property to display as title/label on HTML pages
+                default: true  # if not specified, the first provider definition is considered the default
+                properties:  # if specified, return only the following properties, in order
                     - stn_id
                     - value
+                format:  # default format
+                    name: GeoJSON  # required: format name
+                    mimetype: application/json  # required: format mimetype
+                options:  # optional options to pass to provider (i.e. GDAL creation)
+                    option_name: option_value
+                include_extra_query_parameters: false  # include extra query parameters that are not part of the collection properties (default: false)
                 # editable transactions: DO NOT ACTIVATE unless you have setup access control beyond pygeoapi
                 editable: true  # optional: if backend is writable, default is false
                 # coordinate reference systems (CRS) section is optional
                 # default CRSs are http://www.opengis.net/def/crs/OGC/1.3/CRS84 (coordinates without height)
                 # and http://www.opengis.net/def/crs/OGC/1.3/CRS84h (coordinates with ellipsoidal height)
-                crs: # supported coordinate reference systems (CRS) for 'crs' and 'bbox-crs' query parameters
+                crs:  # supported coordinate reference systems (CRS) for 'crs' and 'bbox-crs' query parameters
                     - http://www.opengis.net/def/crs/EPSG/0/28992
                     - http://www.opengis.net/def/crs/OGC/1.3/CRS84
                     - http://www.opengis.net/def/crs/EPSG/0/4326
-                storage_crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84 # optional CRS in which data is stored, default: as 'crs' field
-                storage_crs_coordinate_epoch: : 2017.23 # optional, if storage_crs is a dynamic coordinate reference system
-                format:  # optional default format
-                    name: GeoJSON  # required: format name
-                    mimetype: application/json  # required: format mimetype
-                options:  # optional options to pass to provider (i.e. GDAL creation)
-                    option_name: option_value
-                include_extra_query_parameters: false  # include extra query parameters that are not part of the collection properties (default: false)
+                storage_crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84  # optional CRS in which data is stored, default: as 'crs' field
+                storage_crs_coordinate_epoch: 2017.23  # optional, if storage_crs is a dynamic coordinate reference system
+                always_xy: false  # optional should CRS respect axis ordering
 
       hello-world:  # name of process
           type: process  # REQUIRED (collection, process, or stac-collection)

docs/source/crs.rst

@@ -3,42 +3,98 @@
 CRS support
 ===========
 
-pygeoapi supports the complete specification: `OGC API - Features - Part 2: Coordinate Reference Systems by Reference corrigendum`_.
-The specified CRS-related capabilities are available for all Feature data Providers.
+pygeoapi supports multiple Coordinate Reference Systems (CRS). 
+This enables the import and export of any data according to dedicated projections.
+A "projection" is specified with a CRS identifier.
+In particular CRS support allows for the following:
+
+* to specify the CRS in which the data is stored, in pygeoapi the `storage_crs` config option
+* to specify the list of valid CRS representations, in pygeoapi the `crs` config option
+* to publish these in the collection metadata
+* the `bbox-crs=` query parameter to indicate that the `bbox=` parameter is encoded in that CRS
+* the `crs=` query parameter for a collection or collection item
+* the HTTP response header `Content-Crs` denotes the CRS of the Feature(s) in the data returned
+
+Although GeoJSON mandates WGS84 in longitude, latitude order, the client and server may still agree on other CRSs.
+
+
+Background
+----------
+
+pygeoapi implements the `OGC API - Features - Part 2: Coordinate Reference Systems by Reference`_ specification.
+
+Under the hood, pygeoapi uses the `pyproj`_ Python package.
+
+.. note::
+
+   For more information on implementing CRS on custom plugins, see `Implementation`_.
+
+CRS support exists for the following OGC APIs:
+
+.. csv-table::
+   :header: OGC API, bbox-crs, filter-crs, crs
+   :align: left
+
+   :ref:`OGC API - Features<ogcapi-features>`,✅,✅,✅
+   :ref:`OGC API - Maps<ogcapi-maps>`,✅,❌,❌
+   :ref:`OGC API - Coverages<ogcapi-coverages>`,✅,❌,❌
 
 Configuration
 -------------
 
-For details visit the :ref:`configuration` section for Feature Providers. At this moment only the 'URI' CRS notation format is supported.
+The CRS of a collection is defined in the provider block of your resource.
+The configuration controls how the `crs` related query parameters behave.
+
+
+* ``crs`` - list of CRSs supported
+* ``storage_crs`` - CRS in which the data is stored (must be in `crs` list)
+* ``storage_crs_coordinate_epoch`` - epoch of `storage_crs` for a dynamic coordinate reference system
+* ``always_xy`` - CRS should ignore authority on axis order, disobeying `ISO-19111`_ (default: false) 
+
+.. note::
+    bbox-crs and filter-crs are used to convert the request geometry to the configured ``storage_crs``.
+    An error will be returned for any interaction with CRS not included in the configured ``crs`` list.
 
+The per-Provider configuration fields are all optional,
+with the following as default configuration:
 
-* `crs` - list of CRSs supported
-* `storage_crs` - CRS in which the data is stored (must be in `crs` list)
-* `storage_crs_coordinate_epoch` - epoch of `storage_crs` for a dynamic coordinate reference system
+.. code-block:: yaml
 
+    crs:
+        - http://www.opengis.net/def/crs/OGC/1.3/CRS84
+        - http://www.opengis.net/def/crs/OGC/1.3/CRS84h
+    storage_crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
 
-These per-Provider configuration fields are all optional. Default for CRS-values is `http://www.opengis.net/def/crs/OGC/1.3/CRS84`, so "WGS84" with lon/lat axis ordering.
-If the storage CRS of the spatial feature collection is a dynamic coordinate reference system,
-`storage_crs_coordinate_epoch` configures the coordinate epoch of the coordinates.
+.. note::
+    Configuration is done with URI formats like http://www.opengis.net/def/crs/OGC/1.3/CRS84. 
+    Both `URI` and `URN` CRS notation format are supported.
+    The `EPSG:` format like EPSG:4326 is outside the scope of the OGC standard.
 
-There is also support for CRSs that support height like `http://www.opengis.net/def/crs/OGC/1.3/CRS84h`. In that case
-bbox parameters (see below) may contain 6 coordinates.
 
 Metadata
 --------
 
-The conformance class `http://www.opengis.net/spec/ogcapi-features-2/1.0/conf/crs` is present as a `conformsTo` field
-in the root landing page response.
+The conformance class http://www.opengis.net/spec/ogcapi-features-2/1.0/conf/crs is
+present as a `conformsTo` field in the root landing page response.
 
 The configured CRSs, or their defaults, `crs` and `storageCrs` and optionally `storageCrsCoordinateEpoch` will be present in the "Describe Collection" response.
 
+.. note::
+    If the storage CRS of the spatial feature collection is a dynamic coordinate reference system,
+    `storage_crs_coordinate_epoch` configures the coordinate epoch of the coordinates.
+
+.. note::
+    There is also support for CRSs that support height like `http://www.opengis.net/def/crs/OGC/1.3/CRS84h`. In that case
+    bbox parameters (see below) may contain 6 coordinates.
+
 Parameters
 ----------
 
 The `items` query supports the following parameters:
 
-* `crs` - the CRS in which Features coordinates should be returned, also for the 'get single item' request
-* `bbox-crs` - the CRS of the `bbox` parameter (only for Providers that support the `bbox` parameter)
+* ``crs`` - the CRS in which Features coordinates should be returned, also for the 'get single item' request
+* ``bbox-crs`` - the CRS of the `bbox` parameter (for Providers that support the `bbox` parameter)
+* ``filter-crs`` - the CRS of the CQL filter expression (for Providers that support `CQL` filters)
 
 If any or both of these parameters are specified, their CRS-value should be from the advertised CRS-list in the Collection metadata (see above).
 
@@ -52,21 +108,29 @@ Note that the values of these parameters may need to be URL-encoded.
 Implementation
 --------------
 
-CRS and BBOX CRS support is implemented for all Feature Providers. Some details may help understanding (performance) implications.
+CRS and BBOX CRS support is implemented for all Feature Providers.
+Some details may help understanding (performance) implications.
 
-BBOX CRS Parameter
+bbox-crs Parameter
 ^^^^^^^^^^^^^^^^^^
 
-The `bbox-crs` parameter is handled at the common level of pygeoapi, thus transparent for Feature Providers.
-Obviously the Provider should support `bbox`.
-A transformation of the `bbox` parameter is performed
+The ``bbox-crs`` parameter is handled at the common level of pygeoapi.
+A transformation of the request `bbox` parameter is performed
 according to the `storage_crs` configuration. Then the (transformed) `bbox` is passed with the
 other query parameters to the Provider instance.
 
-CRS Parameter
+filter-crs Parameter
+^^^^^^^^^^^^^^^^^^^^
+
+The ``filter-crs`` parameter is handled at the common level of pygeoapi.
+A transformation of the request `CQL` filter is performed
+according to the `storage_crs` configuration. Then the (transformed) `filter` is passed with the
+other query parameters to the Provider instance.
+
+crs Parameter
 ^^^^^^^^^^^^^
 
-When the value of the `crs` parameter differs from the Provider data Storage CRS, the response Feature coordinates
+When the value of the ``crs`` parameter differs from the Provider data Storage CRS, the response Feature coordinates
 need to be transformed to that CRS. As some Feature Providers like PostgreSQL or OGR may support native
 coordinate transformation, pygeoapi delegates transformation to those Providers, passing the `crs` with the other query parameters.
 
@@ -245,4 +309,6 @@ Or you may specify both `crs` and `bbox-crs` and thus `bbox` in that CRS `http:/
     .
     .
 
-.. _`OGC API - Features - Part 2: Coordinate Reference Systems by Reference corrigendum`: https://docs.opengeospatial.org/is/18-058r1/18-058r1.html
+.. _`ISO-19111`: https://docs.ogc.org/as/18-005r5/18-005r5.html
+.. _`OGC API - Features - Part 2: Coordinate Reference Systems by Reference`: https://docs.ogc.org/is/18-058r1/18-058r1.html
+.. _`pyproj`: https://pyproj4.github.io/pyproj

docs/source/plugins.rst

@@ -7,6 +7,9 @@ In this section we will explain how pygeoapi provides plugin architecture for da
 
 Plugin development requires knowledge of how to program in Python as well as Python's package/module system.
 
+.. seealso::
+   :ref:`publishing` for configuration of default plugins.
+
 Overview
 --------
 

docs/source/publishing/ogcapi-coverages.rst

@@ -77,7 +77,7 @@ The `Xarray`_ provider plugin reads and extracts `NetCDF`_ and `Zarr`_ data.
          time_field: time
          # optionally specify the coordinate reference system of your dataset
          # else pygeoapi assumes it is WGS84 (EPSG:4326).
-         storage_crs: 4326
+         storage_crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84
          format:
             name: netcdf
             mimetype: application/x-netcdf