-
-
Notifications
You must be signed in to change notification settings - Fork 298
Add auto_webmercator to COG
#1893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 46 commits
05d8465
4108e8b
55a5ee1
df8c954
209224f
a212c37
17d0528
aaed0d1
bfe18cc
3a4a946
c5e4b9a
7981a38
40b5ee3
c8010ec
32dab67
1bab765
f571457
dbc603a
e79b088
0f2ee38
02f425a
1ecd40a
8a9d551
acd389e
cf6ac10
843c016
8292dde
3cbd185
73b8b7c
f5da83c
78c2a99
73331e1
c9a071f
0cd936f
b48a52b
92baf2a
a348677
0145c8e
04c4eb1
7d7ac8a
326fcd8
17308ab
6aad230
fa372c6
0b1cb8c
bf8f726
b6039a7
ada9be0
0833a42
ccba0ee
a348df2
d4b0f83
5f194bb
51270d4
8e35aa1
1f04451
f4ad640
a8e716a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -17,44 +17,34 @@ Martin can also serve raster sources like local [COG(Cloud Optimized GeoTIFF)](h | |||||||
| * Deflate | ||||||||
| * PackBits | ||||||||
|
|
||||||||
| ## Run Martin with CLI to serve cog files | ||||||||
| ## Supported Projection | ||||||||
|
|
||||||||
| ```bash | ||||||||
| # Configured with a directory containing `*.tif` or `*.tiff` TIFF files. | ||||||||
| martin /with/tiff/dir1 /with/tiff/dir2 | ||||||||
| # Configured with dedicated TIFF file | ||||||||
| martin /path/to/target1.tif /path/to/target2.tiff | ||||||||
| # Configured with a combination of directories and dedicated TIFF files. | ||||||||
| martin /with/tiff/files /path/to/target1.tif /path/to/target2.tiff | ||||||||
|
Comment on lines
-23
to
-28
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Enable auto-web in CLI Next PR. |
||||||||
| ``` | ||||||||
|
|
||||||||
| ## Run Martin with configuration file | ||||||||
| Currently we only support COGS with [EPSG:3857](https://epsg.io/3857) by enable the `auto-web` option. | ||||||||
|
|
||||||||
| ```yml | ||||||||
| keep_alive: 75 | ||||||||
|
|
||||||||
| # The socket address to bind [default: 0.0.0.0:3000] | ||||||||
| listen_addresses: '0.0.0.0:3000' | ||||||||
| It's beacause the [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) inside each COG file is highly customized for its extent and tilesize. It's not aligned | ||||||||
| with any well knowed TIle Matrix Set. | ||||||||
sharkAndshark marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
|
||||||||
| # Number of web server workers | ||||||||
| worker_processes: 8 | ||||||||
| To load COG file, there are two approaches generally: | ||||||||
|
|
||||||||
| # Amount of memory (in MB) to use for caching tiles [default: 512, 0 to disable] | ||||||||
| cache_size_mb: 8 | ||||||||
| 1. The client(`Maplibre`, `OpenLayers`,etc) load COG file with the specific customized [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72). As martin support various data sources we need to find one general way to support `not-3857` projections and to tell clients the TMS. [Join our disscussion there](https://github.com/maplibre/martin/issues/343). | ||||||||
| 2. Martin serve COG files with well known [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) and do the clipping internally. Currently we support the [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) with `auto-web` configuration. | ||||||||
sharkAndshark marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
|
|
||||||||
| # Database configuration. This can also be a list of PG configs. | ||||||||
| ## configuration file | ||||||||
sharkAndshark marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||
|
|
||||||||
| ```yml | ||||||||
| cog: | ||||||||
| paths: | ||||||||
| # scan this whole dir, matching all *.tif and *.tiff files | ||||||||
| - /dir-path | ||||||||
| # specific TIFF file will be published as a cog source | ||||||||
| - /path/to/target1.tif | ||||||||
| - /path/to/target2.tiff | ||||||||
| # Default false | ||||||||
| # If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid. | ||||||||
|
||||||||
| # If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid. | |
| # If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service. | |
| # The tiles will be cliped/merged internally to be aligned with the Web Mercator grid. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could you clarify this comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Welcome any better description!
That's said martin would make the output tiles aligned with the WebMercatoQuad so clients no need to set the customized tilegrid. I would try to make it better again, and any suggestion? I'm pain in English 😂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| # inline option. Would override the global dauto_web. | |
| # Allows overriding the global auto_web for just this source. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 do we actually need this?
When I think about projections this is more or less a global thing.
I am Unsure.
This is also motivated by new_sources_with_config being kind of hacky in its manual serialisation..
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -272,6 +272,30 @@ pub fn tile_index(lng: f64, lat: f64, zoom: u8) -> (u32, u32) { | |
| (col, row) | ||
| } | ||
|
|
||
| /// Convert min/max XYZ tile coordinates to a bounding box values in Web Mercator. | ||
| #[must_use] | ||
| pub fn xyz_to_bbox_webmercator( | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we have already a xyz_to_bbox actually but its result is not in 3857
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we add it here?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, comparing in the doc string the two seems sensible. Moreover, we should likely use |
||
| zoom: u8, | ||
| min_x: u32, | ||
| min_y: u32, | ||
| max_x: u32, | ||
| max_y: u32, | ||
| ) -> [f64; 4] { | ||
| assert!(zoom <= MAX_ZOOM, "zoom {zoom} must be <= {MAX_ZOOM}"); | ||
|
|
||
| let tile_length = EARTH_CIRCUMFERENCE / f64::from(1_u32 << zoom); | ||
|
|
||
| let left_down_bbox = tile_bbox(min_x, max_y, tile_length); | ||
| let right_top_bbox = tile_bbox(max_x, min_y, tile_length); | ||
|
|
||
| [ | ||
| left_down_bbox[0], | ||
| left_down_bbox[1], | ||
| right_top_bbox[2], | ||
| right_top_bbox[3], | ||
| ] | ||
| } | ||
|
|
||
| /// Convert min/max XYZ tile coordinates to a bounding box values. | ||
| /// | ||
| /// The result is `[min_lng, min_lat, max_lng, max_lat]` | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,6 +13,11 @@ use crate::{MartinResult, TileInfoSource}; | |
| pub struct CogConfig { | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should not serialize |
||
| #[serde(flatten)] | ||
| pub unrecognized: UnrecognizedValues, | ||
|
|
||
| /// Default false | ||
| /// If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid. | ||
| /// Note: Just work for COG files with a Web Mercator CRS (EPSG:3857). | ||
| pub auto_web: Option<bool>, | ||
| } | ||
|
|
||
| impl ConfigExtras for CogConfig { | ||
|
|
@@ -27,7 +32,32 @@ impl SourceConfigExtras for CogConfig { | |
| } | ||
|
|
||
| async fn new_sources(&self, id: String, path: PathBuf) -> MartinResult<TileInfoSource> { | ||
| let cog = CogSource::new(id, path)?; | ||
| let cog = CogSource::new(id, path, self.auto_web.unwrap_or(false))?; | ||
| Ok(Box::new(cog)) | ||
| } | ||
|
|
||
| async fn new_sources_with_config( | ||
| &self, | ||
| id: String, | ||
| path: PathBuf, | ||
| config: serde_yaml::Value, | ||
| ) -> MartinResult<TileInfoSource> { | ||
| let source_auto_web = if let serde_yaml::Value::Mapping(map) = &config { | ||
| if let Some(auto_web_value) = map.get(serde_yaml::Value::String("auto_web".to_string())) | ||
| { | ||
| match auto_web_value { | ||
| serde_yaml::Value::Bool(b) => Some(*b), | ||
| _ => None, | ||
| } | ||
| } else { | ||
| None | ||
| } | ||
| } else { | ||
| None | ||
| }; | ||
|
|
||
| let auto_web = source_auto_web.unwrap_or_else(|| self.auto_web.unwrap_or(false)); | ||
| let cog = CogSource::new(id, path, auto_web)?; | ||
| Ok(Box::new(cog)) | ||
| } | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as far as I remember these two need to be marked as "please always optimise these at O3" as otherwise debug builds handling images are.. painfull.