Skip to content
Open
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
05d8465
add resolution to Image
sharkAndshark Jun 15, 2025
4108e8b
add auto_web option
sharkAndshark Jun 18, 2025
55a5ee1
add web zoom if auto_web enabled
sharkAndshark Jun 18, 2025
df8c954
add intersect_tiles() to Image
sharkAndshark Jun 18, 2025
209224f
wip
sharkAndshark Jun 18, 2025
a212c37
add origin to image
sharkAndshark Jun 23, 2025
17d0528
add doc
sharkAndshark Jun 23, 2025
aaed0d1
reanme func
sharkAndshark Jun 23, 2025
bfe18cc
refactor
sharkAndshark Jun 23, 2025
3a4a946
wip: finish func clip
sharkAndshark Jun 23, 2025
c5e4b9a
add crates
sharkAndshark Jun 23, 2025
7981a38
finish clip method
sharkAndshark Jun 25, 2025
40b5ee3
clip imag when web friendly enabled
sharkAndshark Jun 25, 2025
c8010ec
wip
sharkAndshark Jul 14, 2025
32dab67
add doc
sharkAndshark Jul 14, 2025
1bab765
update cargo lock
sharkAndshark Jul 14, 2025
f571457
clippy
sharkAndshark Jul 14, 2025
dbc603a
wip: clippy
sharkAndshark Jul 15, 2025
e79b088
clippy
sharkAndshark Jul 16, 2025
0f2ee38
clean up
sharkAndshark Jul 16, 2025
02f425a
add test cases
sharkAndshark Jul 16, 2025
1ecd40a
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Jul 17, 2025
8a9d551
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 17, 2025
acd389e
add auto_web as a source config
sharkAndshark Jul 21, 2025
cf6ac10
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Jul 21, 2025
843c016
bless test
sharkAndshark Jul 21, 2025
8292dde
Merge branch 'main' into cog_web_5
sharkAndshark Jul 22, 2025
3cbd185
add test cases
sharkAndshark Aug 7, 2025
73b8b7c
cleanup
sharkAndshark Aug 7, 2025
f5da83c
cleanup
sharkAndshark Aug 8, 2025
78c2a99
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 8, 2025
73331e1
add test cases
sharkAndshark Aug 8, 2025
c9a071f
Merge branch 'main' into cog_web_5
sharkAndshark Aug 10, 2025
0cd936f
cleanup
sharkAndshark Aug 11, 2025
b48a52b
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 14, 2025
92baf2a
update test
sharkAndshark Aug 14, 2025
a348677
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 15, 2025
0145c8e
bless tests
sharkAndshark Aug 15, 2025
04c4eb1
bless
sharkAndshark Aug 15, 2025
7d7ac8a
fix bug
sharkAndshark Aug 15, 2025
326fcd8
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 16, 2025
17308ab
naming
sharkAndshark Aug 16, 2025
6aad230
bless
sharkAndshark Aug 16, 2025
fa372c6
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 16, 2025
0b1cb8c
bless
sharkAndshark Aug 16, 2025
bf8f726
add doc
sharkAndshark Aug 16, 2025
b6039a7
add tests
sharkAndshark Aug 16, 2025
ada9be0
cargo clippy
sharkAndshark Aug 16, 2025
0833a42
cargo sort
sharkAndshark Aug 16, 2025
ccba0ee
Merge remote-tracking branch 'maplibre/main' into cog_web_5
sharkAndshark Aug 24, 2025
a348df2
sort toml
sharkAndshark Aug 24, 2025
d4b0f83
Merge branch 'main' into cog_web_5
sharkAndshark Aug 26, 2025
5f194bb
refactor xyz_to_bbox function
sharkAndshark Aug 28, 2025
51270d4
better doc
sharkAndshark Aug 28, 2025
8e35aa1
doc improvement
sharkAndshark Aug 28, 2025
1f04451
Merge branch 'main' into cog_web_5
CommanderStorm Aug 31, 2025
f4ad640
Merge branch 'main' into cog_web_5
CommanderStorm Sep 15, 2025
a8e716a
Merge branch 'main' into cog_web_5
CommanderStorm Sep 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
760 changes: 650 additions & 110 deletions Cargo.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ env_logger = "0.11"
flate2 = "1"
flume = "0.11"
futures = "0.3"
image = "0.25.6"
imageproc = "0.25.0"
Comment on lines +52 to +53
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I remember these two need to be marked as "please always optimise these at O3" as otherwise debug builds handling images are.. painfull.

indoc = "2"
insta = "1"
itertools = "0.14"
Expand Down
1 change: 1 addition & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
- [PostgreSQL Connections](pg-connections.md)
- [PostgreSQL Table Sources](sources-pg-tables.md)
- [PostgreSQL Function Sources](sources-pg-functions.md)
- [Cloud-Optimized-GeoTIFF Sources](sources-cog-files.md)
- [Composite Sources](sources-composite.md)
- [Supporting Resources](sources-resources.md)
- [Sprites](sources-sprites.md)
Expand Down
50 changes: 20 additions & 30 deletions docs/src/sources-cog-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,44 +17,34 @@ Martin can also serve raster sources like local [COG(Cloud Optimized GeoTIFF)](h
* Deflate
* PackBits

## Run Martin with CLI to serve cog files
## Supported Projection

```bash
# Configured with a directory containing `*.tif` or `*.tiff` TIFF files.
martin /with/tiff/dir1 /with/tiff/dir2
# Configured with dedicated TIFF file
martin /path/to/target1.tif /path/to/target2.tiff
# Configured with a combination of directories and dedicated TIFF files.
martin /with/tiff/files /path/to/target1.tif /path/to/target2.tiff
Comment on lines -23 to -28
Copy link
Collaborator Author

@sharkAndshark sharkAndshark Aug 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enable auto-web in CLI Next PR.

```

## Run Martin with configuration file
Currently we only support COGS with [EPSG:3857](https://epsg.io/3857) by enable the `auto-web` option.

```yml
keep_alive: 75

# The socket address to bind [default: 0.0.0.0:3000]
listen_addresses: '0.0.0.0:3000'
It's beacause the [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) inside each COG file is highly customized for its extent and tilesize. It's not aligned
with any well knowed TIle Matrix Set.

# Number of web server workers
worker_processes: 8
To load COG file, there are two approaches generally:

# Amount of memory (in MB) to use for caching tiles [default: 512, 0 to disable]
cache_size_mb: 8
1. The client(`Maplibre`, `OpenLayers`,etc) load COG file with the specific customized [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72). As martin support various data sources we need to find one general way to support `not-3857` projections and to tell clients the TMS. [Join our disscussion there](https://github.com/maplibre/martin/issues/343).
2. Martin serve COG files with well known [Tile Matrix Set](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) and do the clipping internally. Currently we support the [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) with `auto-web` configuration.

# Database configuration. This can also be a list of PG configs.
## configuration file

```yml
cog:
paths:
# scan this whole dir, matching all *.tif and *.tiff files
- /dir-path
# specific TIFF file will be published as a cog source
- /path/to/target1.tif
- /path/to/target2.tiff
# Default false
# If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid.
# If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service.
# The tiles will be cliped/merged internally to be aligned with the Web Mercator grid.

# Note: Just work for COG files with a Web Mercator CRS (EPSG:3857).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you clarify this comment?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome any better description!
That's said martin would make the output tiles aligned with the WebMercatoQuad so clients no need to set the customized tilegrid. I would try to make it better again, and any suggestion? I'm pain in English 😂

auto_web: false
sources:
# named source matching source name to a single file
cog-src1: /path/to/cog1.tif
cog-src2: /path/to/cog2.tif
cog-src2: tests/fixtures/cog/rgb_u8.tif
cog-src1: tests/fixtures/cog/rgba_u8.tif
# Test COG with auto_webmercator enabled
cog-auto-web:
path: tests/fixtures/cog/rgba_u8_nodata.tiff
# inline option. Would override the global dauto_web.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# inline option. Would override the global dauto_web.
# Allows overriding the global auto_web for just this source.

auto_web: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 do we actually need this?
When I think about projections this is more or less a global thing.
I am Unsure.

This is also motivated by new_sources_with_config being kind of hacky in its manual serialisation..

```

## About COG
Expand Down
24 changes: 24 additions & 0 deletions martin-tile-utils/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,30 @@ pub fn tile_index(lng: f64, lat: f64, zoom: u8) -> (u32, u32) {
(col, row)
}

/// Convert min/max XYZ tile coordinates to a bounding box values in Web Mercator.
#[must_use]
pub fn xyz_to_bbox_webmercator(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have already a xyz_to_bbox actually but its result is not in 3857

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add it here?

Copy link
Member

@CommanderStorm CommanderStorm Aug 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, comparing in the doc string the two seems sensible.

Moreover, we should likely use xyz_to_bbox_webmercator in xyz_to_bbox (instead of copy and paste) and rename it to xyz_to_bbox_wgs84

zoom: u8,
min_x: u32,
min_y: u32,
max_x: u32,
max_y: u32,
) -> [f64; 4] {
assert!(zoom <= MAX_ZOOM, "zoom {zoom} must be <= {MAX_ZOOM}");

let tile_length = EARTH_CIRCUMFERENCE / f64::from(1_u32 << zoom);

let left_down_bbox = tile_bbox(min_x, max_y, tile_length);
let right_top_bbox = tile_bbox(max_x, min_y, tile_length);

[
left_down_bbox[0],
left_down_bbox[1],
right_top_bbox[2],
right_top_bbox[3],
]
}

/// Convert min/max XYZ tile coordinates to a bounding box values.
///
/// The result is `[min_lng, min_lat, max_lng, max_lat]`
Expand Down
4 changes: 3 additions & 1 deletion martin/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ default = [
"styles",
"webui",
]
cog = ["dep:png", "dep:tiff"]
cog = ["dep:tiff", "dep:png", "dep:image", "dep:imageproc"]
fonts = ["dep:bit-set", "dep:pbf_font_tools", "dep:regex"]
lambda = ["dep:lambda-web"]
mbtiles = ["dep:mbtiles"]
Expand Down Expand Up @@ -108,6 +108,8 @@ deadpool-postgres = { workspace = true, optional = true }
enum-display = { workspace = true, optional = true }
env_logger.workspace = true
futures.workspace = true
image = { workspace = true, optional = true }
imageproc = { workspace = true, optional = true }
itertools.workspace = true
json-patch = { workspace = true, optional = true }
lambda-web = { workspace = true, optional = true }
Expand Down
32 changes: 31 additions & 1 deletion martin/src/cog/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ use crate::{MartinResult, TileInfoSource};
pub struct CogConfig {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not serialize auto_web if None => add the serde_with marker

#[serde(flatten)]
pub unrecognized: UnrecognizedValues,

/// Default false
/// If enabled, martin will automatically serve COG as a [WebMercatorQuad](https://docs.ogc.org/is/17-083r2/17-083r2.html#72) service, the tiles will be cliped and merged internally to be aligned with the Web Mercator grid.
/// Note: Just work for COG files with a Web Mercator CRS (EPSG:3857).
pub auto_web: Option<bool>,
}

impl ConfigExtras for CogConfig {
Expand All @@ -27,7 +32,32 @@ impl SourceConfigExtras for CogConfig {
}

async fn new_sources(&self, id: String, path: PathBuf) -> MartinResult<TileInfoSource> {
let cog = CogSource::new(id, path)?;
let cog = CogSource::new(id, path, self.auto_web.unwrap_or(false))?;
Ok(Box::new(cog))
}

async fn new_sources_with_config(
&self,
id: String,
path: PathBuf,
config: serde_yaml::Value,
) -> MartinResult<TileInfoSource> {
let source_auto_web = if let serde_yaml::Value::Mapping(map) = &config {
if let Some(auto_web_value) = map.get(serde_yaml::Value::String("auto_web".to_string()))
{
match auto_web_value {
serde_yaml::Value::Bool(b) => Some(*b),
_ => None,
}
} else {
None
}
} else {
None
};

let auto_web = source_auto_web.unwrap_or_else(|| self.auto_web.unwrap_or(false));
let cog = CogSource::new(id, path, auto_web)?;
Ok(Box::new(cog))
}

Expand Down
3 changes: 3 additions & 0 deletions martin/src/cog/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,7 @@ pub enum CogError {
"Get full resolution failed for {0}: either a valid ModelPixelScaleTag or ModelPixelScaleTag is required"
)]
GetFullResolutionFailed(PathBuf),

#[error("Failed to create image buffer for {0}: {1}")]
ImageBufferCreationFailed(PathBuf, String),
}
Loading
Loading