Skip to content

Conversation

@jake-low
Copy link

@jake-low jake-low commented Nov 17, 2025

In FeatureMerge.java, merged features are now given a new ID ending in 0 instead of reusing one of the input feature IDs. This indicates the tile feature doesn't correspond to a single source element.

For OSM features where IDs encode as osm_id * 10 + element_type, IDs ending in 0 are reserved for non-OSM or composite features. This helps prevent data consumers from incorrectly linking merged features back to a particular OSM element.

Fixes #1393.

I added new tests for this change and updated some existing tests which were expecting the old behavior in their assertions. Note that there are also a couple of tests in the planetiler-openmaptiles submodule that will need to be updated if this change is merged. Happy to open a PR for that too, but wasn't sure if that should be done in tandem with this one or afterwards.

I also tested this by installing my modified version of Planetiler and using it to build Sourdough tiles, then viewed the resulting PMTiles archive and verified that merged features have IDs ending in 0, and unmerged features have their original IDs (encoding the OSM type and ID of the source feature).

In FeatureMerge.java, merged features are now given a new ID ending in 0
instead of reusing one of the input feature IDs. This indicates the tile
feature doesn't correspond to a single source element.

For OSM features where IDs encode as osm_id * 10 + element_type, IDs
ending in 0 are reserved for non-OSM or composite features. This helps
prevent data consumers from incorrectly linking merged features back to
a particular OSM element.

Fixes onthegomap#1393
@github-actions
Copy link

github-actions bot commented Nov 17, 2025

This Branch 83e308e Base 321baf3
0:01:13 DEB [archive] - Tile stats:
0:01:13 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (160k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:88k)
2. 9/154/190 (148k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:85k)
3. 10/308/380 (136k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
4. 10/308/381 (136k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
5. 14/4941/6092 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (building:62k)
7. 14/4940/6092 (101k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
8. 14/4946/6112 (99k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:60k)
9. 11/616/762 (99k) https://onthegomap.github.io/planetiler-demo/#11.5/41.7057/-71.63086 (landcover:71k)
10. 11/616/764 (97k) https://onthegomap.github.io/planetiler-demo/#11.5/41.44269/-71.63086 (landcover:74k)
0:01:13 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   336   409   544   872   350   464   573   808  1.6k    2k  6.8k  6.2k  5.6k  4.4k  6.8k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   707   796    1k  1.6k  3.1k    6k  3.6k  1.8k   845   978    6k
            landuse    0     0     0     0   549   695  1.6k  6.7k   17k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   418   972  1.4k  4.6k  6.4k   21k   15k   17k   64k   47k   34k   64k
           waterway    0     0     0     0   112   119     0     0     0    3k  2.2k    2k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.3k  4.3k  9.7k   18k   13k  8.2k  3.7k  3.4k  4.4k   18k
transportation_name    0     0     0     0     0     0   287   364  1.1k  1.9k  5.6k  4.8k  3.9k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.8k   29k   85k   72k   81k   53k   30k   25k   85k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.3k  2.8k  1.4k  1.4k   869  4.3k
         water_name    0     0     0     0     0     0     0     0     0   486   461   433   452  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   88k   88k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   59k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.8k  6.3k   21k   42k   84k  201k  183k  135k  114k  128k  252k  252k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.2k    5k   14k   29k   60k  148k  136k   99k   84k   92k  160k  160k
0:01:13 DEB [archive] -    Max tile: 252k (gzipped: 160k)
0:01:13 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:13 DEB [archive] -     # tiles: 4,115,056
0:01:13 DEB [archive] -  # features: 5,691,811
0:01:13 INF [archive] - Finished in 21s cpu:1m16s avg:3.7
0:01:13 INF [archive] -   read    1x(3% 0.6s wait:19s done:1s)
0:01:13 INF [archive] -   encode  4x(57% 12s wait:2s done:1s)
0:01:13 INF [archive] -   write   1x(20% 4s wait:14s)
0:01:13 INF [archive] - Finished in 1m14s cpu:3m54s gc:1s avg:3.2
0:01:13 INF [archive] - FINISHED!
0:01:13 INF [archive] - 
0:01:13 INF [archive] - ----------------------------------------
0:01:13 INF [archive] - data errors:
0:01:13 INF [archive] - 	render_snap_fix_input	16,947
0:01:13 INF [archive] - 	osm_multipolygon_missing_way	360
0:01:13 INF [archive] - 	osm_boundary_missing_way	55
0:01:13 INF [archive] - 	merge_snap_fix_input	15
0:01:13 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	6
0:01:13 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:13 INF [archive] - 	render_snap_fix_input2	1
0:01:13 INF [archive] - 	omt_fix_water_before_ne_intersect	1
0:01:13 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:13 INF [archive] - ----------------------------------------
0:01:13 INF [archive] - 	overall          1m14s cpu:3m54s gc:1s avg:3.2
0:01:13 INF [archive] - 	lake_centerlines 3s cpu:7s avg:2.3
0:01:13 INF [archive] - 	  read     1x(16% 0.5s done:2s)
0:01:13 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:13 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:13 INF [archive] - 	water_polygons   15s cpu:41s avg:2.7
0:01:13 INF [archive] - 	  read     1x(42% 6s done:7s)
0:01:13 INF [archive] - 	  process  4x(26% 4s wait:4s done:6s)
0:01:13 INF [archive] - 	  write    1x(4% 0.5s wait:10s done:5s)
0:01:13 INF [archive] - 	natural_earth    11s cpu:18s avg:1.7
0:01:13 INF [archive] - 	  read     1x(57% 6s done:5s)
0:01:13 INF [archive] - 	  process  4x(8% 0.8s wait:6s done:5s)
0:01:13 INF [archive] - 	  write    1x(0% 0s wait:6s done:5s)
0:01:13 INF [archive] - 	osm_pass1        2s cpu:6s avg:3.1
0:01:13 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:13 INF [archive] - 	  parse    4x(32% 0.7s)
0:01:13 INF [archive] - 	  process  1x(73% 2s)
0:01:13 INF [archive] - 	osm_pass2        20s cpu:1m19s avg:4
0:01:13 INF [archive] - 	  read     1x(0% 0s wait:13s done:8s)
0:01:13 INF [archive] - 	  process  4x(77% 15s)
0:01:13 INF [archive] - 	  write    1x(2% 0.5s wait:20s)
0:01:13 INF [archive] - 	ne_lakes         0s cpu:0s avg:0
0:01:13 INF [archive] - 	boundaries       0s cpu:0s avg:1.4
0:01:13 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:13 INF [archive] - 	sort             1s cpu:4s avg:2.6
0:01:13 INF [archive] - 	  worker  1x(49% 0.7s)
0:01:13 INF [archive] - 	archive          21s cpu:1m16s avg:3.7
0:01:13 INF [archive] - 	  read    1x(3% 0.6s wait:19s done:1s)
0:01:13 INF [archive] - 	  encode  4x(57% 12s wait:2s done:1s)
0:01:13 INF [archive] - 	  write   1x(20% 4s wait:14s)
0:01:13 INF [archive] - ----------------------------------------
0:01:13 INF [archive] - 	archive	108MB
0:01:13 INF [archive] - 	features	292MB
-rw-r--r-- 1 runner runner 109M Nov 21 06:18 run.jar
0:01:09 DEB [archive] - Tile stats:
0:01:09 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (160k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:88k)
2. 9/154/190 (148k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:85k)
3. 10/308/380 (136k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
4. 10/308/381 (136k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
5. 14/4941/6092 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (building:62k)
7. 14/4940/6092 (101k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
8. 14/4946/6112 (99k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:60k)
9. 11/616/762 (99k) https://onthegomap.github.io/planetiler-demo/#11.5/41.7057/-71.63086 (landcover:71k)
10. 11/616/764 (97k) https://onthegomap.github.io/planetiler-demo/#11.5/41.44269/-71.63086 (landcover:74k)
0:01:09 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   336   409   544   872   350   464   573   808  1.6k    2k  6.8k  6.2k  5.6k  4.4k  6.8k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   707   796    1k  1.6k  3.1k    6k  3.6k  1.8k   845   978    6k
            landuse    0     0     0     0   549   695  1.6k  6.7k   17k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   418   972  1.4k  4.6k  6.4k   21k   15k   17k   64k   47k   34k   64k
           waterway    0     0     0     0   112   119     0     0     0    3k  2.2k    2k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.3k  4.3k  9.7k   18k   13k  8.2k  3.7k  3.4k  4.4k   18k
transportation_name    0     0     0     0     0     0   287   364  1.1k  1.9k  5.6k  4.8k  3.9k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.8k   29k   85k   72k   81k   53k   30k   25k   85k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.3k  2.8k  1.4k  1.4k   869  4.3k
         water_name    0     0     0     0     0     0     0     0     0   486   461   433   452  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   88k   88k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   59k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.8k  6.3k   21k   42k   84k  201k  183k  135k  114k  128k  252k  252k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.2k    5k   14k   29k   60k  148k  136k   99k   84k   92k  160k  160k
0:01:09 DEB [archive] -    Max tile: 252k (gzipped: 160k)
0:01:09 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:09 DEB [archive] -     # tiles: 4,115,056
0:01:09 DEB [archive] -  # features: 5,691,811
0:01:09 INF [archive] - Finished in 21s cpu:1m18s avg:3.7
0:01:09 INF [archive] -   read    1x(3% 0.6s wait:19s done:1s)
0:01:09 INF [archive] -   encode  4x(56% 12s wait:2s)
0:01:09 INF [archive] -   write   1x(21% 4s wait:15s)
0:01:09 INF [archive] - Finished in 1m10s cpu:3m53s gc:1s avg:3.3
0:01:09 INF [archive] - FINISHED!
0:01:09 INF [archive] - 
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - data errors:
0:01:09 INF [archive] - 	render_snap_fix_input	16,947
0:01:09 INF [archive] - 	osm_multipolygon_missing_way	360
0:01:09 INF [archive] - 	osm_boundary_missing_way	55
0:01:09 INF [archive] - 	merge_snap_fix_input	15
0:01:09 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	6
0:01:09 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:09 INF [archive] - 	render_snap_fix_input2	1
0:01:09 INF [archive] - 	omt_fix_water_before_ne_intersect	1
0:01:09 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - 	overall          1m10s cpu:3m53s gc:1s avg:3.3
0:01:09 INF [archive] - 	lake_centerlines 2s cpu:6s avg:2.5
0:01:09 INF [archive] - 	  read     1x(22% 0.5s done:2s)
0:01:09 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:09 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:09 INF [archive] - 	water_polygons   16s cpu:44s avg:2.8
0:01:09 INF [archive] - 	  read     1x(41% 6s done:7s)
0:01:09 INF [archive] - 	  process  4x(26% 4s wait:4s done:6s)
0:01:09 INF [archive] - 	  write    1x(3% 0.5s wait:10s done:5s)
0:01:09 INF [archive] - 	natural_earth    7s cpu:13s avg:2
0:01:09 INF [archive] - 	  read     1x(96% 6s)
0:01:09 INF [archive] - 	  process  4x(12% 0.8s wait:6s)
0:01:09 INF [archive] - 	  write    1x(0% 0s wait:6s)
0:01:09 INF [archive] - 	osm_pass1        2s cpu:7s avg:3.2
0:01:09 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:09 INF [archive] - 	  parse    4x(31% 0.7s wait:1s)
0:01:09 INF [archive] - 	  process  1x(72% 2s)
0:01:09 INF [archive] - 	osm_pass2        20s cpu:1m20s avg:4
0:01:09 INF [archive] - 	  read     1x(0% 0s wait:13s done:7s)
0:01:09 INF [archive] - 	  process  4x(76% 15s)
0:01:09 INF [archive] - 	  write    1x(2% 0.5s wait:20s)
0:01:09 INF [archive] - 	ne_lakes         0s cpu:0s avg:13.9
0:01:09 INF [archive] - 	boundaries       0s cpu:0s avg:1.4
0:01:09 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:09 INF [archive] - 	sort             1s cpu:4s avg:2.6
0:01:09 INF [archive] - 	  worker  1x(52% 0.7s)
0:01:09 INF [archive] - 	archive          21s cpu:1m18s avg:3.7
0:01:09 INF [archive] - 	  read    1x(3% 0.6s wait:19s done:1s)
0:01:09 INF [archive] - 	  encode  4x(56% 12s wait:2s)
0:01:09 INF [archive] - 	  write   1x(21% 4s wait:15s)
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - 	archive	108MB
0:01:09 INF [archive] - 	features	292MB
-rw-r--r-- 1 runner runner 109M Nov 21 06:19 run.jar

Full logs: https://github.com/onthegomap/planetiler/actions/runs/19561838010

Copy link
Contributor

@msbarry msbarry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this change! A few minor comments and it looks like some tests in openmaptiles need to be changed due to this behavior. Easiest might be to disable those tests in this repo temporararily then we can update omt after this merges. Can you revert #1339 and add the failing tests to exclude.txt?

*/
private static long makeMergedId(long id) {
return (id / 10) * 10;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! I'm realizing this will update IDs on all elements, not just OSM elements. I'm not sure that is too important as the IDs are less meaningful in other sources.

I just double checked and confirmed that all features (even non-osm ones) get multiplied by config.featureSourceIdMultiplier() so it should be safe to do this for all features without causing unwanted collisions. Could we use that instead of a hardcoded 10 constant though?

@sonarqubecloud
Copy link

@jake-low
Copy link
Author

Thanks for the review! I implemented the suggested VectorTile.Feature.copyWithIdAndGeometry() helper function, and I added an exclude.txt list based on the PR you linked in order to temporarily disable the failing OMT tests.

I wasn't sure how to use config.featureSourceIdMultiplier() instead of the hardcoded constant. I don't think FeatureMerge has access to config itself, so I think the caller would have to pass either the config or the multiplier in as an argument when calling a merge function such as mergeLineStrings()? This would be a breaking change to the API though, unless we provide a version that uses a default value like 10.

There are still two checks that are failing. One is the standalone example, which I think is failing since the tests have been changed to expect the new behavior but it's fetching the current 0.9.4 build to test with, which has the old behavior. The other is the link checker, which looks like it's failing to fetch some links from Medium, so probably unrelated to these changes? FWIW I can load those links in my browser just fine, so maybe Medium is blocking requests from GitHub Actions as part of an anti-scraping effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] FeatureMerge should (optionally?) assign new IDs to merged features instead of reusing one of the input feature's IDs

2 participants