-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Description
Currently, there's only metadata support for partition aligned stitching (disjunctive constraints, i.e., constraints which only apply along discrete boundaries, and which must be OR'd together to form the complete result set). We would like to add support for conjunctive predicates, meaning, predicates which would need to be ANDed to each discrete boundary. An example of this could include a time travel identifier used to incrementally scanned only changed data in an Iceberg table.
Expected Behavior or Use Case
Include support for conjunctive predicates in materialized views
Presto Component, Service, or Connector
Optimizer, SPI
Possible Implementation
After implementing the data structure in the MaterializedViewStatus class, enhance MaterializedViewRewrite to include a more sophisticated rewrite for compatible queries which can take advantage of any conjunctive predicates. This could include filter queries and joins where only one side is stale.
Example Screenshots (if appropriate):
Context
We would like to use Iceberg's incremental scan techniques to provide more efficient stitching and refresh capabilities.
Metadata
Metadata
Assignees
Type
Projects
Status