Skip to content

Add support for automatic incremental refresh of materialized views #26627

@tdcmeehan

Description

@tdcmeehan

We recently added support for WHERE clause-less REFRESH of materialized views, however it has the significant limitation in that it only supports full refresh. We can enhance the plan generated for REFRESH to include the disjunctive predicates to create a partition-by-partition incremental refresh, and for append-only tables, also include support for conjunctive predicates.

Expected Behavior or Use Case

Avoid more instances which require full refresh and make incremental refresh the default

Presto Component, Service, or Connector

Planner

Possible Implementation

Update the planner to incorporate the predicates from the MaterializedViewStatus, propagate them to relevant table scan nodes to improve the performance of refresh and limit data required to be rewritten.

Example Screenshots (if appropriate):

Context

Incremental refresh is a usability improvement for materialized views which makes them easier and more user friendly.

Metadata

Metadata

Assignees

Type

No type

Projects

Status

🆕 Unprioritized

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions