-
Notifications
You must be signed in to change notification settings - Fork 9
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Feature request
As per the discussion here: MaterializeInc/materialize#27907 and based on the design doc here: #645 we need to introduce the support for tables from sources on the Terraform provider side as well.
Extra details to be discussed in the design doc pull request, but the main things to keep in mind are:
### Initial tasks
- [x] Deprecated the related table attributes on the source resources, eg. MySQL, Postgres, Loadgens
- [x] Introduce new `resource_source_table_{mysql|postgres|load_generator}` resources
- [x] Add tests including unit tests and acceptance tests
- [x] Handle source-specific attributes like the MySQL `ignore_columns` in the `resource_table_from_source` resource
- [x] Update all of the related documentation
- [x] Write a migrations guide for users using the old implementation
- [x] Add data sources `resource_source_table` resources
- [x] Add import and documentation examples
- [x] Once ready add upstream table name to read query and read funcs + acceptance tests
- [x] Add integration tests
### Kafka Tasks
- [x] Deprecated the related table attributes on the source resources for the Kafka source resource
- [x] Introduce a new `resource_source_table_kafka` resource
- [x] Add tests including unit tests
- [x] Update all of the related Kafka source documentation
- [x] Add import and documentation examples
- [x] Update the migrations guide for users using the old implementation to include Kafka sources
- [x] Add acceptance tests
- [x] Add integration tests
- [ ] Add the envelope, key_format and value_format columns to the read func
### Webhook Tasks
- [x] Deprecated the related table attributes on the source resources for the Webhook source resource
- [x] Introduce a new `resource_source_table_webhook` resource
- [ ] Add tests including unit tests, integration tests and acceptance tests
- [ ] Update all of the related Webhook source documentation
- [x] Add import and documentation examples
- [ ] Update the migrations guide for users using the old implementation to include webhooks sources
### Additional Tasks
- [x] Data source for `mz_internal.available_source_references`
- [x] Add unit tests for the source reference data source
- [x] Add acceptance tests for the source reference data source
- [x] Add `ALTER SOURCE .. REFRESH REFERENCES`
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request