Dataflow Templates 2025-11-18-00_RC00

[tests] Add integration tests for IDENTITY columns to bulk, live, and…

Dataflow Templates 2025-11-11-00_RC00

What's Changed

Package Upgrades

• chore(deps): update Python dependencies by @github-actions[bot] in #2979

Full Changelog: 2025-11-06-00_RC00...2025-11-11-00_RC00

Dataflow Templates 2025-11-06-00_RC00

What's Changed

Improvements

• Delete v2/datastream-to-postgres directory by @dhercher in #2959
• [SpannerToSourceDb] Improving error classification in Reverse replication template by @darshan-sj in #2964
• Update to beam 2.69 by @Abacn in #2966

Bug fixes

• fixing retry count in reverse replication by @shreyakhajanchi in #2949

Package Upgrades

• chore(deps): update Python dependencies by @github-actions[bot] in #2956

Full Changelog: 2025-10-28-01_RC00...2025-11-06-00_RC00

Dataflow Templates 2025-11-05-00_RC00

fix(build): update surefire report processing to handle time format (…

Dataflow Templates 2025-10-28-01_RC00

What's Changed

Improvements

• fix(kafka): handle domain-scoped project IDs in bootstrap server URL by @liferoad in #2908
• Yaml: Add yaml syntax UT by @derrickaw in #2872
• [SpannerToSourceDb] Fixing CDC events processing logic for Reverse replication template by @darshan-sj in #2919
• Apply GCP Application Default authentication when using a Managed Service for Apache Kafka cluster. by @fozzie15 in #2925

Bug fixes

• Remove special handling for only PK column tables by @manitgupta in #2929
• [yaml] - move syntax test verification to validate section by @derrickaw in #2936

Package Upgrades

• chore(deps): update Python dependencies by @github-actions[bot] in #2899
• build: update jetty dependencies to 12.0.12 to fix CVE-2024-6763 by @stankiewicz in #2921
• build: update org.apache.commons:commons-configuration to resolve CVE-2024-29131 by @stankiewicz in #2922

New Contributors

• @LucaDai made their first contribution in #2898

Full Changelog: 2025-10-21-00_RC00...2025-10-28-01_RC00

Dataflow Templates 2025-10-28-00_RC00

bump org.apache.commons:commons-configuration to resolve CVE-2024-291…

Dataflow Templates 2025-10-21-00_RC00

What's Changed

Improvements

• KafkaToBigQueryYaml - change hidden to false by @derrickaw in #2879
• Add option to unwrap messages replayed from DLQ to avoid nesting loops. by @claudevdm in #2888

Bug fixes

• fix dependabot labeling workflow name by @derrickaw in #2886
• fix label dependabot workflow by @derrickaw in #2890

Full Changelog: 2025-10-14-00_RC00...2025-10-21-00_RC00

Dataflow Templates 2025-10-14-00_RC00

What's Changed

Improvements

• feat(kafka-to-bigquery): Fix dynamic destination connection churn by @anpag in #2873

Bug fixes

• changed mysql event ordering in datastream to spanner by @bharadwaj-aditya in #2867

New Contributors

• @tianz101 made their first contribution in #2871
• @anpag made their first contribution in #2873

Full Changelog: 2025-10-07-00_RC00...2025-10-14-00_RC00

Dataflow Templates 2025-10-07-00_RC00

What's Changed

Improvements

• Fixed a performance regression in the Spanner change streams to BigQuery template that was introduced in the 2024-09-10-00_RC00.
• Only read schema when the col in mod is not in the stored schema. by @jiangzzhu in #2840
• Skip reading schema if the value capture mode is NEW_ROW_AND_OLD_VALUES. by @jiangzzhu in #2839
• feat: Add support for property graph on view by @jennyjiang98 in #2813
• Remove jinja variables option code by @derrickaw in #2826

New Contributors

• @jiangzzhu made their first contribution in #2840

Full Changelog: 2025-10-01-00_RC00...2025-10-07-00_RC00

Dataflow Templates 2025-10-01-00_RC00

What's Changed

Improvements

• Add Run, Staging and Validation stages for KafkaToBigquery Yaml template by @derrickaw in #2782

Bug fixes

• Fix vector index detection by @fbiville in #2151
• Change default CPU priority of bulk template to HIGH by @manitgupta in #2836

Package Upgrades

• Upgrade to Beam 2.68.0 by @Amar3tto in #2830

Known issues

Users of Spanner change streams to BigQuery template are advised to use the 2024-09-02-00_RC00 version of the template due to a performance regression introduced in the 2024-09-10-00_RC00 release that might affect certain use-cases.

Full Changelog: 2025-09-23-00_RC00...2025-10-01-00_RC00