terraform test: re-enable aws terraform upgrade CI tests

[bobbyiliev]

Motivation

The yaml merge issue should now be fixed in v0.5.3 as per: MaterializeInc/terraform-aws-materialize#78 so re-enabling the tests.

Missed re-enabling the upgrade pipeline when initially re-enabled the other AWS test here: #33789

Testing this here: https://buildkite.com/materialize/nightly/builds/13686#0199bf4d-2cfe-4c58-9577-a7e6d31b738c

Tips for reviewer

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

[bobbyiliev]

The upgrade is failing because of:

2025-10-07T20:24:59.747575Z  thread 'main' panicked at src/environmentd/src/environmentd/main.rs:670:9:
--
  | environmentd: fatal: incompatible persist version 0.160.0-dev.0, current: 0.147.17, make sure to upgrade the catalog one version forward at a time
  | 6: core::panicking::panic_fmt
  | 7: mz_environmentd::environmentd::main::main
  | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
  | /usr/local/bin/entrypoint.sh: line 19:     8 Aborted                 (core dumped) environmentd "$@"

[def-]

It's an upgrade test, so it's expected that we start out at v0.147.X (self-managed v25.2) and then upgrade to main, which should work! This sounds like our allowlist for what versions we allow upgrading from might be failing?

materialize/src/persist-client/src/cfg.rs

Lines 39 to 45 in 22a04c2

	// Ignores the patch version
	const SELF_MANAGED_VERSIONS: &[Version] = &[
	// 25.1
	Version::new(0, 130, 0),
	// 25.2
	Version::new(0, 147, 0),
	];

[bobbyiliev]

It's an upgrade test, so it's expected that we start out at v0.147.X (self-managed v25.2) and then upgrade to main, which should work! This sounds like our allowlist for what versions we allow upgrading from might be failing?

Oh I see, you mean we need 0.160.x in there? Is it because we're now doing weekly releases of the Helm chart that we'd need to bump the SELF_MANAGED_VERSIONS list every week to match the latest dev releases so the upgrade tests pass? I might be missing some of the recent context here on how the new SH releases work now.

[def-]

No. The way it's supposed to work is that you can upgrade to main from self-managed v25.1 and v25.2. @bkirwi or @DAlperin might have an idea as to why it's failing now. This definitely worked before I disabled the tests a few months ago.

[bkirwi]

That error is reporting a downgrade: from 0.160.0-dev.0 in the shard to a current version of 0.147.17. Since we don't support downgrades, I suspect the test is either doing something weird or this is correctly fencing out an old version and your problem is elsewhere.

Note that the test reports a second panic:

2025-10-07T20:28:19.857220Z  thread 'tokio:work-3' panicked at src/repr/src/row/encode.rs:150:9:
--
  | tried pushing Null into non-nullable column
  | 6: core::panicking::panic_fmt
  | 7: <mz_repr::row::encode::RowColumnarEncoder as mz_persist_types::columnar::ColumnEncoder<mz_repr::row::Row>>::append
  | 8: <mz_persist_types::part::PartBuilder<mz_storage_types::sources::SourceData, mz_repr::relation::RelationDesc, (), mz_persist_types::codec_impls::UnitSchema>>::push::<mz_repr::timestamp::Timestamp, i64>

This error was introduced temporarily, and then fixed, on main recently. You may want to rebase to avoid the issue.

[def-]

Rebased and new run triggered: https://buildkite.com/materialize/nightly/builds/13747

[bobbyiliev]

It seems to have worked this time: https://buildkite.com/materialize/nightly/builds/13754#0199e885-7df0-4d35-b5f7-3936937ae827

I'll do another test just to be safe.

[def-]

Awesome, thank you Bobby for getting this working!