Skip to content

Conversation

@pixlwave
Copy link
Member

@pixlwave pixlwave commented Oct 30, 2025

The UI and integration tests are currently failing due to a missing (OS version) in the run_tests. Whilst playing with #4662 and #4664 I came up with a few simplifications so applied them all here at once.

UI Tests run: https://github.com/element-hq/element-x-ios/actions/runs/18945869921
Integration Tests run: https://github.com/element-hq/element-x-ios/actions/runs/18945866100
Accessibility Test run: https://github.com/element-hq/element-x-ios/actions/runs/18945872002

@pixlwave pixlwave requested a review from a team as a code owner October 30, 2025 15:23
@pixlwave pixlwave requested review from Velin92 and removed request for a team October 30, 2025 15:23
@pixlwave pixlwave changed the title Stop repeating the simulator version through the Fastfile Stop repeating the simulator version through the Fastfile. Oct 30, 2025
@pixlwave pixlwave added the pr-build for changes related to build, tools, CI/CD label Oct 30, 2025
@sonarqubecloud
Copy link

@codecov
Copy link

codecov bot commented Oct 30, 2025

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
1143 4 1139 0
View the top 1 failed test(s) by shortest run time
RoomMembersListScreenUITests::testJoinedAndInvitedMembers()
Stack Traces | 16.2s run time
failed - Snapshot "testJoinedAndInvitedMembers-iPhone-26.0-en-GB" does not match reference.

@−
"file:.../__Snapshots__/Application/roomMembersListScreen.testJoinedAndInvitedMembers-iPhone-26-0-en-GB.png"
@+
"file:.../__FailedScreenshots__/Application/roomMembersListScreen.testJoinedAndInvitedMembers-iPhone-26-0-en-GB.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.95947266 is less than required 0.99
The lowest perceptual color precision 0.0006250143 is less than required 0.98 (UITests/Sources/Application.swift:89)
View the full list of 3 ❄️ flaky test(s)
DeferredFulfillmentTests::testObservableAsynchronousUpdate()

Flake rate in main: 32.42% (Passed 221 times, Failed 106 times)

Stack Traces | 11.9s run time
Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Awaiting stream". (.../Other/Extensions/XCTestCase.swift:77)
EncryptionSettingsUITests::testFlow()

Flake rate in main: 33.67% (Passed 195 times, Failed 99 times)

Stack Traces | 28.6s run time
failed - Snapshot "testFlow-iPhone-26.0-en-GB-1" does not match reference.

@−
"file:.../__Snapshots__/Application/encryptionSettings.testFlow-iPhone-26-0-en-GB-1.png"
@+
"file:.../__FailedScreenshots__/Application/encryptionSettings.testFlow-iPhone-26-0-en-GB-1.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.28466797 is less than required 0.99
The lowest perceptual color precision 0.0006250143 is less than required 0.98 (UITests/Sources/Application.swift:89)
UserFlowTests::testUserFlow()

Flake rate in main: 41.42% (Passed 99 times, Failed 70 times)

Stack Traces | 400s run time
XCTAssertTrue failed (IntegrationTests/Sources/UserFlowTests.swift:44)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@pixlwave pixlwave changed the title Stop repeating the simulator version through the Fastfile. Stop repeating the simulator version throughout the Fastfile. Oct 30, 2025
@pixlwave
Copy link
Member Author

Seems like all the test suites run on the simulators fine (even if they aren't passing).

@pixlwave pixlwave merged commit 010500a into develop Oct 31, 2025
10 of 15 checks passed
@pixlwave pixlwave deleted the doug/fastlane branch October 31, 2025 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-build for changes related to build, tools, CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants