Skip to content

Conversation

@bassner
Copy link
Member

@bassner bassner commented Nov 27, 2025

Checklist

General

Server

Motivation and Context

The test testCreateLectureSeriesAndCorrectLectureAndChannelNames in LectureIntegrationTest was flaky, failing approximately 50% of the time in CI/CD with:

org.opentest4j.AssertionFailedError: 
expected: "Requirements"
 but was: "Lecture 1"

Description

Root Cause: In initTestCase(), course1.getLectures() returns a Set<Lecture> with non-deterministic iteration order. The test then called getFirst() and getLast() on this set converted to a list, causing lecture1 and lecture2 to be assigned to different physical lectures between test runs.

Fix: Sort lectures by startDate before assigning them to ensure lecture1 is always the chronologically earlier lecture and lecture2 is the later one.

// Before:
var lectures = this.course1.getLectures().stream().toList();

// After:
var lectures = this.course1.getLectures().stream().sorted(Comparator.comparing(Lecture::getStartDate)).toList();

Steps for Testing

  1. Run the test multiple times locally:
    ./gradlew test --tests "de.tum.cit.aet.artemis.lecture.LectureIntegrationTest.testCreateLectureSeriesAndCorrectLectureAndChannelNames" --rerun-tasks
  2. Verify it passes consistently (tested 5 consecutive runs successfully)

Review Progress

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1 (ran test 5 times locally, all passed)

Test Coverage

Class/File Line Coverage Confirmation (assert/expect)
LectureIntegrationTest.java N/A (test file)

Summary by CodeRabbit

  • Tests
    • Improved test reliability by enforcing deterministic ordering when preparing and retrieving lecture lists, ensuring consistent assertions and more stable test execution.

✏️ Tip: You can customize this high-level summary in your review settings.

@bassner bassner requested a review from a team as a code owner November 27, 2025 12:58
@bassner bassner added this to the 8.6.2 milestone Nov 27, 2025
@bassner bassner self-assigned this Nov 27, 2025
@github-project-automation github-project-automation bot moved this to Work In Progress in Artemis Development Nov 27, 2025
@github-actions github-actions bot added tests lecture Pull requests that affect the corresponding module labels Nov 27, 2025
@bassner bassner changed the title Development: Fix flaky LectureIntegrationTest Development: Fix flaky LectureIntegrationTest Nov 27, 2025
The test testCreateLectureSeriesAndCorrectLectureAndChannelNames was
flaky because course1.getLectures() returns a Set with non-deterministic
iteration order. This caused lecture1 and lecture2 to be assigned
inconsistently between test runs, leading to ~50% failure rate.

Fix: Sort lectures by startDate before assigning to ensure consistent
assignment of lecture1 (earlier date) and lecture2 (later date).
@bassner bassner force-pushed the bugfix/lectures/flaky-series-test branch from 671f0c3 to e72e2ef Compare November 27, 2025 13:00
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 27, 2025

Walkthrough

Test updated to enforce deterministic ordering by sorting lecture lists by startDate in both initialization and post-creation retrieval paths; local variable lectures renamed to lecturesAfter and all assertions and downstream references updated to use the sorted collection.

Changes

Cohort / File(s) Change Summary
Deterministic test ordering
src/test/java/de/tum/cit/aet/artemis/lecture/LectureIntegrationTest.java
Adds sorting by startDate to lecture lists in initTestCase and in the post-creation retrieval (lecturesAfter). Renames local variable from lectures to lecturesAfter and updates all assertions and downstream derivations (firstLecture, secondLecture, etc.) to use the sorted collection. No public API or exported declarations changed.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Single test file with consistent, repetitive edits (sorting insertion and variable rename).
  • Review focus:
    • Ensure all occurrences of the renamed variable were updated.
    • Confirm both locations apply identical sorting by startDate.
    • Validate assertions still reflect intended test semantics (order-independent comparisons now use a consistently ordered source).

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Development: Fix flaky LectureIntegrationTest' directly addresses the main change: fixing a flaky test in LectureIntegrationTest by sorting lectures deterministically.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch bugfix/lectures/flaky-series-test

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

  • Provide your own instructions using the high_level_summary_instructions setting.
  • Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
  • Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

  1. 📝 Description — Summarize the main change in 50–60 words, explaining what was done.
  2. 📓 References — List relevant issues, discussions, documentation, or related PRs.
  3. 📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.
  4. 📊 Contributor Summary — Include a Markdown table showing contributions:
    | Contributor | Lines Added | Lines Removed | Files Changed |
  5. ✔️ Additional Notes — Add any extra reviewer context.
    Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link

End-to-End (E2E) Test Results Summary

TestsPassed ☑️Skipped ⚠️Failed ❌️Time ⏱
End-to-End (E2E) Test Report215 ran211 passed3 skipped1 failed1h 3m 29s 143ms
TestResultTime ⏱
End-to-End (E2E) Test Report
e2e/exercise/programming/ProgrammingExerciseParticipation.spec.ts
ts.Programming exercise participation › Programming exercise team participation › Check team participation › Instructor checks the participation❌ failure8s 252ms

@maximiliansoelch maximiliansoelch added ready for review maintainer-approved The feature maintainer has approved the PR labels Nov 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lecture Pull requests that affect the corresponding module maintainer-approved The feature maintainer has approved the PR ready for review tests

Projects

Status: Work In Progress

Development

Successfully merging this pull request may close these issues.

3 participants