Skip to content

Conversation

@kcons
Copy link
Member

@kcons kcons commented Nov 8, 2025

DataCondition failures are currently reported to Sentry/logs individually, then treated as a non-triggering.
Here we return a special error type, so that the caller knows the result isn't necessarily reliable, and in DataConditionGroup logic we aggregate these results in such a way that we know at the end if an untrustworthy result influenced the outcome.

These "tainted" results are reported for each Detector evaluation, so we can track how often our results were influenced by errors, giving us a better high level picture of user impact than we'd have purely by looking at individual reported error counts.

This pattern will be extended to cover over DataCondition evaluation cases.

@kcons kcons requested review from a team as code owners November 8, 2025 00:16
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 8, 2025
@codecov
Copy link

codecov bot commented Nov 8, 2025

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
29552 2 29550 243
View the top 2 failed test(s) by shortest run time
tests.sentry.workflow_engine.models.test_data_condition.EvaluateValueTest::test_condition_result_comparison_fails
Stack Traces | 1.11s run time
#x1B[1m#x1B[.../workflow_engine/models/test_data_condition.py#x1B[0m:97: in test_condition_result_comparison_fails
    assert dc.evaluate_value(2) is None
#x1B[1m#x1B[31mE   AssertionError: assert ConditionError(msg='Invalid condition result') is None#x1B[0m
#x1B[1m#x1B[31mE    +  where ConditionError(msg='Invalid condition result') = <bound method DataCondition.evaluate_value of <DataCondition at 0x7f3fec3b9130: id=129, type=<Condition.GREATER: 'gt'>, comparison=1.0, condition_result='wrong', condition_group_id=86>>(2)#x1B[0m
#x1B[1m#x1B[31mE    +    where <bound method DataCondition.evaluate_value of <DataCondition at 0x7f3fec3b9130: id=129, type=<Condition.GREATER: 'gt'>, comparison=1.0, condition_result='wrong', condition_group_id=86>> = <DataCondition at 0x7f3fec3b9130: id=129, type=<Condition.GREATER: 'gt'>, comparison=1.0, condition_result='wrong', condition_group_id=86>.evaluate_value#x1B[0m
tests.sentry.workflow_engine.models.test_data_condition.GetConditionResultTest::test_str
Stack Traces | 1.15s run time
#x1B[1m#x1B[.../workflow_engine/models/test_data_condition.py#x1B[0m:21: in test_str
    assert dc.get_condition_result() is None
#x1B[1m#x1B[31mE   AssertionError: assert ConditionError(msg='Invalid condition result') is None#x1B[0m
#x1B[1m#x1B[31mE    +  where ConditionError(msg='Invalid condition result') = <bound method DataCondition.get_condition_result of <DataCondition at 0x7f1e28340cb0: id=144, type=<Condition.EQUAL: 'eq'>, comparison='10', condition_result='wrong', condition_group_id=96>>()#x1B[0m
#x1B[1m#x1B[31mE    +    where <bound method DataCondition.get_condition_result of <DataCondition at 0x7f1e28340cb0: id=144, type=<Condition.EQUAL: 'eq'>, comparison='10', condition_result='wrong', condition_group_id=96>> = <DataCondition at 0x7f1e28340cb0: id=144, type=<Condition.EQUAL: 'eq'>, comparison='10', condition_result='wrong', condition_group_id=96>.get_condition_result#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@kcons
Copy link
Member Author

kcons commented Nov 8, 2025

TODO: document how DataConditions should handle failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants