Skip to content

Conversation

@leeandher
Copy link
Member

Should resolve some issues with data forwarders

  • The SQS forwarder was using as_dict instead of serialize
  • The rate limiting wasn't being used at all
  • It was hard to compare the plugin implementation to the data forwarding since the methods had their name changed, I reused the existing ones where possible

For Splunk, I opted to use the legacy SplunkApiClient, we can migrate off of it later and it's easier to match it for now.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 24, 2025
@codecov
Copy link

codecov bot commented Nov 24, 2025

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
30041 4 30037 241
View the top 3 failed test(s) by shortest run time
tests.sentry.core.endpoints.test_organization_index.OrganizationsCreateTest::test_valid_slugs
Stack Traces | 1.85s run time
#x1B[1m#x1B[.../core/endpoints/test_organization_index.py#x1B[0m:190: in test_valid_slugs
    response = self.get_success_response(name=input_slug, slug=input_slug)
#x1B[1m#x1B[.../sentry/testutils/cases.py#x1B[0m:628: in get_success_response
    assert_status_code(response, 200, 300)
#x1B[1m#x1B[.../sentry/testutils/asserts.py#x1B[0m:47: in assert_status_code
    assert minimum <= response.status_code < maximum, (
#x1B[1m#x1B[31mE   AssertionError: (500, b'{"detail":"Internal Error","errorId":null}')#x1B[0m
#x1B[1m#x1B[31mE   assert 500 < 300#x1B[0m
#x1B[1m#x1B[31mE    +  where 500 = <Response status_code=500, "application/json">.status_code#x1B[0m
tests.sentry.integrations.data_forwarding.splunk.test_forwarder.SplunkDataForwarderTest::test_simple_notification
Stack Traces | 2.75s run time
#x1B[1m#x1B[.../data_forwarding/splunk/test_forwarder.py#x1B[0m:43: in test_simple_notification
    assert len(responses.calls) == 1
#x1B[1m#x1B[31mE   assert 0 == 1#x1B[0m
#x1B[1m#x1B[31mE    +  where 0 = len(<responses.CallList object at 0x7f786402d010>)#x1B[0m
#x1B[1m#x1B[31mE    +    where <responses.CallList object at 0x7f786402d010> = responses.calls#x1B[0m
tests.sentry.integrations.data_forwarding.splunk.test_forwarder.SplunkDataForwarderTest::test_reraise_error
Stack Traces | 2.81s run time
#x1B[1m#x1B[.../data_forwarding/splunk/test_forwarder.py#x1B[0m:76: in test_reraise_error
    with pytest.raises(ApiError):
#x1B[1m#x1B[31mE   Failed: DID NOT RAISE <class 'sentry.shared_integrations.exceptions.ApiError'>#x1B[0m
tests.sentry.integrations.data_forwarding.amazon_sqs.test_forwarder.AmazonSQSDataForwarderTest::test_simple_notification
Stack Traces | 3.01s run time
#x1B[1m#x1B[.../data_forwarding/amazon_sqs/test_forwarder.py#x1B[0m:54: in test_simple_notification
    mock_client.return_value.send_message.assert_called_once_with(
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/unittest/mock.py#x1B[0m:989: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/unittest/mock.py#x1B[0m:977: in assert_called_with
    raise AssertionError(_error_message()) from cause
#x1B[1m#x1B[31mE   AssertionError: expected call not found.#x1B[0m
#x1B[1m#x1B[31mE   Expected: send_message(QueueUrl='https://sqs.us-east-1.amazonaws.com/12345678/myqueue', MessageBody='{"event_id":"275df5df24734e1985f4e419454b248f","project":4557175778770944,"release":null,"dist":null,"platform":"other","message":"","datetime":"2025-11-25T21:19:53.990643Z","tags":[["level","error"],["user","id:1"]],"_metrics":{"bytes.stored.event":825},"_ref":4557175778770944,"_ref_version":2,"culprit":"","exception":{"values":[{"type":"ValueError","value":"foo bar"}]},"fingerprint":["{{ default }}"],"grouping_config":{"id":"newstyle:2023-01-11","enhancements":"KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA#KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA#KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA"},"hashes":["8fcbb655febbd3953506cb000b250870"],"level":"error","location":null,"logger":"","metadata":{"value":"foo bar","type":"ValueError"},"nodestore_insert":1764105594.061727,"received":1764105593.990643,"timestamp":1764105593.990643,"title":"ValueError: foo bar","type":"error","user":{"id":"1","email":"[email protected]","sentry_user":"id:1"},"version":"5"}')#x1B[0m
#x1B[1m#x1B[31mE     Actual: send_message(QueueUrl='https://sqs.us-east-1.amazonaws.com/12345678/myqueue', MessageBody='{"id":"275df5df24734e1985f4e419454b248f","groupID":"80","eventID":"275df5df24734e1985f4e419454b248f","projectID":"4557175778770944","size":886,"entries":[{"data":{"values":[{"type":"ValueError","value":"foo bar","mechanism":null,"threadId":null,"module":null,"stacktrace":null,"rawStacktrace":null}],"hasSystemFrames":false,"excOmitted":null},"type":"exception"}],"dist":null,"message":"","title":"ValueError: foo bar","location":null,"user":{"id":"1","email":"[email protected]","username":null,"ip_address":null,"name":null,"geo":null,"data":null},"contexts":{},"sdk":null,"context":{},"packages":{},"type":"error","metadata":{"value":"foo bar","type":"ValueError"},"tags":[{"key":"level","value":"error"},{"key":"user","value":"id:1","query":"user.id:\\"1\\""}],"platform":"other","dateReceived":"2025-11-25T21:19:53.990643Z","errors":[],"occurrence":null,"_meta":{"entries":{},"message":null,"user":null,"contexts":null,"sdk":null,"context":null,"packages":null,"tags":{}},"crashFile":null,"culprit":"","dateCreated":"2025-11-25T21:19:53.990643Z","fingerprints":["8fcbb655febbd3953506cb000b250870"],"groupingConfig":{"id":"newstyle:2023-01-11","enhancements":"KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA#KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA#KLUv_SAd6QAAkwORuGFsbC1wbGF0Zm9ybXM6MjAyMy0wMS0xMZA"}}')#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@leeandher leeandher marked this pull request as ready for review November 25, 2025 21:58
@leeandher leeandher requested review from a team as code owners November 25, 2025 21:58
) -> dict[str, Any]:
return {
"time": int(event.datetime.timestamp()),
"time": int(event.datetime.strftime("%s")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Non-portable timestamp conversion using strftime

The strftime("%s") format code is a Unix-specific extension not supported on Windows or in standard Python. This will cause the Splunk forwarder to fail on Windows systems. The portable way to get a Unix timestamp is using event.datetime.timestamp() instead of int(event.datetime.strftime("%s")).

Fix in Cursor Fix in Web

}
props["tags"] = [[tagstore.backend.get_standardized_key(k), v] for k, v in event.tags]
props["tags"] = [
[k.format(tagstore.backend.get_standardized_key(k)), v] for k, v in event.tags
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Incorrect string format call on tag key

The tag processing incorrectly calls k.format(tagstore.backend.get_standardized_key(k)) where k is a tag key string. The format() method expects format placeholders like {}, but tag keys don't contain these. This should be just tagstore.backend.get_standardized_key(k) without the .format() call, which will either fail or produce unexpected output.

Fix in Cursor Fix in Web

@leeandher leeandher merged commit 991fec4 into master Nov 26, 2025
66 checks passed
@leeandher leeandher deleted the leander/ref-forwarding branch November 26, 2025 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants