ref: Create a prevent config model, no longer use organization options #101831

JerrySentry · 2025-10-20T21:34:44Z

Creating a new table to manage prevent ai configurations, no longer use the previous method of managing it in organization_options. This will give more flexibility to storing configs across multiple git orgs and sentry orgs. Also we got feedback that we want members to be allowed to update config for the PR Review in order to change the type and frequency of reviews. Since this is in beta, there are no billing implications (where a member can create much higher usage that would then get billed). This is aligned with like Seer project settings and Github Integrations settings which allows members to edit.

codecov · 2025-10-20T21:44:03Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #101831      +/-   ##
===========================================
+ Coverage   76.91%    80.96%   +4.05%     
===========================================
  Files        8728      8718      -10     
  Lines      388272    387495     -777     
  Branches    24611     24477     -134     
===========================================
+ Hits       298646    313754   +15108     
+ Misses      89268     73393   -15875     
+ Partials      358       348      -10

github-actions · 2025-10-20T21:45:35Z

This PR has a migration; here is the generated SQL for src/sentry/prevent/migrations/0001_initial_prevent_ai_configuration.py

for 0001_initial_prevent_ai_configuration in prevent

--
-- Create model PreventAIConfiguration
--
CREATE TABLE "sentry_preventaiconfiguration" ("id" bigint NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "date_updated" timestamp with time zone NOT NULL, "date_added" timestamp with time zone NOT NULL, "organization_id" bigint NOT NULL, "provider" varchar(64) NOT NULL, "git_organization_id" varchar(255) NOT NULL, "data" jsonb NOT NULL);
CREATE UNIQUE INDEX CONCURRENTLY "sentry_preventaiconfigur_organization_id_provider_eb547f9a_uniq" ON "sentry_preventaiconfiguration" ("organization_id", "provider", "git_organization_id");
ALTER TABLE "sentry_preventaiconfiguration" ADD CONSTRAINT "sentry_preventaiconfigur_organization_id_provider_eb547f9a_uniq" UNIQUE USING INDEX "sentry_preventaiconfigur_organization_id_provider_eb547f9a_uniq";
CREATE INDEX CONCURRENTLY "sentry_preventaiconfiguration_organization_id_a95756f3" ON "sentry_preventaiconfiguration" ("organization_id");

src/sentry/prevent/endpoints/pr_review_config.py

src/sentry/prevent/migrations/0001_initial_prevent_ai_configuration.py

suejung-sentry

nice stuff!
I'm not sure the protocols but we may want to have the migration PR more isolated and do the deletion off the old stuff in a separate PR? Maybe doesn't matter much

JerrySentry · 2025-10-22T19:19:28Z

I'm not sure the protocols but we may want to have the migration PR more isolated and do the deletion off the old stuff in a separate PR? Maybe doesn't matter much

Our old implementation haven't been live yet minus the few testing orgs, we can probably manually migrate it over and delete the old ones.

seer-by-sentry · 2025-10-22T21:41:43Z

src/sentry/overwatch/endpoints/overwatch_rpc.py

 )
 from sentry.models.organization import Organization
 from sentry.models.repository import Repository
+from sentry.prevent.types.config import PREVENT_AI_CONFIG_GITHUB_DEFAULT
 from sentry.silo.base import SiloMode
-from sentry.types.prevent_config import PREVENT_AI_CONFIG_GITHUB_DEFAULT

 logger = logging.getLogger(__name__)



Bug: The RPC endpoint reads config from the old organization_options, while the new UI endpoint writes to the new PreventAIConfiguration model, with no data migration.
_{(Severity: Critical 0.85 | Confidence: 1.00)}

🔍 Detailed Analysis

The PreventPrReviewResolvedConfigsEndpoint in overwatch_rpc.py reads configuration from the old organization.get_option storage. However, the new OrganizationPreventGitHubConfigEndpoint writes settings to the new PreventAIConfiguration model. There is no data migration to move existing settings from the old storage to the new model, nor is there any fallback logic for the read endpoint to check the new model. As a result, any configuration changes made through the new UI will be saved to the new model but will be completely ignored by the Overwatch service, which will continue to use old or default settings from the previous storage location.

💡 Suggested Fix

Update the PreventPrReviewResolvedConfigsEndpoint to read from the new PreventAIConfiguration model. Additionally, create a data migration to transfer existing configurations from organization_options to the PreventAIConfiguration table to ensure continuity for existing users.

🤖 Prompt for AI Agent

Fix this bug. In src/sentry/overwatch/endpoints/overwatch_rpc.py at lines 21-28: The `PreventPrReviewResolvedConfigsEndpoint` in `overwatch_rpc.py` reads configuration from the old `organization.get_option` storage. However, the new `OrganizationPreventGitHubConfigEndpoint` writes settings to the new `PreventAIConfiguration` model. There is no data migration to move existing settings from the old storage to the new model, nor is there any fallback logic for the read endpoint to check the new model. As a result, any configuration changes made through the new UI will be saved to the new model but will be completely ignored by the Overwatch service, which will continue to use old or default settings from the previous storage location.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

Feature is not live, don't need to make migration.

seer-by-sentry · 2025-10-22T21:41:44Z

src/sentry/prevent/endpoints/pr_review_config.py

+        response_data: dict[str, Any] = deepcopy(PREVENT_AI_CONFIG_GITHUB_DEFAULT)
+        response_data["github_organization"][git_organization_id] = request.data
+
+        return Response(response_data, status=200)


Bug: A key in the default config object was renamed from github_organizations to github_organization, but the frontend code was not updated to match.
_{(Severity: High 0.75 | Confidence: 1.00)}

🔍 Detailed Analysis

The default configuration object, PREVENT_AI_CONFIG_GITHUB_DEFAULT, has been modified, renaming the key github_organizations to github_organization. However, multiple frontend components, such as useUpdatePreventAIFeature.tsx and manageReposPanel.tsx, were not updated and still reference the old plural key github_organizations. When the frontend receives a configuration object from the backend using the new singular key, attempts to access config.github_organizations will result in undefined, breaking the UI for managing Prevent AI configurations.

💡 Suggested Fix

Update all frontend references from github_organizations to github_organization to match the new schema defined in the PREVENT_AI_CONFIG_GITHUB_DEFAULT constant. This includes updating type definitions in .tsx files and accessors in hooks and components.

🤖 Prompt for AI Agent

Fix this bug. In src/sentry/prevent/endpoints/pr_review_config.py at lines 87-90: The default configuration object, `PREVENT_AI_CONFIG_GITHUB_DEFAULT`, has been modified, renaming the key `github_organizations` to `github_organization`. However, multiple frontend components, such as `useUpdatePreventAIFeature.tsx` and `manageReposPanel.tsx`, were not updated and still reference the old plural key `github_organizations`. When the frontend receives a configuration object from the backend using the new singular key, attempts to access `config.github_organizations` will result in `undefined`, breaking the UI for managing Prevent AI configurations.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

Frontend code is not live yet, it will be updated next.

cursor

Bug: AI Config Endpoint Bypasses New Model

The PreventPrReviewResolvedConfigsEndpoint still reads prevent AI configuration from organization.get_option("sentry:prevent_ai_config_github"). This bypasses the new PreventAIConfiguration model, causing the endpoint to return default or stale configurations instead of the actual values, impacting external services.

src/sentry/overwatch/endpoints/overwatch_rpc.py#L75-L88

sentry/src/sentry/overwatch/endpoints/overwatch_rpc.py

Lines 75 to 88 in d9ce127

    
                   return auth[0].lower() == self.token_name 
        
               def authenticate_token(self, request: Request, token: str) -> tuple[Any, Any]: 
        
                   compare_signature(request, token) 
        
                   sentry_sdk.get_isolation_scope().set_tag("overwatch_rpc_auth", True) 
        
                   return (AnonymousUser(), token) 
        
           def _can_use_prevent_ai_features(org: Organization) -> bool: 
        
               """Check if organization has opted in to Prevent AI features.""" 
        
               hide_ai_features = org.get_option("sentry:hide_ai_features", HIDE_AI_FEATURES_DEFAULT) 
        
               pr_review_test_generation_enabled = bool(

src/sentry/overwatch/endpoints/overwatch_rpc.py#L103-L107

sentry/src/sentry/overwatch/endpoints/overwatch_rpc.py

Lines 103 to 107 in d9ce127

    
               """ 
        
               publish_status = { 
        
                   "GET": ApiPublishStatus.EXPERIMENTAL, 
        
               }

wedamija

I'd probably suggest splitting out the new migration into a separate pr so that we can get it through more easily

wedamija · 2025-10-22T23:06:16Z

src/sentry/prevent/models.py

+
+    class Meta:
+        app_label = "prevent"
+        db_table = "sentry_preventaiconfiguration"


Since this is in your own app, you can feel free to call this prevent_*

wedamija · 2025-10-22T23:07:16Z

src/sentry/prevent/models.py

+
+    __relocation_scope__ = RelocationScope.Excluded
+
+    organization_id = BoundedBigIntegerField(db_index=True)


Let's use a proper foreign key here

On a similar note, should we consider which silo this data should live in? I could see a case being made for either region or control, depending on where this is used the most.

wedamija · 2025-10-22T23:07:30Z

src/sentry/prevent/models.py

+
+    organization_id = BoundedBigIntegerField(db_index=True)
+    provider = models.CharField(max_length=64)
+    git_organization_id = models.CharField(max_length=255)


Can we link to the integration instead?

wedamija · 2025-10-22T23:08:27Z

src/sentry/prevent/models.py

+    organization_id = BoundedBigIntegerField(db_index=True)
+    provider = models.CharField(max_length=64)
+    git_organization_id = models.CharField(max_length=255)
+    data = models.JSONField(default=dict)


Can we use more explicit fields here, instead of a json blob? What kind of data are we putting here?

JerrySentry · 2025-10-23T14:57:53Z

@wedamija

I'd probably suggest splitting out the new migration into a separate pr so that we can get it through more easily

Thanks for looking! I will move the table to a separate PR and make those schema changes. On using explicit fields instead of a json, it would make the table(s) a lot more confusing and inflexible. The current json object structure is defined like this, and being at very early stages of this product our configs change all the time so I feel its better to not lock it in with DB tables.

wedamija · 2025-10-23T21:31:45Z

@wedamija

I'd probably suggest splitting out the new migration into a separate pr so that we can get it through more easily

Thanks for looking! I will move the table to a separate PR and make those schema changes. On using explicit fields instead of a json, it would make the table(s) a lot more confusing and inflexible. The current json object structure is defined like this, and being at very early stages of this product our configs change all the time so I feel its better to not lock it in with DB tables.

The structure there looks pretty regular to me. Should the config have a link to a 1 -> M table that is defined like

ConfigurationFeature
 - preventaiconfiguration_id
 - name
 - enabled
 - sensitivity
 - triggers list[str] # just list the active triggers rather than a dict of true/false. Could also be a M2M to a triggers table, not sure if it's worth that level of normalization though

My experience with unstructured json blobs is that we always end up regretting it

JerrySentry · 2025-10-24T19:42:58Z

@wedamija

My experience with unstructured json blobs is that we always end up regretting it

Yeah I hear you, thing is the configs for this product is so fluid right now, in a week I can see us use a whole different structure. So I'd rather get this config object more matured and when it comes to migrating to a DB schema later we will have to take that pill. While we are still building this pre-GA product I would prefer to be more flexible and not make DB migrations if possible.

Also the structure would almost work except there's a nested "Repos" object in there that overrides the existing configs for the main "Org" object. 😿

getsantry · 2025-11-18T08:00:14Z

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

suejung-sentry and others added 4 commits October 17, 2025 16:48

fix(prevent): Allow members to update config

2c472c4

cleanup

d00ee36

Create a prevent config model, no longer use organization options

c79771e

update tests

95f1d6b

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 20, 2025

JerrySentry added 2 commits October 20, 2025 17:36

cleanup

03dc73b

cleanup unsued var

4c72266

JerrySentry and others added 3 commits October 20, 2025 18:02

fix import

a266d31

Merge branch 'master' into jerry/extend-config-perm

1a463a9

fix mypy

b872e4a

suejung-sentry reviewed Oct 20, 2025

View reviewed changes

src/sentry/prevent/endpoints/pr_review_config.py Outdated Show resolved Hide resolved

suejung-sentry reviewed Oct 20, 2025

View reviewed changes

src/sentry/prevent/migrations/0001_initial_prevent_ai_configuration.py Outdated Show resolved Hide resolved

suejung-sentry reviewed Oct 20, 2025

View reviewed changes

Merge branch 'master' into jerry/extend-config-perm

c1d2f28

vercel bot deployed to Preview October 22, 2025 15:14 View deployment

rename column to be _id & provider, create new audit log event

03560b1

vercel bot deployed to Preview October 22, 2025 19:18 View deployment

rename to provider column

160fbd6

vercel bot deployed to Preview October 22, 2025 21:12 View deployment

Merge branch 'master' into jerry/extend-config-perm

42d0454

vercel bot deployed to Preview October 22, 2025 21:23 View deployment

JerrySentry marked this pull request as ready for review October 22, 2025 21:38

JerrySentry requested review from a team as code owners October 22, 2025 21:38

JerrySentry requested a review from suejung-sentry October 22, 2025 21:39

This comment was marked as outdated.

Sign in to view

seer-by-sentry bot reviewed Oct 22, 2025

View reviewed changes

fix audit event

d9ce127

vercel bot deployed to Preview October 22, 2025 21:47 View deployment

cursor bot reviewed Oct 22, 2025

View reviewed changes

wedamija reviewed Oct 22, 2025

View reviewed changes

JerrySentry mentioned this pull request Oct 27, 2025

feat(prevent): add table for prevent AI configs #102097

Merged

getsantry bot added the Stale label Nov 18, 2025

	return auth[0].lower() == self.token_name

	def authenticate_token(self, request: Request, token: str) -> tuple[Any, Any]:
	compare_signature(request, token)

	sentry_sdk.get_isolation_scope().set_tag("overwatch_rpc_auth", True)

	return (AnonymousUser(), token)


	def _can_use_prevent_ai_features(org: Organization) -> bool:
	"""Check if organization has opted in to Prevent AI features."""
	hide_ai_features = org.get_option("sentry:hide_ai_features", HIDE_AI_FEATURES_DEFAULT)
	pr_review_test_generation_enabled = bool(


		__relocation_scope__ = RelocationScope.Excluded

		organization_id = BoundedBigIntegerField(db_index=True)

Uh oh!

ref: Create a prevent config model, no longer use organization options #101831

Are you sure you want to change the base?

ref: Create a prevent config model, no longer use organization options #101831

Uh oh!

Conversation

JerrySentry commented Oct 20, 2025

Uh oh!

codecov bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

suejung-sentry left a comment

Choose a reason for hiding this comment

Uh oh!

JerrySentry commented Oct 22, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

seer-by-sentry bot Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

JerrySentry Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

seer-by-sentry bot Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

JerrySentry Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Bug: AI Config Endpoint Bypasses New Model

Uh oh!

wedamija left a comment

Choose a reason for hiding this comment

Uh oh!

wedamija Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

wedamija Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

GabeVillalobos Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

wedamija Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

wedamija Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

JerrySentry commented Oct 23, 2025

Uh oh!

wedamija commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JerrySentry commented Oct 24, 2025

Uh oh!

getsantry bot commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Oct 20, 2025 •

edited

Loading

github-actions bot commented Oct 20, 2025 •

edited

Loading

wedamija commented Oct 23, 2025 •

edited

Loading