Feat: change ingester data insertion policy

[MarceloRobert]

Description

Changes the insertion policy of the ingester from "never update" to "only update null fields". This follows the policy adopted by kcidb, it always prioritizes the incoming data in the following logic (considering existing / incoming):

• null + new data = new data
• existing data + new data = existing data
• existing data + null = existing data

Changes

• Added a new command to generate the SQL insertion query for any model
  • This command generates a python file with a dictionary relating a table with its insertion query and which fields should be updated
• Used SQL queries on consume_buffer instead of Django's bulk_create
• Updated unit tests

How to test

Start the ingester and insert some data to it referencing the same object, check if the policy is respected.
Example realistic data for testing can be found here, but other tests are encouraged.

Future changes

2. Adding automated integration tests so that we can guarantee that this policy is kept regardless of changes in the ingester is also useful.

Closes #1552

[Copilot]

Pull request overview

This PR changes the ingester's data insertion policy from "never update" to "only update null fields", controlled by a new PRIO_DB environment variable. The implementation replaces Django's ORM bulk_create with raw SQL queries using PostgreSQL's ON CONFLICT clause with COALESCE operations to selectively update fields based on the priority policy.

Key changes:

• Added dynamic SQL query generation function _generate_model_insert_query that creates upsert queries with field-level update control
• Replaced bulk_create with raw SQL execution using cursor.executemany() in consume_buffer
• Introduced PRIO_DB environment variable to toggle between database-priority and data-priority update modes

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.

File	Description
backend/kernelCI_app/constants/ingester.py	Adds PRIO_DB constant with documentation (contains typo in env var name)
backend/kernelCI_app/models.py	Adds module docstring noting explicit id column requirement for ingester
backend/kernelCI_app/management/commands/helpers/kcidbng_ingester.py	Implements dynamic SQL query generation and replaces ORM with raw SQL for data insertion
backend/kernelCI_app/tests/unitTests/commands/monitorSubmissions/kcidbng_ingester_test.py	Updates test to mock database connections instead of Django ORM methods

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tales-aparecida]

kind of related: https://forum.djangoproject.com/t/speeding-up-postgres-bulk-create-by-using-unnest/36508

I wonder how much time we are taking on the make_*_instance methods

[gustavobtflores]

as id is the conflict target we could remove it from here (and the other tables too) if it is easy, checkouts.id and EXCLUDED.id will be the same

[gustavobtflores]

LGTM, just a comment about the id's in the insert_queries.py, pre-approved