-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
Add, delete, and insert times on incremental DBT logs, currently showing:
INFO - 17:24:45 Running with dbt=1.10.13
INFO - 17:24:45 Registered adapter: redshift=1.9.5
INFO - 17:24:46 Found 7 models, 17 sources, 524 macros
INFO - 17:24:46
INFO - 17:24:46 Concurrency: 4 threads (target='dev')
INFO - 17:24:46
INFO - 17:24:47 1 of 1 START sql table model collections_stage.sc_pmt_cb_information ........... [RUN]
INFO - 17:27:32 1 of 1 OK created sql table model collections_stage.sc_pmt_cb_information ...... [SUCCESS in 165.19s]
INFO - 17:27:32
INFO - 17:27:32 Finished running 1 table model in 0 hours 2 minutes and 46.52 seconds (166.52s).
INFO - 17:27:32
INFO - 17:27:32 Completed successfully
INFO - 17:27:32
INFO - 17:27:32 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 NO-OP=0 TOTAL=1
Describe alternatives you've considered
proposal:
ADDING DELETING and INSERTING on logs, like this:
INFO - 17:24:45 Running with dbt=1.10.13
INFO - 17:24:45 Registered adapter: redshift=1.9.5
INFO - 17:24:46 Found 7 models, 17 sources, 524 macros
INFO - 17:24:46
INFO - 17:24:46 Concurrency: 4 threads (target='dev')
INFO - 17:24:46
INFO - 17:24:47 1 of 1 START sql table model collections_stage.sc_pmt_cb_information ........... [RUN]
INFO - 17:24:47 1 of 1 START INCREMENTAL delete table model collections_stage.sc_pmt_cb_information ........... [RUN]
INFO - 17:27:32 1 of 1 INCREMENTAL delete OK on table model collections_stage.sc_pmt_cb_information ...... [SUCCESS in 165.19s]
INFO - 17:24:47 1 of 1 START INSERT table model collections_stage.sc_pmt_cb_information ........... [RUN]
INFO - 17:27:32 1 of 1 INSERT OK on table model collections_stage.sc_pmt_cb_information ...... [SUCCESS in 165.19s]
INFO - 17:27:32 1 of 1 MODEL OK created sql table model collections_stage.sc_pmt_cb_information ...... [SUCCESS in 165.19s]
INFO - 17:27:32
INFO - 17:27:32 Finished running 1 table model in 0 hours 2 minutes and 46.52 seconds (166.52s).
INFO - 17:27:32
INFO - 17:27:32 Completed successfully
INFO - 17:27:32
INFO - 17:27:32 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 NO-OP=0 TOTAL=1
Who will this benefit?
It will help debug long-running DBTs, on databases that are slow to delete records and/or identify ways to improve the DBT, looking only at the main logs.
Having a ton of DBTs makes it hard to go through detailed logs.
Are you interested in contributing this feature?
No response
Anything else?
No response