Skip to content

InvalidColumnNameException["_sdc_deleted_at" conflicts with system column pattern] #53

New issue
New issue
Open
#57
Open
InvalidColumnNameException["_sdc_deleted_at" conflicts with system column pattern]#53
#57
@amotl

Description

@amotl
amotl
opened on Mar 29, 2025

Problem

Singer/Meltano uses column names prefixed with _ to signal privateness. CrateDB does the same, and blocks this namespace completely.

This is detrimental for wider adoption in this and other ecosystems, because it can not always be worked around easily, as those private columns are mostly at the core of the relevant framework adapter subsystem.

meltano-target-cratedb/target_cratedb/sinks.py

Lines 40 to 50 in e10e76f

record["__sdc_extracted_at"] = message.get("time_extracted")
record["__sdc_received_at"] = datetime.datetime.now(
tz=datetime.timezone.utc,
).isoformat()
record["__sdc_batched_at"] = (
context.get("batch_start_time", None) or datetime.datetime.now(tz=datetime.timezone.utc)
).isoformat()
record["__sdc_deleted_at"] = record.get("__sdc_deleted_at")
record["__sdc_sequence"] = int(round(time.time() * 1000))
record["__sdc_table_version"] = message.get("version")
record["__sdc_sync_started_at"] = self.sync_started_at

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions