Skip to content

Result is wrong when iceberg table with partition transform day(timestamp) #26604

@shenh062326

Description

@shenh062326

Your Environment

  • Presto version used: master
  • Storage (HDFS/S3/GCS..): hdfs
  • Data source and connector used: iceberg
  • Deployment (Cloud or On-prem): lcocal debug
  • Pastebin link to the complete debug logs:

Expected Behavior

Current Behavior

presto:default> create table iceberg.default.iceberg_ts1( id bigint, category varchar, ts timestamp ) WITH ( location = 'hdfs://localhost:8020/warehouse/iceberg/db_name/iceberg_ts1', partitioning = ARRAY['bucket(id,8)','day(ts)','category']);
CREATE TABLE

Query 20251113_035359_00002_7fggk, FINISHED, 0 nodes
Splits: 0 total, 0 done (0.00%)
[Latency: client-side: 0:01, server-side: 0:01] [0 rows, 0B] [0 rows/s, 0B/s]

presto:default> insert into iceberg.default.iceberg_ts1 select 11, 'cc1', cast('2022-05-06 07:08:09' as timestamp);
INSERT: 1 row

Query 20251113_035402_00003_7fggk, FINISHED, 1 node
Splits: 35 total, 35 done (100.00%)
[Latency: client-side: 0:02, server-side: 0:02] [0 rows, 0B] [0 rows/s, 0B/s]

presto:default> select * from iceberg.default.iceberg_ts1;
 id | category |           ts            
----+----------+-------------------------
 11 | cc1      | 2022-05-06 07:08:09.000 
(1 row)

Query 20251113_035407_00004_7fggk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
[Latency: client-side: 220ms, server-side: 208ms] [1 rows, 514B] [4 rows/s, 2.41KB/s]

presto:default> select * from iceberg.default.iceberg_ts1 where ts=cast('2022-05-06 07:08:09' as timestamp);
 id | category | ts 
----+----------+----
(0 rows)

Query 20251113_035412_00005_7fggk, FINISHED, 1 node
Splits: 1 total, 1 done (100.00%)
[Latency: client-side: 107ms, server-side: 94ms] [0 rows, 0B] [0 rows/s, 0B/s]

My local time zone is Asia/Shanghai. 2022-05-06 07:08:09 should be 1651792089000, but
at IcebergPageSink, presto will adjustTimestampForPartitionTransform for timestamp. it will be adjust to 1651820889000.

Image

Possible Solution

Steps to Reproduce

Screenshots (if appropriate)

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    🆕 Unprioritized

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions