Configuring a local file system w/ Iceberg #27217

szisiu · 2025-11-05T10:55:34Z

szisiu
Nov 5, 2025

This issue specifically focuses on the confusion around file:// protocol and the undocumented requirement for shared/local file system access between Trino/Iceberg and HMS in local/test development setups.

Documentation URL: https://trino.io/docs/current/object-storage/file-system-local.html

Environment

Trino version: 478 w/ Iceberg connector (single-node development)
Connector: Iceberg with HMS 4.1.0 backed by MySQL
Deployment: Docker Compose

Issues

The documentation mentions that Trino includes support to access a local file system with a catalog using Iceberg connectors.

On the other hand at Iceberg connector page it is missing: https://trino.io/docs/current/connector/iceberg.html#file-system-access-configuration

local:// protocol is used in examples but doesn't clearly explain the relationship with file:// protocol. No explanation of when to use which protocol or how they relate to each other. How local.location relates to file://?

Using `file://` with `/`

Tested and it works with some constraints.
Trino and HMS must share the same file system view (dirty hacks): have the same host path mounted at identical container paths, share the same GUID/UID to avoid permission issues, have chmod o+rw permissions on the host directory

Catalog Configuration (catalog/iceberg_local.properties):

connector.name=iceberg
iceberg.catalog.type=hive_metastore
hive.metastore.uri=thrift://hms:9083

fs.native-local.enabled=true
local.location=/ # does not work for other paths

Docker Compose (simplified, showing shared volume approach):

services:
  setup:
    image: alpine:3.22
    command: >
      /bin/sh -c '
        mkdir -p /tmp/iceberg-data &&
        chmod o+rw /tmp/iceberg-data &&
        exit 0;
      '
    volumes:
      - /var/gat:/mnt:rw,z
    restart: no

  mysql:
    image: mysql:8.0

  hms:
    image: apache/hive:standalone-metastore-4.1.0
    volumes:
      - /tmp/iceberg-data:/iceberg-data:rw,z

  trino-crd:
    image: trino:478
    volumes:
      - /tmp/iceberg-data:/iceberg-data:rw,z

HMS Configuration (metastore-site.xml):

    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.cj.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://mysql:3306/hms</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>user</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>pass</value>
    </property>

Verification:

CREATE SCHEMA IF NOT EXISTS iceberg_local.testschema
      WITH ( "location" = 'file:/iceberg_data/testschema' );
-- or WITH ( "location" = 'file:///iceberg_data/testschema' );

CREATE TABLE iceberg_local.testschema.testtbl (
    c1,
    c2
)
WITH (
    format = 'PARQUET'
)
AS VALUES
    (2021, 'a'),
    (2022, 'b'),
    (2022, 'c');

select * from iceberg_local.testschema.testtbl;

drop schema iceberg_local.testschema cascade;

The file:/iceberg_data/testschema prefix path is stored in HMS DB and in Iceberg files.

Using `file://` with a local path other than `/`

It fails and two separate local path structures are managed by HMS and Trino.

local.location = /tmp: tell Trino's local filesystem module, "My root warehouse path is eg /tmp"

WITH ( location = 'file:/tmp/iceberg-data/testschema' ): tell the CREATE SCHEMA command, "Create this schema at the absolute path /tmp/iceberg-data/testschema."

The filesystem module is combining them. It's taking the "root" path and appending "absolute" path, leading to the nested structure:

(<local.location> + <location_from_sql>) /tmp + /tmp/iceberg-data/testschema = /tmp/tmp/iceberg-data/testschema

This is the path inside the container.

Additionally there is an empty iceberg-data/testschema directory likely created by the HMS logic before the filesystem module got involved and wrote the table data to the wrong, nested path.

Using `local://`

I tried using local:// with the environment deployed as above but without luck.

CREATE SCHEMA IF NOT EXISTS iceberg_local.testschema
      WITH ( "location" = 'local:/iceberg_data/testschema' );
-- or WITH ( "location" = 'local:///iceberg_data/testschema' );

client response log (dbeaver)

org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [16777216]: Query failed (#20251105_101625_00020_3ckj6): Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null
	at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:134)
	at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:675)
	at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:568)
	at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:189)
	at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleElement(SQLQueryJob.java:587)
	at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:449)
	at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:1061)
	at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4581)
	at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:128)
	at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:189)
	at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:126)
	at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5214)
	at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:119)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.sql.SQLException: Query failed (#20251105_101625_00020_3ckj6): Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null
	at io.trino.jdbc.ResultUtils.resultsException(ResultUtils.java:33)
	at io.trino.jdbc.AsyncResultIterator.lambda$new$1(AsyncResultIterator.java:94)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: io.trino.spi.TrinoException: Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.createDatabase(ThriftHiveMetastore.java:799)
	at io.trino.plugin.hive.metastore.thrift.BridgingHiveMetastore.createDatabase(BridgingHiveMetastore.java:165)
	at io.trino.metastore.tracing.TracingHiveMetastore.lambda$createDatabase$0(TracingHiveMetastore.java:188)
	at io.trino.metastore.tracing.Tracing.lambda$withTracing$0(Tracing.java:35)
	at io.trino.metastore.tracing.Tracing.withTracing(Tracing.java:43)
	at io.trino.metastore.tracing.Tracing.withTracing(Tracing.java:34)
	at io.trino.metastore.tracing.TracingHiveMetastore.createDatabase(TracingHiveMetastore.java:188)
	at io.trino.metastore.MeasuredHiveMetastore.lambda$createDatabase$0(MeasuredHiveMetastore.java:135)
	at io.trino.metastore.MeasuredHiveMetastore.lambda$wrap$0(MeasuredHiveMetastore.java:423)
	at io.trino.metastore.MeasuredHiveMetastore.wrap(MeasuredHiveMetastore.java:433)
	at io.trino.metastore.MeasuredHiveMetastore.wrap(MeasuredHiveMetastore.java:422)
	at io.trino.metastore.MeasuredHiveMetastore.createDatabase(MeasuredHiveMetastore.java:135)
	at io.trino.metastore.cache.CachingHiveMetastore.createDatabase(CachingHiveMetastore.java:585)
	at io.trino.plugin.iceberg.catalog.hms.TrinoHiveCatalog.createNamespace(TrinoHiveCatalog.java:243)
	at io.trino.plugin.iceberg.IcebergMetadata.createSchema(IcebergMetadata.java:1153)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.createSchema(ClassLoaderSafeConnectorMetadata.java:429)
	at io.trino.tracing.TracingConnectorMetadata.createSchema(TracingConnectorMetadata.java:351)
	at io.trino.metadata.MetadataManager.createSchema(MetadataManager.java:842)
	at io.trino.tracing.TracingMetadata.createSchema(TracingMetadata.java:409)
	at io.trino.execution.CreateSchemaTask.internalExecute(CreateSchemaTask.java:128)
	at io.trino.execution.CreateSchemaTask.execute(CreateSchemaTask.java:82)
	at io.trino.execution.CreateSchemaTask.execute(CreateSchemaTask.java:54)
	at io.trino.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:152)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:284)
	at io.trino.dispatcher.LocalDispatchQuery.startExecution(LocalDispatchQuery.java:150)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$waitForMinimumWorkers$1(LocalDispatchQuery.java:134)
	at io.airlift.concurrent.MoreFutures.lambda$addSuccessCallback$0(MoreFutures.java:570)
	at io.airlift.concurrent.MoreFutures$3.onSuccess(MoreFutures.java:545)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
	at io.trino.$gen.Trino_478____20251105_085852_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
	at java.base/java.lang.Thread.run(Thread.java:1474)
Caused by: io.trino.hive.thrift.metastore.MetaException: Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:45685)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:45652)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result.read(ThriftHiveMetastore.java:45585)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:93)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$Client.recvCreateDatabase(ThriftHiveMetastore.java:1104)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$Client.createDatabase(ThriftHiveMetastore.java:1091)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at io.trino.plugin.base.util.LoggingInvocationHandler.handleInvocation(LoggingInvocationHandler.java:60)
	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:86)
	at jdk.proxy8/jdk.proxy8.$Proxy370.createDatabase(Unknown Source)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient.createDatabase(ThriftHiveMetastoreClient.java:241)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.lambda$createDatabase$0(FailureAwareThriftMetastoreClient.java:107)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.runWithHandle(FailureAwareThriftMetastoreClient.java:484)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.createDatabase(FailureAwareThriftMetastoreClient.java:107)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.lambda$createDatabase$0(ThriftHiveMetastore.java:790)
	at io.trino.plugin.hive.metastore.thrift.ThriftMetastoreApiStats.lambda$wrap$0(ThriftMetastoreApiStats.java:41)
	at io.trino.plugin.hive.metastore.thrift.RetryDriver.run(RetryDriver.java:117)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.createDatabase(ThriftHiveMetastore.java:788)
	... 32 more

trino server log

INFO	dispatcher-query-25	io.trino.event.QueryMonitor	TIMELINE: Query 20251105_101625_00020_3ckj6 :: FAILED (HIVE_METASTORE_ERROR) :: elapsed 234ms :: planning 234ms :: waiting 0ms :: scheduling 0ms :: running 0ms :: finishing 0ms :: begin 2025-11-05T10:16:23.448Z :: end 2025-11-05T10:16:23.682Z
DEBUG	Query-20251105_101625_00020_3ckj6-381	io.trino.execution.QueryStateMachine	Query 20251105_101625_00020_3ckj6 failed
io.trino.spi.TrinoException: Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.createDatabase(ThriftHiveMetastore.java:799)
	at io.trino.plugin.hive.metastore.thrift.BridgingHiveMetastore.createDatabase(BridgingHiveMetastore.java:165)
	at io.trino.metastore.tracing.TracingHiveMetastore.lambda$createDatabase$0(TracingHiveMetastore.java:188)
	at io.trino.metastore.tracing.Tracing.lambda$withTracing$0(Tracing.java:35)
	at io.trino.metastore.tracing.Tracing.withTracing(Tracing.java:43)
	at io.trino.metastore.tracing.Tracing.withTracing(Tracing.java:34)
	at io.trino.metastore.tracing.TracingHiveMetastore.createDatabase(TracingHiveMetastore.java:188)
	at io.trino.metastore.MeasuredHiveMetastore.lambda$createDatabase$0(MeasuredHiveMetastore.java:135)
	at io.trino.metastore.MeasuredHiveMetastore.lambda$wrap$0(MeasuredHiveMetastore.java:423)
	at io.trino.metastore.MeasuredHiveMetastore.wrap(MeasuredHiveMetastore.java:433)
	at io.trino.metastore.MeasuredHiveMetastore.wrap(MeasuredHiveMetastore.java:422)
	at io.trino.metastore.MeasuredHiveMetastore.createDatabase(MeasuredHiveMetastore.java:135)
	at io.trino.metastore.cache.CachingHiveMetastore.createDatabase(CachingHiveMetastore.java:585)
	at io.trino.plugin.iceberg.catalog.hms.TrinoHiveCatalog.createNamespace(TrinoHiveCatalog.java:243)
	at io.trino.plugin.iceberg.IcebergMetadata.createSchema(IcebergMetadata.java:1153)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.createSchema(ClassLoaderSafeConnectorMetadata.java:429)
	at io.trino.tracing.TracingConnectorMetadata.createSchema(TracingConnectorMetadata.java:351)
	at io.trino.metadata.MetadataManager.createSchema(MetadataManager.java:842)
	at io.trino.tracing.TracingMetadata.createSchema(TracingMetadata.java:409)
	at io.trino.execution.CreateSchemaTask.internalExecute(CreateSchemaTask.java:128)
	at io.trino.execution.CreateSchemaTask.execute(CreateSchemaTask.java:82)
	at io.trino.execution.CreateSchemaTask.execute(CreateSchemaTask.java:54)
	at io.trino.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:152)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:284)
	at io.trino.dispatcher.LocalDispatchQuery.startExecution(LocalDispatchQuery.java:150)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$waitForMinimumWorkers$1(LocalDispatchQuery.java:134)
	at io.airlift.concurrent.MoreFutures.lambda$addSuccessCallback$0(MoreFutures.java:570)
	at io.airlift.concurrent.MoreFutures$3.onSuccess(MoreFutures.java:545)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
	at io.trino.$gen.Trino_478____20251105_085852_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
	at java.base/java.lang.Thread.run(Thread.java:1474)
Caused by: MetaException(message:Failed to create external path local:/iceberg_local/testschema for database testschema. This may result in access not being allowed if the StorageBasedAuthorizationProvider is enabled: null)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:45685)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result$create_database_resultStandardScheme.read(ThriftHiveMetastore.java:45652)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$create_database_result.read(ThriftHiveMetastore.java:45585)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:93)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$Client.recvCreateDatabase(ThriftHiveMetastore.java:1104)
	at io.trino.hive.thrift.metastore.ThriftHiveMetastore$Client.createDatabase(ThriftHiveMetastore.java:1091)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at io.trino.plugin.base.util.LoggingInvocationHandler.handleInvocation(LoggingInvocationHandler.java:60)
	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:86)
	at jdk.proxy8/jdk.proxy8.$Proxy370.createDatabase(Unknown Source)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient.createDatabase(ThriftHiveMetastoreClient.java:241)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.lambda$createDatabase$0(FailureAwareThriftMetastoreClient.java:107)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.runWithHandle(FailureAwareThriftMetastoreClient.java:484)
	at io.trino.plugin.hive.metastore.thrift.FailureAwareThriftMetastoreClient.createDatabase(FailureAwareThriftMetastoreClient.java:107)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.lambda$createDatabase$0(ThriftHiveMetastore.java:790)
	at io.trino.plugin.hive.metastore.thrift.ThriftMetastoreApiStats.lambda$wrap$0(ThriftMetastoreApiStats.java:41)
	at io.trino.plugin.hive.metastore.thrift.RetryDriver.run(RetryDriver.java:117)
	at io.trino.plugin.hive.metastore.thrift.ThriftHiveMetastore.createDatabase(ThriftHiveMetastore.java:788)
	... 32 more

The local:// protocol is specifically for shared local filesystems, no?

Please review if there is some bug and/or the documentation needs to be updated.

Also please explain and correct me if I'm wrong or heading in the wrong direction.

szisiu · 2025-11-05T10:56:32Z

szisiu
Nov 5, 2025
Author

Found some related tasks:

0 replies

szisiu · 2025-11-06T18:13:45Z

szisiu
Nov 6, 2025
Author

Today tested using iceberg-rest-fixture docker image instead of HMS with the same outcome.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Configuring a local file system w/ Iceberg #27217

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Configuring a local file system w/ Iceberg #27217

Uh oh!

Uh oh!

szisiu Nov 5, 2025

Environment

Issues

Using file:// with /

Using file:// with a local path other than /

Using local://

Replies: 2 comments

Uh oh!

szisiu Nov 5, 2025 Author

Uh oh!

szisiu Nov 6, 2025 Author

szisiu
Nov 5, 2025

Using `file://` with `/`

Using `file://` with a local path other than `/`

Using `local://`

szisiu
Nov 5, 2025
Author

szisiu
Nov 6, 2025
Author