Skip to content

The Browse data menu does not show database's schema and tables when use aws glue hive metastore #134

@ahululu

Description

@ahululu

Metastore: AWS Glue

trino helm settings:

image:
  tag: "450"
server:
  workers: 1
  exchangeManager:
    baseDir: "s3://xxxx"
  autoscaling:
    enabled: true
    maxReplicas: 20
    targetCPUUtilizationPercentage: 70
  config:
    query:
      maxMemory: "4GB"
service:
  type: ClusterIP
  port: 8080
coordinator:
  jvm:
    maxHeapSize: "4G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "2GB"
worker:
  jvm:
    maxHeapSize: "12G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "8GB"
  resources:
    requests:
      cpu: 4000m
      memory: 12Gi
additionalConfigProperties:
  - "retry-policy=TASK"
  - "exchange.compression-enabled=true"
  - "query.low-memory-killer.delay=0s"
  - "query.remote-task.max-error-duration=1m"
  - "query.hash-partition-count=50"
additionalExchangeManagerProperties:
  - "exchange.s3.region=xxxxx"
  - "exchange.s3.iam-role=xxxxxx"
additionalCatalogs:
  hive: |-
    connector.name=hive
    hive.metastore=glue
    hive.metastore.glue.region=xxxxx
    hive.metastore.glue.endpoint-url=https://glue.xxxxx.amazonaws.com
    hive.metastore.glue.max-connections=30
    hive.metastore.glue.max-error-retries=3
    hive.metastore.glue.catalogid=1111111111
    hive.metastore.glue.iam-role=xxxxx
serviceAccount:
  create: true
  name: trino-sa
accessControl:
  type: configmap
  refreshPeriod: 60s
  configFile: "rules.json"
  rules:
    rules.json: |-
      {
        "catalogs": [
          {
            "user": "xxxxx",
            "catalog": ".*",
            "allow": "all"
          }
        ],
        "schemas": [
          {
            "user": "xxxxx",
            "schema": ".*",
            "owner": true
          }

      ],
        "tables": [
          {
            "user": "xxxx",
            "privileges": ["SELECT", "OWNERSHIP"]
          }
        ]
      }
ingress:
  enabled: true
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: xxx
    alb.ingress.kubernetes.io/group.name: xxx
    alb.ingress.kubernetes.io/healthcheck-path: /
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/subnets: sxxx
    alb.ingress.kubernetes.io/success-codes: xx
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
  hosts:
  - paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: trino
            port:
              number: 8080
    host: xx.xx.xx

Metabase database settings and operation:

1. Database type: Starburst
2. Host: xxxx
3. Port: 443
4. Catalog: hive
5. Schema: 
6. Username: xxxxx
7. Use a secure connection (SSL): enable
8. Rerun queries for simple explorations: enable

click button:  'Sync database schema now' and 'Re-scan field values now' 
Then go back to the metabase home page, Browse data select the corresponding database, there is no schema display, but you can use 'Sql query' to query the data

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions