Loki Ingester: Working Set Memory vs Resident Set Memory

[a7i]

We are noticing a big different in Memory WSS vs RSS. This presents a challenge for HPA scaling on Memory, given that metrics-server only scrapes container_memory_working_set_bytes. Is there some configuration or flag that we need to set to reclaim memory more aggressively?

Ingester config:

    ingester:
      autoforget_unhealthy: true
      chunk_block_size: 10490000
      chunk_encoding: zstd
      chunk_target_size: 0
      max_chunk_age: 1h
      wal:
        enabled: true
        replay_memory_ceiling: 24GB

[Nachtfalkeaw]

Did you try with Environment variables e.g.
GOMEMLIMIT=xGB
GOGC=50

[a7i]

Thanks @Nachtfalkeaw - will look into that. FWIW, we don't see that with any other Loki components, but only ingestor.