[opentelemetry-collector]: add kubernetesObjects presets

[KyriosGN0]

It would be beneficial to have a preset that will enable the k8objects recevier configured with default k8s apis, it would allow to have better DRY as different clusters can have different CRD which users might want to stream to log backend

[TylerHelmuth]

k8sobjects is one of the receivers that needs either a single replica deployment or leader election so it is a good candidate for a opinionated preset. What isn't clear is what objects the preset would watch/pull

[KyriosGN0]

I Think that watching the a set of default resources in core api groups (deployment, configmap, secret, cronjob, hpa, ingress) can help with state tracking of k8s cluster (this is my use case for it)

[cyrille-leclerc]

FYI the otel-kube-stack chart supports a preset kubernetesEvents that relies on the k8sobjects receiver, instead of relying on the k8s_events receiver for reasons I'm not clear on, maybe because the K8s_events receiver is still in alpha status.

Here are the details of the otel-kube-stack chart presets/kubernetesEvents, it implements the K8s leader election that @TylerHelmuth mentioned.

OTel Kube Stack values.yaml config:

opentelemetry-helm-charts/charts/opentelemetry-kube-stack/values.yaml

Lines 455 to 462 in 50d87c7

	kubernetesEvents:
	enabled: false
	# Configures the Kubernetes Cluster Receiver to collect cluster-level metrics.
	# Adds the k8s_cluster receiver to the metrics pipeline
	# and adds the necessary rules to ClusteRole.
	# Best used with mode = deployment or statefulset.
	# MUST be used by a collector with a single replica.
	# See https://opentelemetry.io/docs/kubernetes/collector/components/#kubernetes-cluster-receiver for details on the receiver.

OTel Kube Stack - Generated OTel Collector config

When used on a daemon collector , the OTel Kube Stack helm chart generates the following config. We can see it narrows down the k8s_objects scope to event objects, not the objects listed by @KyriosGN0 above.

receivers:
  k8sobjects:
    k8s_leader_elector: k8s_leader_elector/k8s_objects
    objects:
      - exclude_watch_type:
          - DELETED
        group: events.k8s.io
        mode: watch
        name: events
  # ...
extensions:
  k8s_leader_elector/k8s_objects:
    auth_type: serviceAccount
    lease_name: k8s.objects.receiver.opentelemetry.io
    lease_namespace: otel-demo
  # ...

OTel Kube Stack - Source code

opentelemetry-helm-charts/charts/opentelemetry-kube-stack/templates/_config.tpl

Lines 334 to 360 in 50d87c7

	{{- define "opentelemetry-kube-stack.collector.applyKubernetesEventsConfig" -}}
	{{- $electorName := "k8s_objects" }}
	{{- $config := mustMergeOverwrite (include "opentelemetry-kube-stack.collector.kubernetesEventsConfig" (dict "collector" .collector "namespace" .namespace "electorName" $electorName) | fromYaml) .collector.config }}
	{{- if and (dig "service" "pipelines" "logs" false $config) (not (has "k8sobjects" (dig "service" "pipelines" "logs" "receivers" list $config))) }}
	{{- $_ := set $config.service.pipelines.logs "receivers" (append ($config.service.pipelines.logs.receivers | default list) "k8sobjects" | uniq) }}
	{{- $_ := set $config.service "extensions" (append ($config.service.extensions | default list) (printf "k8s_leader_elector/%s" $electorName) | uniq) }}
	{{- end }}
	{{- $config | toYaml }}
	{{- end }}
	{{- define "opentelemetry-kube-stack.collector.kubernetesEventsConfig" -}}
	{{- $disableLeaderElection := .collector.presets.kubernetesEvents.disableLeaderElection}}
	{{- if not $disableLeaderElection}}
	{{- include "opentelemetry-kube-stack.collector.leaderElectionConfig" (dict "name" .electorName "leaseName" "k8s.objects.receiver.opentelemetry.io" "leaseNamespace" .namespace)}}
	{{- end}}
	receivers:
	k8sobjects:
	{{- if not $disableLeaderElection}}
	k8s_leader_elector: k8s_leader_elector/{{ .electorName }}
	{{- end}}
	objects:
	- name: events
	mode: "watch"
	group: "events.k8s.io"
	exclude_watch_type:
	- "DELETED"
	{{- end }}

[TylerHelmuth]

I'm ok with a preset that scopes specifically to events like the kube-stack chart.

[TylerHelmuth]

actually the collector chart already has a kubernetesEvents preset

[KyriosGN0]

@TylerHelmuth @cyrille-leclerc, the events preset is what inspired me to think about a different one for core k8s objects, we started to use the k8sObjects receiver with core api and some CRD, we ran into a DRY problem since we have some CRD we want to dump to our logging backend but it doesn't exist on every cluster, and this can mean we need create special config and then pass it to each cluster with the CRD with manual tracking/docs.
a preset can help since it would reduce boilerplate for the core APIs