Skip to content

kops scalability tests doesnt have a way to configure scheduler QPS #17751

@alaypatel07

Description

@alaypatel07

/kind bug

1. What kops version are you running? The command kops version, will display
this information.

master branch

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

master branch

3. What cloud provider are you using?
GCE

4. What commands did you run? What is the simplest way to reproduce this issue?

/tests/e2e/scenarios/scalability/run-test.sh

5. What happened after the commands executed?
This PR creates an env variable to allow for qps configuration for kube-scheduler #17742. However, it is not respected.

I1110 03:09:34.558041      12 configfile.go:94] "Using component config" config=<
	apiVersion: kubescheduler.config.k8s.io/v1
	clientConnection:
	  acceptContentTypes: ""
	  burst: 100
	  contentType: application/vnd.kubernetes.protobuf
	  kubeconfig: /var/lib/kube-scheduler/kubeconfig
	  qps: 50

The component config that is uses a lower qps burst value then the flags:

I1110 03:09:34.550578      12 flags.go:64] FLAG: --kube-api-burst="500"
I1110 03:09:34.550580      12 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
I1110 03:09:34.550582      12 flags.go:64] FLAG: --kube-api-qps="-1"

6. What did you expect to happen?
I expected the configuration from run-test.sh to change the component config so that scheduler is configured with right qps for the test.

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: "2025-11-10T03:05:58Z"
  name: scalability.k8s.local
spec:
  api:
    loadBalancer:
      type: Public
  authorization:
    rbac: {}
  channel: stable
  cloudConfig:
    gceServiceAccount: default
  cloudControllerManager:
    concurrentNodeSyncs: 10
  cloudLabels:
    group: sig-cluster-lifecycle
    subproject: kops
  cloudProvider: gce
  configBase: gs://k8s-infra-e2e-scale-5k-project-state-adfe/scalability.k8s.local
  containerd:
    configAdditions:
      plugins."io.containerd.grpc.v1.cri".containerd.runtimes.test-handler.runtime_type: io.containerd.runc.v2
  etcdClusters:
  - cpuRequest: 200m
    etcdMembers:
    - instanceGroup: control-plane-us-east1-b
      name: b
      volumeIops: 10000
      volumeSize: 120
      volumeThroughput: 1000
      volumeType: hyperdisk-balanced
    manager:
      backupRetentionDays: 90
      env:
      - name: ETCD_QUOTA_BACKEND_BYTES
        value: "8589934592"
      - name: ETCD_ENABLE_PPROF
        value: "true"
      listenMetricsURLs:
      - http://localhost:2382
    memoryRequest: 100Mi
    name: main
  - cpuRequest: 100m
    etcdMembers:
    - instanceGroup: control-plane-us-east1-b
      name: b
      volumeIops: 10000
      volumeSize: 120
      volumeThroughput: 1000
      volumeType: hyperdisk-balanced
    manager:
      backupRetentionDays: 90
      env:
      - name: ETCD_QUOTA_BACKEND_BYTES
        value: "8589934592"
      - name: ETCD_ENABLE_PPROF
        value: "true"
    memoryRequest: 100Mi
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
  kubeAPIServer:
    anonymousAuth: true
    enableContentionProfiling: true
    enableProfiling: true
    logLevel: 2
    maxRequestsInflight: 4000
  kubeControllerManager:
    enableContentionProfiling: true
    enableProfiling: true
    endpointSliceUpdatesBatchPeriod: 500ms
    endpointUpdatesBatchPeriod: 500ms
    kubeAPIBurst: 500
    kubeAPIQPS: "500"
  kubeProxy:
    metricsBindAddress: 0.0.0.0:10249
    proxyMode: nftables
  kubeScheduler:
    authorizationAlwaysAllowPaths:
    - /healthz
    - /livez
    - /readyz
    - /metrics
    enableContentionProfiling: true
    enableProfiling: true
    kubeAPIBurst: 500
    kubeAPIQPS: "-1"
  kubelet:
    anonymousAuth: false
    maxPods: 96
  kubernetesApiAccess:
  - 0.0.0.0/0
  kubernetesVersion: https://storage.googleapis.com/k8s-release-dev/ci/v1.35.0-alpha.3.258+183892b2c96aa9
  networking:
    gce: {}
  nodePortAccess:
  - 0.0.0.0/0
  nonMasqueradeCIDR: 10.64.0.0/10
  podCIDR: 10.64.0.0/10
  project: k8s-infra-e2e-scale-5k-project
  serviceClusterIPRange: 10.130.0.0/15
  sshAccess:
  - 0.0.0.0/0
  subnets:
  - cidr: 10.128.0.0/15
    name: us-east1
    region: us-east1
    type: Private
  topology:
    dns:
      type: None

https://storage.googleapis.com/kubernetes-ci-logs/logs/ci-kubernetes-e2e-kops-gce-5000-node-dra-with-workload-ipalias-using-cl2/1987716860530921472/artifacts/cluster.yaml

8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.

9. Anything else do we need to know?

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions