Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 20, 2025

Go 1.25 changed default behavior when GOPATH is unset—it now defaults to GOROOT, causing build failures in the kube-cross Docker container with permission errors writing to /usr/local/go/pkg/mod.

Changes

  • Makefile: Explicitly set GOPATH=/go and GOMODCACHE=/app/.modcache in DOCKER function
  • .gitignore: Add .modcache to ignore module cache directory

Context

The Makefile's DOCKER function mounts the host's GOPATH to /go but didn't set the environment variable. In Go 1.25, this causes:

warning: both GOPATH and GOROOT are the same directory (/usr/local/go)
go: writing go.mod cache: mkdir /usr/local/go/pkg/mod/cache/download: permission denied

Setting GOMODCACHE to a writable location in /app (project directory) ensures module downloads work regardless of GOPATH mount permissions.

Fixes builds with MINIKUBE_BUILD_IN_DOCKER=y on Jenkins and Functional_Verified workflow.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • registry.k8s.io
    • Triggering command: /usr/bin/docker docker run --rm registry.k8s.io/build-image/kube-cross:v1.34.0-go1.24.6-bullseye.0 /bin/bash -c echo "GOROOT: $GOROOT"; echo "GOPATH: $GOPATH"; go env GOROOT; go env GOPATH (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>bumping golang 1.25 failure when building using "kube-cross image"</issue_title>
<issue_description>When trying to bump the golang from 1.24 to 1.25 I the build fails in jenkins when it is built in Docker using the Kube Cross image (this is after I removed hyperkit which is can not be built with 1.25) but now got a new blocker

# NOTE: "latest" as of 2021-02-06. kube-cross images aren't updated as often as Kubernetes
# https://github.com/kubernetes/kubernetes/blob/master/build/build-image/cross/VERSION

# BUILD_IMAGE 	?= registry.k8s.io/build-image/kube-cross:$(GO_K8S_VERSION_PREFIX)-go$(GO_VERSION)-bullseye.0

I noticed build fails in Jenkins (which runs in docker) "MINIKUBE_BUILD_IN_DOCKER" and also Functiona_Verified job also fails (builds minikube in docker) but "Functional" workflow passes (builds it on Ubuntu Baremetal

Functional_Verified Workflow Build (builds in Docker

MINIKUBE_BUILD_IN_DOCKER=y make e2e-linux-arm64

MINIKUBE_BUILD_IN_DOCKER=y make e2e-linux-arm64

Functional Normal build passee

make e2e-linux-amd64 e2e-darwin-amd64

We need to find out why new KubeCorss Image fails and how to fix it, because on Jenkins we should not rely on building on host

Logs From Jenkins build (trimmed)

12:35:43 Unable to find image 'registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0' locally
12:35:43 v1.35.0-go1.25.3-bullseye.0: Pulling from build-image/kube-cross
12:35:43 3adbbc83e99d: Waiting
12:37:12 a949bcef8c2a: Pull complete
12:37:12 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:12 Status: Downloaded newer image for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Status: Downloaded newer image for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:12 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:12 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:13 Digest: sha256:2d42755343ada0f56e9d381916edf5d2dbd00050643d4f558b9bcd891d3bdc81
12:37:13 Status: Image is up to date for registry.k8s.io/build-image/kube-cross:v1.35.0-go1.25.3-bullseye.0
12:37:13 warning: both GOPATH and GOROOT are the same directory (/usr/local/go); see https://go.dev/wiki/InstallTroubleshooting
12:37:13 warning: both GOPATH and GOROOT are the same directory (/usr/local/go); see https://go.dev/wiki/InstallTroubleshooting
12:37:13 warning: both GOPATH and GOROOT are the same directory (/usr/local/go); see https://go.dev/wiki/InstallTroubleshooting
# Get Repeated hunddreds of times...

12:37:13 GOOS="linux" GOARCH="s390x"  \
12:37:13 go build -tags "libvirt_dlopen" -ldflags="-X k8s.io/minikube/pkg/version.version=v1.37.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.37.0-1763503576-21924 -X k8s.io/minikube/pkg/version.gitCommitID="2d19d546682a9aac70700f5e2d85cb8e7ae0d67a" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v5" -a -o out/minikube-linux-s390x k8s.io/minikube/cmd/minikube

12:37:13 GOOS="windows" GOARCH="amd64"  \
12:37:13 go build -tags "libvirt_dlopen" -ldflags="-X k8s.io/minikube/pkg/version.version=v1.37.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.37.0-1763503576-21924 -X k8s.io/minikube/pkg/version.gitCommitID="2d19d546682a9aac70700f5e2d85cb8e7ae0d67a" -X k8s.io/minikube/pkg/version.storageProvisionerVersion=v5" -a -o out/minikube-windows-amd64 k8s.io/minikube/cmd/minikube
12:37:13 warning: both GOPATH and GOROOT are the same directory (/usr/local/go); see https://go.dev/wiki/InstallTroubleshooting
# ...

12:37:13 go: downloading gopkg.in/yaml.v2 v2.4.0
12:37:13 go: downloading golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c
12:37:13 warning: both GOPATH and GOROOT are the same directory (/usr/local/go); see https://go.dev/wiki/InstallTroubleshooting
12:37:13 GOOS="linux" GOARCH="386"  \
12:37:13 go build -tags "libvirt_dlopen" -ldflags="-X k8...

</details>

- Fixes kubernetes/minikube#21942

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 20, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @copilot. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Nov 20, 2025
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

Copilot AI changed the title [WIP] Fix golang 1.25 build failure in kube-cross image Fix Go 1.25 build failure in kube-cross Docker by setting GOPATH and GOMODCACHE Nov 20, 2025
Copilot finished work on behalf of medyagh November 20, 2025 21:39
Copilot AI requested a review from medyagh November 20, 2025 21:39
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 21, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Copilot, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@medyagh
Copy link
Member

medyagh commented Nov 21, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 21, 2025
@medyagh medyagh changed the title Fix Go 1.25 build failure in kube-cross Docker by setting GOPATH and GOMODCACHE CI: Fix Go 1.25 build failure in kube-cross Docker by setting GOPATH and GOMODCACHE Nov 21, 2025
@medyagh medyagh marked this pull request as ready for review November 21, 2025 23:05
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 21, 2025
@medyagh medyagh merged commit d7efd69 into master Nov 21, 2025
33 of 53 checks passed
@medyagh medyagh deleted the copilot/fix-golang-1-25-build branch November 21, 2025 23:05
@k8s-ci-robot
Copy link
Contributor

@Copilot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
integration-kvm-containerd-linux-x86-64 f8e5c88 link true /test integration-kvm-containerd-linux-x86-64
integration-kvm-crio-linux-x86-64 f8e5c88 link true /test integration-kvm-crio-linux-x86-64
integration-docker-containerd-linux-x86-64 f8e5c88 link true /test integration-docker-containerd-linux-x86-64
integration-docker-crio-linux-x86-64 f8e5c88 link true /test integration-docker-crio-linux-x86-64
integration-none-docker-linux-x86-64 f8e5c88 link true /test integration-none-docker-linux-x86-64
integration-docker-docker-linux-x86-64 f8e5c88 link true /test integration-docker-docker-linux-x86-64

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants