feat(tests): generate conformance report on demand (#2555)

Author: programmer04

.github/ISSUE_TEMPLATE/---release.md

@@ -33,10 +33,15 @@ If the troubleshooting section does not contain the answer to the problem you en
     This will add the necessary skaffolding so that the reference is rendered correctly on docs.konghq.com.
 
     Example:
-    ```
+
+    ```sh
     ${KUBERNETES_CONFIGURATION_REPO}/scripts/apidocs-gen/post-process-for-konghq.sh ${KUBERNETES_CONFIGURATION_REPO}/docs/gateway-operator-api-reference.md ${KONGHQ_DOCS_REPO}/app/_src/gateway-operator/reference/custom-resources/1.2.x.md
     ```
 
+## Conformance tests report
+
+Trigger for released version CI workflow [Generate Kubernetes Gateway API conformance tests report](https://github.com/Kong/kong-operator/actions/workflows/conformance_tests_report.yaml), verify artifacts and submit them via pull request to [Kubernetes Gateway API Conformance Reports](https://github.com/kubernetes-sigs/gateway-api/tree/main/conformance/reports).
+
 **Only for major and minor releases**:
 
 - [ ] After the release tag is created, bump the `fromVersion` in the `upgrade from one before latest to latest minor` [upgrade E2E test][helm_upgrade_test] to the one before the latest minor release.

.github/workflows/conformance_tests_report.yaml

@@ -0,0 +1,64 @@
+name: Generate Kubernetes Gateway API conformance tests report
+run-name: "Generate Kubernetes Gateway API conformance tests report ${{ format('ref:{0}', github.event.inputs.tag) }}"
+
+on:
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: The version of code to checkout (e.g. v1.2.3 or commit hash)
+        required: false
+        default: main
+
+permissions:
+  contents: read
+
+jobs:
+  conformance-tests:
+    timeout-minutes: ${{ fromJSON(vars.GHA_DEFAULT_TIMEOUT) }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - router-flavor: traditional_compatible
+          - router-flavor: expressions
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.13.1
+        with:
+          egress-policy: audit
+
+      - name: Checkout repository
+        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+        with:
+          fetch-depth: 1
+          fetch-tags: true
+          ref: ${{ github.event.inputs.tag }}
+
+      - uses: jdx/mise-action@be3be2260bc02bc3fbf94c5e2fed8b7964baf074 # v3.4.0
+        with:
+          install: false
+
+      - name: Setup Golang
+        uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
+        with:
+          go-version-file: go.mod
+
+      - name: Run conformance tests
+        env:
+          TEST_KONG_ROUTER_FLAVOR: ${{ matrix.router-flavor }}
+          # Add GITHUB_TOKEN env so that mise plugins can can access GitHub's API
+          # authenticated and thus not get rate-limited (which causes failures).
+          # Source ref: https://github.com/pirackr/asdf-telepresence/blob/c9eeab4d28b9851bb904c4c67239a118c8dd384d/bin/list-all#L7-L9
+          GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: make test.conformance
+
+      # Generated report should be submitted to
+      # https://github.com/kubernetes-sigs/gateway-api/tree/main/conformance/reports
+      # in future automate creating PR (add to release workflow).
+      # TODO: https://github.com/Kong/kong-operator/issues/131
+      - name: Collect conformance report
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: conformance-report-${{ matrix.router-flavor }}
+          path: experimental-*-report.yaml