Skip to content

[v1alpha1] Error Updating BucketClaim Status #173

@kaiwalyajoshi

Description

@kaiwalyajoshi

Bug Report

We're using the COSI Controller Manager with the v1alpha1 API from release-0.2 branch.

We're setting the following errors:

E1106 04:33:08.574841       1 bucketclaim.go:205] "Failed to update status of BucketClaim" err="Operation cannot be fulfilled on bucketclaims.objectstorage.k8s.io \"nkp-insights-bucketclaim\": the object has been modified; please apply your changes to the latest version and try again" name=""
E1106 04:33:08.574907       1 bucketclaim.go:56] "name" err="Operation cannot be fulfilled on bucketclaims.objectstorage.k8s.io \"nkp-insights-bucketclaim\": the object has been modified; please apply your changes to the latest version and try again" nkp-insights-bucketclaim="ns" kommander="err" Operation cannot be fulfilled on bucketclaims.objectstorage.k8s.io "nkp-insights-bucketclaim": the object has been modified; please apply your changes to the latest version and try again="(MISSING)"
I1106 04:33:10.063861       1 reflector.go:389] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:243: forcing resync

We suspect this is causing a BucketClaim object not having its not having its status.bucketName field set.

And this causes the ceph-cosi-driver to fail with errors like the following:

I1104 21:28:16.453522       1 bucketaccess_controller.go:83] "Add BucketAccess" name="nkp-insights-bucketaccess" BucketAccessClassName="cosi-ceph-nkp"
E1104 21:28:16.475926       1 bucketaccess_controller.go:116] "Invalid arguments" err="BucketName cannot be empty or BucketNotReady in bucketClaim" bucketClaim="nkp-insights-bucketclaim" bucketAccess="nkp-insights-bucketaccess"
I1104 21:28:16.495742       1 bucket_controller.go:69] "Add Bucket" name="cosi-ceph-nkpe256e982-ae0a-43e5-8fb2-7436532c3ec2"

What happened:

What you expected to happen:

How to reproduce this bug (as minimally and precisely as possible):

Anything else relevant for this bug report?:

A proposed fix would be to use RetryOnConflict to wrap wherever we're making updates to the Status subresource, like here for instance.

Resources and logs to submit:

Copy all relevant COSI resources here in yaml format:

# BucketClass
# BucketAccessClass
# BucketClaim
# Bucket
# BucketAccess
# Copy COSI controller pod logs here
# Copy COSI sidecar logs here for the relevant driver

Environment:

  • Kubernetes version (use kubectl version), please list client and server:
  • Sidecar version (provide the release tag or commit hash):
  • Provisoner name and version (provide the release tag or commit hash):
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Network plugin and version (if this is a network-related bug):
  • Others:

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions