Skip to content

Commit ab726d7

Browse files
author
Christopher Bradford
committed
Refactored OLM files to use bundle format
Added bundle files from Red Hat Updated packaging documentation
1 parent a757549 commit ab726d7

34 files changed

+23076
-12
lines changed

‎.github/workflows/release.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ jobs:
1818
AWS_ACCESS_KEY_ID: ${{ secrets.ECR_ID }}
1919
AWS_SECRET_ACCESS_KEY: ${{ secrets.ECR_SECRET }}
2020
run: aws ecr get-login-password --region us-east-1 | skopeo login -u AWS --password-stdin ${ECR_REPO}
21+
- name: Login Skopeo Red Hat
22+
env:
23+
REDHAT_REPO: ${{ secrets.REDHAT_REPO }}
24+
REDHAT_SECRET: ${{ secrets.REDHAT_SECRET }}
25+
run: docker login -u unused --password "$REDHAT_SECRET" $REDHAT_REPO
2126
- name: Publish to Dockerhub
2227
env:
2328
ECR_REPO: ${{ secrets.ECR_REPO }}

‎docs/developer/csv-description.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
## Apache Cassandra
2+
3+
The Apache Cassandra database is the right choice when you need scalability and
4+
high availability without compromising performance. Linear scalability and
5+
proven fault-tolerance on commodity hardware or cloud infrastructure make it the
6+
perfect platform for mission-critical data. Cassandra's support for replicating
7+
across multiple datacenters is best-in-class, providing lower latency for your
8+
users and the peace of mind of knowing that you can survive regional outages.
9+
10+
## DataStax Enterprise
11+
12+
The most advanced distribution of Apache Cassandra™ on the market, with the
13+
enterprise functionality needed for serious production systems and backed up and
14+
supported by the best distributed-experts in the world. It's one platform for
15+
all types of applications anywhere, any cloud, any model: key-value, graph,
16+
tabular, JSON.
17+
18+
DataStax Enterprise is a fully integrated and optimized database, with graph,
19+
analytics, and search included, all with a unified security model. Simply put,
20+
it's the only database capable of meeting today's demanding requirements
21+
22+
## Operator Details
23+
24+
`cass-operator` is designed as a modular operator for Apache Cassandra and
25+
derived distributions. Apache Cassandra is a distributed database consisting of
26+
multiple nodes working in concert to store data and process queries along a
27+
number of fault domains. `cass-operator` has the deployment of a Cassandra
28+
cluster around the logical domain of a datacenter with the `CassandraDatacenter`
29+
custom resource. Upon submission of one of these resources it handles
30+
provisioning the underlying stateful sets (analogous to C\* logical racks),
31+
services, and configuration. Additionally through monitoring pod state via
32+
Kubernetes callbacks it handles day to day operations such as restarting failed
33+
processes, scaling clusters up, and deploying configuration changes in a
34+
rolling, non-disruptive, fashion. This operator is designed to be `Namespace`
35+
scoped. A single Kubernetes cluster may be running multiple instances of this
36+
operator, in separate namespaces, to support a number of C\* clusters and
37+
environments. Configuration is simple with the usage of YAML based overrides in
38+
the Custom Resource paired with an `init` container. In C\* clusters ordering and
39+
timing of certain operations are important to keep the system evenly
40+
distributed. `cass-operator` takes advantage of a sidecar process within the
41+
main container to handle the orchestration of starting our main C* process.
42+
43+
## Pre-requisites
44+
45+
### WebHook Secret
46+
47+
This secret is intentionally left blank and will be filled in as the operator is
48+
brought is started and certificates are generated.
49+
50+
```yaml
51+
apiVersion: v1
52+
data:
53+
tls.crt: ""
54+
tls.key: ""
55+
kind: Secret
56+
metadata:
57+
name: cass-operator-webhook-config
58+
```
59+
60+
### WebHook Configuration
61+
62+
Note the namespace _must_ be updated in the configuration below.
63+
64+
```yaml
65+
apiVersion: admissionregistration.k8s.io/v1beta1
66+
kind: ValidatingWebhookConfiguration
67+
metadata:
68+
name: "cassandradatacenter-webhook-registration"
69+
webhooks:
70+
- name: "cassandradatacenter-webhook.cassandra.datastax.com"
71+
rules:
72+
- apiGroups: ["cassandra.datastax.com"]
73+
apiVersions: ["v1beta1"]
74+
operations: ["CREATE", "UPDATE", "DELETE"]
75+
resources: ["cassandradatacenters"]
76+
scope: "*"
77+
clientConfig:
78+
service:
79+
name: "cassandradatacenter-webhook-service"
80+
namespace: "cass-operator"
81+
path: /validate-cassandra-datastax-com-v1beta1-cassandradatacenter
82+
admissionReviewVersions: ["v1beta1"]
83+
failurePolicy: "Ignore"
84+
matchPolicy: "Equivalent"
85+
sideEffects: None
86+
timeoutSeconds: 10
87+
```
88+
89+
### WebHook Service
90+
91+
Note the namespace _must_ be updated in the configuration below.
92+
93+
```yaml
94+
apiVersion: v1
95+
kind: Service
96+
metadata:
97+
name: cassandradatacenter-webhook-service
98+
labels:
99+
name: cass-operator-webhook
100+
spec:
101+
ports:
102+
- port: 443
103+
targetPort: 8443
104+
selector:
105+
name: cass-operator
106+
```

‎docs/developer/olm/catalogsource.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: operators.coreos.com/v1alpha1
2+
kind: CatalogSource
3+
metadata:
4+
name: local
5+
namespace: olm
6+
spec:
7+
displayName: Local Catalog
8+
publisher: Developer
9+
sourceType: grpc
10+
image: bradfordcp/catalog-index:latest

‎docs/developer/olm/operatorgroup.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: operators.coreos.com/v1alpha2
2+
kind: OperatorGroup
3+
metadata:
4+
name: group
5+
namespace: test-operator
6+
spec:
7+
targetNamespaces:
8+
- test-operator

‎docs/developer/olm/prereqs.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
apiVersion: v1
3+
data:
4+
tls.crt: ""
5+
tls.key: ""
6+
kind: Secret
7+
metadata:
8+
name: cass-operator-webhook-config
9+
namespace: test-operator
10+
---
11+
apiVersion: admissionregistration.k8s.io/v1beta1
12+
kind: ValidatingWebhookConfiguration
13+
metadata:
14+
name: cassandradatacenter-webhook-registration
15+
webhooks:
16+
- admissionReviewVersions:
17+
- v1beta1
18+
clientConfig:
19+
service:
20+
name: cassandradatacenter-webhook-service
21+
namespace: test-operator
22+
path: /validate-cassandra-datastax-com-v1beta1-cassandradatacenter
23+
failurePolicy: Ignore
24+
matchPolicy: Equivalent
25+
name: cassandradatacenter-webhook.cassandra.datastax.com
26+
rules:
27+
- apiGroups:
28+
- cassandra.datastax.com
29+
apiVersions:
30+
- v1beta1
31+
operations:
32+
- CREATE
33+
- UPDATE
34+
resources:
35+
- cassandradatacenters
36+
scope: '*'
37+
sideEffects: None
38+
timeoutSeconds: 10
39+
---
40+
apiVersion: v1
41+
kind: Service
42+
metadata:
43+
labels:
44+
name: cass-operator-webhook
45+
name: cassandradatacenter-webhook-service
46+
namespace: test-operator
47+
spec:
48+
ports:
49+
- port: 443
50+
targetPort: 8443
51+
selector:
52+
name: cass-operator

‎docs/developer/olm/subscription.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: operators.coreos.com/v1alpha1
2+
kind: Subscription
3+
metadata:
4+
name: cass-operator-subscription
5+
namespace: test-operator
6+
spec:
7+
channel: stable
8+
name: cass-operator
9+
source: local
10+
sourceNamespace: olm

‎docs/developer/packaging.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Packaging for Operator Hub / Red Hat
2+
3+
## Tools
4+
* `opm`
5+
1. Checkout a copy of [operator-framework/operator-registry](https://github.com/operator-framework/operator-registry)
6+
2. Build with `make build`
7+
3. Binary resides at `bin/opm`
8+
* `k3d`
9+
10+
### Setup Test Cluster
11+
12+
Spin up local cluster for testing and install OLM
13+
14+
```console
15+
k3d cluster create
16+
kubectl create namespace test-operator
17+
operator-sdk olm install
18+
19+
# Optional
20+
kubectl delete catalogsource operatorhubio-catalog -n olm
21+
```
22+
23+
### Unsupported Resource Types
24+
The following types may not be included in the operator bundle
25+
26+
1. Namespace
27+
2. Secret
28+
3. ValidatingWebhookConfiguration
29+
4. Service
30+
31+
## Common Issues
32+
* `runAsUser` set to `999` in the deployment - OpenShift prefers to set a randomly assigned user at container start time. If this field is not left empty the user field must be set extremely high. 999 is too low.
33+
* Prerequisite custom resources have not been included in the appropriate section of the operator metadata testing page
34+
* The package has not been marked as published in the Red Hat repo.
35+
36+
# Upgrade Workflow
37+
38+
_Note see Red Hat's [Gitbook](https://redhat-connect.gitbook.io/partner-guide-for-red-hat-openshift-and-container/certify-your-operator/upgrading-your-operator)_
39+
40+
1. Copy an existing version as the base
41+
42+
```console
43+
OLD_VERSION=1.4.0
44+
NEW_VERSION=1.4.1
45+
cd operator/bundle
46+
cp -r $OLD_VERSION $NEW_VERSION
47+
cp bundle-$OLD_VERSION.Dockerfile bundle-$NEW_VERSION.Dockerfile
48+
```
49+
2. Rename files with version numbers
50+
51+
```console
52+
mv $NEW_VERSION/manifests/cass-operator.v$OLD_VERSION.clusterserviceversion.yaml $NEW_VERSION/manifests/cass-operator.v$NEW_VERSION.clusterserviceversion.yaml
53+
```
54+
55+
3. Update version numbers in ClusterServiceVersion files
56+
57+
```console
58+
sed s/"$OLD_VERSION"/$NEW_VERSION/g $NEW_VERSION/cass-operator.v$NEW_VERSION.clusterserviceversion.yaml
59+
sed s/"$OLD_VERSION"/$NEW_VERSION/g bundle-$NEW_VERSION.Dockerfile
60+
```
61+
62+
4. Copy in updated CRD
63+
64+
```console
65+
cp ../deploy/crds/* $NEW_VERSION/manifests/
66+
```
67+
68+
5. Compare the old and new CRDs for differences and update `$NEW_VERSION/manifests/cass-operator.v$NEW_VERSION.lusterserviceversion.yaml`
69+
1. Update `metadata.annotations.containerImage` version
70+
2. Update `metadata.annotations.createdAt` datestamp
71+
3. Update `metadata.annotations.name` field
72+
4. Update `spec.install.spec.deployments[0].template.spec.containers[0].image` version
73+
5. Update `spec.customresourcedefinitions.owned[0].specDescriptors` to include any new spec fields. See [Descriptor](https://github.com/openshift/console/blob/master/frontend/packages/operator-lifecycle-manager/src/components/descriptors/reference/reference.md) [Documentation](https://github.com/openshift/console/tree/release-4.3/frontend/packages/operator-lifecycle-manager/src/components/descriptors).
74+
6. Update `spec.customresourcedefinitions.owned[0].statusDescriptors` to include any new status fields
75+
7. Update `spec.replaces` to replace previous version
76+
8. Update `spec.version` with new version value
77+
6. Build bundle container and push to staging repo
78+
79+
```console
80+
docker build -t bradfordcp/cass-operator-bundle:$NEW_VERSION -f bundle-$NEW_VERSION.Dockerfile .
81+
docker push bradfordcp/cass-operator-bundle:NEW_VERSION
82+
```
83+
7. Build local catalog index for testing
84+
85+
```console
86+
opm index add --bundles bradfordcp/cass-operator-bundle:$NEW_VERSION --tag bradfordcp/catalog-index:latest -u docker
87+
docker push bradfordcp/catalog-index:latest
88+
```
89+
8. Add index as a [`CatalogSource`](olm/catalogsource.yaml) in k8s
90+
91+
```console
92+
kubectl apply -f docs/developer/olm/catalogsource.yaml
93+
```
94+
9. Verify packagemanifests are being pulled from index
95+
96+
```console
97+
kubectl describe packagemanifests cass-operator -n olm
98+
```
99+
10. Create an [`OperatorGroup`](olm/operatorgroup.yaml) to tell cass-operator where to watch for CassDC instances
100+
101+
```console
102+
kubectl apply -f docs/developer/olm/operatorgroup.yaml
103+
```
104+
11. Install prereqs that OLM doesn't handle
105+
106+
```console
107+
kubectl apply -f docs/developer/olm/prereqs.yaml
108+
```
109+
12. Install cass-operator with OLM via a [`Subscription`](olm/subscription.yaml)
110+
111+
```console
112+
kubectl apply -f docs/developer/olm/cass-operator.subscription.yaml
113+
```
114+
13. Check the `InstallPlan` to see the operator successfully installed
115+
116+
```console
117+
kubectl describe installplan -n test-operator | less
118+
```
119+
120+
If there is a failure, fix the issue locally, remove the `Subscription` and `CatalogSource`. Goto step 6 and repackage everything.
121+
14. Check the operator is running
122+
123+
```console
124+
kubectl get pods -n test-operator
125+
```
126+
14. Provision a sample cassdc in the default namespace
127+
15. Push to Red Hat
128+
129+
```console
130+
docker tag bradfordcp/cass-operator-bundle:$NEW_VERSION $REDHAT_REGISTRY/cass-operator-bundle:$NEW_VERSION
131+
docker push $REDHAT_REGISTRY/cass-operator-bundle:$NEW_VERSION
132+
```
133+
16. Red Hat automatically runs certification tests on push. These take 1-2 hours
134+
17. Login to the Red Hat project and verify certification results. Repeat steps 5-7 until certification passes
135+
18. Publish certified bundle

‎operator/.osdk-scorecard.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Note the structure below is intentional to avoid unnecessary warnings
2+
scorecard:
3+
bundle: deploy/olm-catalog/cass-operator
4+
plugins:
5+
# Basic CR Tests
6+
- basic:
7+
cr-manifest:
8+
- "example-cassdc-yaml/cassandra-3.11.6/example-cassdc-scorecard.yaml"
9+
# - basic:
10+
# cr-manifest:
11+
# - "example-cassdc-yaml/dse-6.8.x/example-cassdc-scorecard.yaml"
12+
13+
# # v1.0.0
14+
# - olm:
15+
# cr-manifest:
16+
# - "example-cassdc-yaml/cassandra-3.11.6/example-cassdc-scorecard.yaml"
17+
# csv-path: "deploy/olm-catalog/cass-operator/1.0.0/cass-operator.v1.0.0.clusterserviceversion.yaml"
18+
# - olm:
19+
# cr-manifest:
20+
# - "example-cassdc-yaml/dse-6.8.x/example-cassdc-scorecard.yaml"
21+
# csv-path: "deploy/olm-catalog/cass-operator/1.0.0/cass-operator.v1.0.0.clusterserviceversion.yaml"
22+
23+
# # v1.1.0
24+
# - olm:
25+
# cr-manifest:
26+
# - "example-cassdc-yaml/cassandra-3.11.6/example-cassdc-scorecard.yaml"
27+
# csv-path: "deploy/olm-catalog/cass-operator/1.1.0/cass-operator.v1.1.0.clusterserviceversion.yaml"
28+
# - olm:
29+
# cr-manifest:
30+
# - "example-cassdc-yaml/dse-6.8.x/example-cassdc-scorecard.yaml"
31+
# csv-path: "deploy/olm-catalog/cass-operator/1.1.0/cass-operator.v1.1.0.clusterserviceversion.yaml"
32+
33+
# v1.2.0
34+
# - olm:
35+
# cr-manifest:
36+
# - "example-cassdc-yaml/cassandra-3.11.6/example-cassdc-scorecard.yaml"
37+
# csv-path: "deploy/olm-catalog/cass-operator/1.2.0/cass-operator.v1.2.0.clusterserviceversion.yaml"
38+
# - olm:
39+
# cr-manifest:
40+
# - "example-cassdc-yaml/dse-6.8.x/example-cassdc-scorecard.yaml"
41+
# csv-path: "deploy/olm-catalog/cass-operator/1.2.0/cass-operator.v1.2.0.clusterserviceversion.yaml"
42+
43+
# v1.3.0
44+
# - olm:
45+
# cr-manifest:
46+
# - "example-cassdc-yaml/cassandra-3.11.6/example-cassdc-scorecard.yaml"
47+
# csv-path: "deploy/olm-catalog/cass-operator/1.3.0/cass-operator.v1.3.0.clusterserviceversion.yaml"
48+
# - olm:
49+
# cr-manifest:
50+
# - "example-cassdc-yaml/dse-6.8.x/example-cassdc-scorecard.yaml"
51+
# csv-path: "deploy/olm-catalog/cass-operator/1.3.0/cass-operator.v1.3.0.clusterserviceversion.yaml"
52+

0 commit comments

Comments
 (0)