Skip to content

Commit 282b598

Browse files
jaehyeon-kimjaehyeon-kim
authored
Merge pull request #17 from factorhouse/feature/FAC-637_helm-tidyup
Helm Chart Tidyup
2 parents 5012153 + fd5e6d3 commit 282b598

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+4305
-222
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.idea
2+
.vscode

‎README.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,12 @@
55

66
Official Helm Charts for Factor House products. Currently supported:
77

8-
* [Kpow](charts/kpow/README.md) (`factorhouse/kpow`)
9-
* [Kpow Community Edition](charts/kpow-ce/README.md) (`factorhouse/kpow-ce`)
10-
* [Kpow AWS Marketplace (Kpow Annual)](charts/kpow-annual/README.md)(`factorhouse/kpow-annual-chart`)
11-
* [Flex](charts/flex/README.md) (`factorhouse/flex`)
12-
* [Flex Community Edition](charts/flex-ce/README.md) (`factorhouse/flex-ce`)
8+
- [Kpow](charts/kpow/README.md) (`factorhouse/kpow`)
9+
- [Kpow Community Edition](charts/kpow-ce/README.md) (`factorhouse/kpow-ce`)
10+
- [Kpow AWS Annual (AWS Marketplace)](charts/kpow-aws-annual/README.md)(`factorhouse/kpow-aws-annual`)
11+
- [Kpow AWS Hourly (AWS Marketplace)](charts/kpow-aws-hourly/README.md)(`factorhouse/kpow-aws-hourly`)
12+
- [Flex](charts/flex/README.md) (`factorhouse/flex`)
13+
- [Flex Community Edition](charts/flex-ce/README.md) (`factorhouse/flex-ce`)
1314

1415
# Installation
1516

‎_readme_gen/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# Helm Chart README Generator
2+
3+
This directory contains the source code and automation script for generating the `README.md` files for our various Helm charts. The purpose of this tool is to solve the problem of keeping multiple, similar `README.md` files in sync.
4+
5+
## How It Works
6+
7+
The system is based on two key concepts: **Partials** and **Templates**.
8+
9+
- **Partials (`_partials/`):** These are small, reusable Markdown files that contain a single, specific piece of documentation, like installation instructions or prerequisites.
10+
- **Templates (`templates/`):** These are the master "blueprint" files for each final `README.md`. They define the structure of the document and use a special directive, `@@include(...)`, to pull in the content from the partials.
11+
12+
The `generate.py` script reads a template, finds all the `@@include` directives, replaces them with the content from the corresponding partial files, and writes the final, assembled `README.md` to the correct chart directory.
13+
14+
### Directory Structure
15+
16+
The project is organized by product to keep the source files separate and clean.
17+
18+
```
19+
_readme_gen/
20+
├── README.md # This file
21+
├── generate.py # The master Python script
22+
│
23+
├── flex/ # Source files for the "flex" product
24+
│ ├── _partials/ # Reusable Markdown snippets for flex
25+
│ │ └── _... .md
26+
│ └── templates/ # Master templates for each flex README
27+
│ └── flex.template.md
28+
│
29+
└── kpow/ # Source files for the "kpow" product
30+
├── _partials/ # Reusable Markdown snippets for kpow
31+
│ └── _... .md
32+
└── templates/ # Master templates for each kpow README
33+
└── kpow.template.md
34+
```
35+
36+
---
37+
38+
## Usage
39+
40+
The script can be run from the command line to generate READMEs for a specific product or for all products at once.
41+
42+
### Generating READMEs for a Specific Product
43+
44+
Provide the product name (`kpow`, `flex`, etc.) as an argument to the script. This is useful for local development when you are only working on one product.
45+
46+
**Syntax:**
47+
48+
```bash
49+
python generate.py <product_name>
50+
```
51+
52+
**Examples:**
53+
54+
```bash
55+
# Generate all READMEs for Kpow
56+
python generate.py kpow
57+
58+
# Generate all READMEs for Flex
59+
python generate.py flex
60+
```
61+
62+
### Generating All READMEs
63+
64+
Running the script without any arguments will default to generating the READMEs for **all** products defined in the configuration. This is the mode used by our CI/CD automation.
65+
66+
**Examples:**
67+
68+
```bash
69+
# Generate all READMEs for all products
70+
python generate.py
71+
```
72+
73+
Or explicitly:
74+
75+
```bash
76+
python generate.py all
77+
```
78+
79+
### Getting Help
80+
81+
To see the list of available products and options, use the `--help` flag.
82+
83+
```bash
84+
python generate.py --help
85+
```
86+
87+
---
88+
89+
## Adding a New Product
90+
91+
To add a new product (e.g., "platform") to the generator, follow these steps:
92+
93+
1. **Create the Directory Structure:**
94+
Create a new folder inside `_readme_gen/` with the product's name, and add `_partials` and `templates` subdirectories.
95+
96+
```
97+
_readme_gen/
98+
└── platform/
99+
├── _partials/
100+
└── templates/
101+
```
102+
103+
2. **Add Content:**
104+
Create your reusable `.md` files in `_partials/` and your master `platform.template.md` files in `templates/`.
105+
106+
3. **Update the Configuration:**
107+
Open `generate.py` and add a new top-level entry for `"platform"` inside the `README_CONFIG` dictionary. Follow the existing structure to define the template and output paths for each of your new chart's READMEs.
108+
109+
**Example addition to `README_CONFIG`:**
110+
111+
```python
112+
"gizmo": [
113+
{
114+
"template": BASE_DIR / "platform" / "templates/platform.template.md",
115+
"output": BASE_DIR / f"../{CHART_FOLDER}/gizmo/README.md",
116+
},
117+
# ... more chart variations for platform
118+
],
119+
```

‎_readme_gen/flex/_partials/_configure-eks-environment.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
### Configure Kubernetes/EKS
2+
3+
You need to connect to a Kubernetes environment before you can install Kpow.
4+
5+
The following examples demonstrate installing Kpow in [Amazon EKS](https://aws.amazon.com/eks/).
6+
7+
```bash
8+
aws eks --region <your-aws-region> update-kubeconfig --name <your-eks-cluster-name>
9+
10+
Updated context arn:aws:eks:<your-aws-region>:123123123:cluster/<your-eks-cluster-name> in /your/.kube/config
11+
```
12+
13+
#### Confirm Kubernetes Cluster Availability
14+
15+
```bash
16+
kubectl get nodes
17+
18+
NAME STATUS ROLES AGE VERSION
19+
ip-192-168-...-21.ec2.internal Ready <none> 2m15s v1.32.9-eks-113cf36
20+
...
21+
```

‎_readme_gen/flex/_partials/_configure-helm-flex.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Add the Factor House Helm Repository in order to use the Flex Helm Chart.
2+
3+
```bash
4+
helm repo add factorhouse https://charts.factorhouse.io
5+
```
6+
7+
Update Helm repositories to ensure you install the latest version of Flex.
8+
9+
```bash
10+
helm repo update
11+
```

‎_readme_gen/flex/_partials/_feature-manage-flex-instance.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
#### Set the $POD_NAME variable and test the Flex UI
2+
3+
Follow the notes instructions to set the $POD_NAME variable and configure port forwarding to the Flex UI.
4+
5+
```bash
6+
export POD_NAME=$(kubectl get pods --namespace factorhouse -l "app.kubernetes.io/name=flex,app.kubernetes.io/instance=flex" -o jsonpath="{.items[0].metadata.name}")
7+
echo "Visit http://127.0.0.1:3000 to use your application"
8+
kubectl --namespace factorhouse port-forward $POD_NAME 3000:3000
9+
```
10+
11+
Flex is now available on [http://127.0.0.1:3000](http://127.0.0.1:3000).
12+
13+
#### Check the Flex Pod
14+
15+
```bash
16+
kubectl describe pods --namespace factorhouse
17+
18+
Name: flex-9988df6b6-vvf8z
19+
Namespace: factorhouse
20+
Priority: 0
21+
Node: ip-172-31-33-42.ap-southeast-2.compute.internal/172.31.33.42
22+
Start Time: Mon, 31 May 2021 17:22:22 +1000
23+
Labels: app.kubernetes.io/instance=flex
24+
app.kubernetes.io/name=flex
25+
pod-template-hash=9988df6b6
26+
Annotations: kubernetes.io/psp: eks.privileged
27+
Status: Running
28+
```
29+
30+
#### View the Flex Pod Logs
31+
32+
```bash
33+
kubectl logs --namespace factorhouse $POD_NAME
34+
35+
11:36:49.111 INFO [main] flex.system ? start Flex
36+
...
37+
```
38+
39+
#### Remove Flex
40+
41+
```bash
42+
helm delete --namespace factorhouse flex
43+
```

‎_readme_gen/flex/_partials/_feature-manage-sensitive-env-vars.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
This helm chart accepts the name of a secret containing sensitive parameters, e.g.
2+
3+
```yaml
4+
apiVersion: v1
5+
kind: Secret
6+
metadata:
7+
name: flex-secrets
8+
data:
9+
SASL_JAAS_CONFIG: a3JnLmFwYWNoXS5rYWZrYS5jb21tb24uc2VjdXJpdHkucGxhaW4uUGxhaW5Mb2dpbk2vZHVsZSByZXF1aXJiZCB1c2VybmFtZT0iTFQ1V0ZaV1BRWUpHNzRJQyIgcGFzc3dvcmQ9IjlYUFVYS3BLYUQxYzVJdXVNRjRPKzZ2NxJ0a1E4aS9yWUp6YlppdlgvZnNiTG51eGY4SnlFT1dUeXMvTnJ1bTAiBwo=
10+
CONFLUENT_API_SECRET: NFJSejlReFNTTXlTcGhXdjNLMHNYY1F6UGNURmdadlNYT0ZXSXViWFJySmx2N3A2WStSenROQnVpYThvNG1NSRo=
11+
```
12+
13+
```bash
14+
kubectl apply -f ./flex-secrets.yaml --namespace factorhouse
15+
```
16+
17+
Then run the helm chart (this can be used in conjunction with `envFromConfigMap`)
18+
19+
See the Kubernetes documentation
20+
on [configuring all key value pairs in a secret as environment variables](https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#configure-all-key-value-pairs-in-a-secret-as-container-environment-variables)
21+
for more information.

‎_readme_gen/flex/_partials/_feature-memory-cpu-requirements.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
The chart runs Flex with Guaranteed QoS, having resource request and limit set to these values by default:
2+
3+
```yaml
4+
resources:
5+
limits:
6+
cpu: 2
7+
memory: 8Gi
8+
requests:
9+
cpu: 2
10+
memory: 8Gi
11+
```
12+
13+
These default resource settings are conservative, suited to a deployment of Flex that manages multiple Flink clusters
14+
and associated resources.
15+
16+
When running Flex with a single Flink cluster you can experiment with reducing those resources as far as our suggested
17+
minimum:
18+
19+
#### Minimum Resource Requirements
20+
21+
```yaml
22+
resources:
23+
limits:
24+
cpu: 1
25+
memory: 2Gi
26+
requests:
27+
cpu: 1
28+
memory: 2Gi
29+
```

‎_readme_gen/flex/_partials/_feature-provide-files-to-flex-pod.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
There are occasions where you must provide files to the Flex Pod in order for Flex to run correctly, such files include:
2+
3+
- RBAC configuration
4+
- SSL Keystores
5+
- SSL Truststores
6+
7+
How you provide these files is down to user preference, we are not able to provide any support or instruction in this
8+
regard.
9+
10+
You may find the Kubernetes documentation
11+
on [injecting data into applications](https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/#create-a-pod-that-has-access-to-the-secret-data-through-a-volume)
12+
useful.

‎_readme_gen/flex/_partials/_general-footer.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
### Get Help!
2+
3+
If you have any issues or errors, please contact [email protected].
4+
5+
### Licensing and Modifications
6+
7+
This repository is Apache 2.0 licensed, you are welcome to clone and modify as required.

0 commit comments

Comments
 (0)