Skip to content

Commit 9755eab

Browse files
bobbyilievbobbyiliev
committed
Initial lgalloc support
1 parent 80bacb1 commit 9755eab

File tree

9 files changed

+503
-11
lines changed

9 files changed

+503
-11
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,4 @@ tests/.terraform.lock.hcl
3838
*.env
3939
.envrc
4040
*.bak
41+
venv/

‎README.md‎

Lines changed: 53 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,55 @@ pip install -r requirements.txt
6666

6767
This will install the required Python packages in a virtual environment.
6868

69+
## Disk Support for Materialize on Azure
70+
71+
This module supports configuring disk support for Materialize on Azure using **local NVMe SSDs** available in specific VM families, along with **OpenEBS** and `lgalloc` for volume management.
72+
73+
### Recommended Azure VM Types with Local NVMe Disks
74+
75+
Materialize benefits from fast ephemeral storage and recommends a **minimum 2:1 disk-to-RAM ratio**. The [Lsv3-series](https://learn.microsoft.com/en-us/azure/virtual-machines/lsv3-series) virtual machines offer high-throughput local NVMe SSD storage and are ideal for performance-intensive workloads.
76+
77+
| VM Size | vCPUs | Memory | NVMe Storage | Disk-to-RAM Ratio |
78+
|------------------|-------|---------|--------------|-------------------|
79+
| Standard\_L8s\_v3 | 8 | 64 GiB | 1,900 GiB | ~29:1 |
80+
| Standard\_L16s\_v3 | 16 | 128 GiB | 3,800 GiB | ~29:1 |
81+
| Standard\_L32s\_v3 | 32 | 256 GiB | 7,600 GiB | ~29:1 |
82+
83+
> [!NOTE]
84+
> These VM types provide **ephemeral local NVMe SSD disks**. Data is lost when the VM is stopped or deleted, so they should only be used for **temporary or performance-critical data** managed by Materialize.
85+
86+
### Enabling Disk Support on Azure
87+
88+
When `enable_disk_support` is set to `true`, the module:
89+
90+
1. Uses a bootstrap container to identify and configure available NVMe disks
91+
1. Sets up **OpenEBS** with `lvm-localpv` to manage the ephemeral disks
92+
1. Creates a StorageClass for Materialize
93+
94+
Example configuration:
95+
96+
```hcl
97+
enable_disk_support = true
98+
99+
aks_config = {
100+
node_count = 2
101+
vm_size = "Standard_L8s_v3"
102+
os_disk_size_gb = 100
103+
min_nodes = 2
104+
max_nodes = 4
105+
}
106+
107+
disk_support_config = {
108+
install_openebs = true
109+
run_disk_setup_script = true
110+
create_storage_class = true
111+
112+
openebs_version = "4.2.0"
113+
openebs_namespace = "openebs"
114+
storage_class_name = "openebs-lvm-instance-store-ext4"
115+
}
116+
```
117+
69118
## Requirements
70119

71120
| Name | Version |
@@ -100,11 +149,14 @@ No resources.
100149

101150
| Name | Description | Type | Default | Required |
102151
|------|-------------|------|---------|:--------:|
103-
| <a name="input_aks_config"></a> [aks\_config](#input\_aks\_config) | AKS cluster configuration | <pre>object({<br/> vm_size = string<br/> disk_size_gb = number<br/> min_nodes = number<br/> max_nodes = number<br/> })</pre> | <pre>{<br/> "disk_size_gb": 100,<br/> "max_nodes": 5,<br/> "min_nodes": 1,<br/> "vm_size": "Standard_E8ps_v6"<br/>}</pre> | no |
152+
| <a name="input_aks_config"></a> [aks\_config](#input\_aks\_config) | AKS cluster configuration | <pre>object({<br/> vm_size = string<br/> disk_size_gb = number<br/> min_nodes = number<br/> max_nodes = number<br/> })</pre> | <pre>{<br/> "disk_size_gb": 100,<br/> "max_nodes": 5,<br/> "min_nodes": 1,<br/> "vm_size": "Standard_L8s_v3"<br/>}</pre> | no |
104153
| <a name="input_cert_manager_chart_version"></a> [cert\_manager\_chart\_version](#input\_cert\_manager\_chart\_version) | Version of the cert-manager helm chart to install. | `string` | `"v1.17.1"` | no |
105154
| <a name="input_cert_manager_install_timeout"></a> [cert\_manager\_install\_timeout](#input\_cert\_manager\_install\_timeout) | Timeout for installing the cert-manager helm chart, in seconds. | `number` | `300` | no |
106155
| <a name="input_cert_manager_namespace"></a> [cert\_manager\_namespace](#input\_cert\_manager\_namespace) | The name of the namespace in which cert-manager is or will be installed. | `string` | `"cert-manager"` | no |
107156
| <a name="input_database_config"></a> [database\_config](#input\_database\_config) | Azure Database for PostgreSQL configuration | <pre>object({<br/> sku_name = optional(string, "GP_Standard_D2s_v3")<br/> postgres_version = optional(string, "15")<br/> password = string<br/> username = optional(string, "materialize")<br/> db_name = optional(string, "materialize")<br/> })</pre> | n/a | yes |
157+
| <a name="input_disk_setup_image"></a> [disk\_setup\_image](#input\_disk\_setup\_image) | Docker image for the disk setup script | `string` | `"materialize/ephemeral-storage-setup-image:v0.1.2"` | no |
158+
| <a name="input_disk_support_config"></a> [disk\_support\_config](#input\_disk\_support\_config) | Advanced configuration for disk support (only used when enable\_disk\_support = true) | <pre>object({<br/> install_openebs = optional(bool, true)<br/> run_disk_setup_script = optional(bool, true)<br/> create_storage_class = optional(bool, true)<br/> openebs_version = optional(string, "4.2.0")<br/> openebs_namespace = optional(string, "openebs")<br/> storage_class_name = optional(string, "openebs-lvm-instance-store-ext4")<br/> storage_class_provisioner = optional(string, "local.csi.openebs.io")<br/> storage_class_parameters = optional(object({<br/> storage = optional(string, "lvm")<br/> fsType = optional(string, "ext4")<br/> volgroup = optional(string, "instance-store-vg")<br/> }), {})<br/> })</pre> | `{}` | no |
159+
| <a name="input_enable_disk_support"></a> [enable\_disk\_support](#input\_enable\_disk\_support) | Enable disk support for Materialize using OpenEBS and local SSDs. When enabled, this configures OpenEBS, runs the disk setup script, and creates appropriate storage classes. | `bool` | `true` | no |
108160
| <a name="input_helm_chart"></a> [helm\_chart](#input\_helm\_chart) | Chart name from repository or local path to chart. For local charts, set the path to the chart directory. | `string` | `"materialize-operator"` | no |
109161
| <a name="input_helm_values"></a> [helm\_values](#input\_helm\_values) | Additional Helm values to merge with defaults | `any` | `{}` | no |
110162
| <a name="input_install_cert_manager"></a> [install\_cert\_manager](#input\_install\_cert\_manager) | Whether to install cert-manager. | `bool` | `true` | no |

‎docs/header.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,3 +64,52 @@ pip install -r requirements.txt
6464
```
6565

6666
This will install the required Python packages in a virtual environment.
67+
68+
## Disk Support for Materialize on Azure
69+
70+
This module supports configuring disk support for Materialize on Azure using **local NVMe SSDs** available in specific VM families, along with **OpenEBS** and `lgalloc` for volume management.
71+
72+
### Recommended Azure VM Types with Local NVMe Disks
73+
74+
Materialize benefits from fast ephemeral storage and recommends a **minimum 2:1 disk-to-RAM ratio**. The [Lsv3-series](https://learn.microsoft.com/en-us/azure/virtual-machines/lsv3-series) virtual machines offer high-throughput local NVMe SSD storage and are ideal for performance-intensive workloads.
75+
76+
| VM Size | vCPUs | Memory | NVMe Storage | Disk-to-RAM Ratio |
77+
|------------------|-------|---------|--------------|-------------------|
78+
| Standard_L8s_v3 | 8 | 64 GiB | 1,900 GiB | ~29:1 |
79+
| Standard_L16s_v3 | 16 | 128 GiB | 3,800 GiB | ~29:1 |
80+
| Standard_L32s_v3 | 32 | 256 GiB | 7,600 GiB | ~29:1 |
81+
82+
> [!NOTE]
83+
> These VM types provide **ephemeral local NVMe SSD disks**. Data is lost when the VM is stopped or deleted, so they should only be used for **temporary or performance-critical data** managed by Materialize.
84+
85+
### Enabling Disk Support on Azure
86+
87+
When `enable_disk_support` is set to `true`, the module:
88+
89+
1. Uses a bootstrap container to identify and configure available NVMe disks
90+
1. Sets up **OpenEBS** with `lvm-localpv` to manage the ephemeral disks
91+
1. Creates a StorageClass for Materialize
92+
93+
Example configuration:
94+
95+
```hcl
96+
enable_disk_support = true
97+
98+
aks_config = {
99+
node_count = 2
100+
vm_size = "Standard_L8s_v3"
101+
os_disk_size_gb = 100
102+
min_nodes = 2
103+
max_nodes = 4
104+
}
105+
106+
disk_support_config = {
107+
install_openebs = true
108+
run_disk_setup_script = true
109+
create_storage_class = true
110+
111+
openebs_version = "4.2.0"
112+
openebs_namespace = "openebs"
113+
storage_class_name = "openebs-lvm-instance-store-ext4"
114+
}
115+
```

‎main.tf‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,22 @@ locals {
33
managed_by = "terraform"
44
module = "materialize"
55
})
6+
7+
# Disk support configuration
8+
disk_config = {
9+
install_openebs = var.enable_disk_support ? lookup(var.disk_support_config, "install_openebs", true) : false
10+
run_disk_setup_script = var.enable_disk_support ? lookup(var.disk_support_config, "run_disk_setup_script", true) : false
11+
create_storage_class = var.enable_disk_support ? lookup(var.disk_support_config, "create_storage_class", true) : false
12+
openebs_version = lookup(var.disk_support_config, "openebs_version", "4.2.0")
13+
openebs_namespace = lookup(var.disk_support_config, "openebs_namespace", "openebs")
14+
storage_class_name = lookup(var.disk_support_config, "storage_class_name", "openebs-lvm-instance-store-ext4")
15+
storage_class_provisioner = lookup(var.disk_support_config, "storage_class_provisioner", "local.csi.openebs.io")
16+
storage_class_parameters = {
17+
storage = try(var.disk_support_config.storage_class_parameters.storage, "lvm")
18+
fsType = try(var.disk_support_config.storage_class_parameters.fsType, "ext4")
19+
volgroup = try(var.disk_support_config.storage_class_parameters.volgroup, "instance-store-vg")
20+
}
21+
}
622
}
723

824
module "networking" {
@@ -36,6 +52,13 @@ module "aks" {
3652
min_nodes = var.aks_config.min_nodes
3753
max_nodes = var.aks_config.max_nodes
3854

55+
# Disk support configuration
56+
enable_disk_setup = local.disk_config.run_disk_setup_script
57+
install_openebs = local.disk_config.install_openebs
58+
openebs_namespace = local.disk_config.openebs_namespace
59+
openebs_version = local.disk_config.openebs_version
60+
disk_setup_image = var.disk_setup_image
61+
3962
tags = local.common_labels
4063
}
4164

@@ -133,6 +156,14 @@ locals {
133156
}
134157
}
135158
} : {}
159+
storage = var.enable_disk_support ? {
160+
storageClass = {
161+
create = local.disk_config.create_storage_class
162+
name = local.disk_config.storage_class_name
163+
provisioner = local.disk_config.storage_class_provisioner
164+
parameters = local.disk_config.storage_class_parameters
165+
}
166+
} : {}
136167
}
137168

138169
merged_helm_values = merge(local.default_helm_values, var.helm_values)

0 commit comments

Comments
 (0)