Skip to content

Commit 091063a

Browse files
authored
Disconnected (#17)
Files for restricted network installation and proxy configuration
1 parent 21a107d commit 091063a

File tree

16 files changed

+611
-126
lines changed

16 files changed

+611
-126
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ downloads
33
install-dir
44
**/*.yml.git
55
**/*.yml.orig
6+
**/pull-secret*.json

.images/virtual-switch-final.png

161 KB
Loading

.images/virtual-switch.png

131 KB
Loading

README.md

Lines changed: 71 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11
# OCP4 on VMware vSphere UPI Automation
22

3-
The goal of this repo is to make deploying and redeploying a new OpenShift v4 cluster a snap. Using the same repo and with minor tweaks, it can be applied to any version of OpenShift higher than the current version of 4.3.
3+
The goal of this repo is to make deploying and redeploying a new OpenShift v4 cluster a snap. Using the same repo and with minor tweaks, it can be applied to any version of OpenShift higher than the current version of 4.4.
4+
5+
As it stands right now, the repo works for several installation usecases:
6+
* DHCP with OVA template
7+
* DHCP with PXE boot (needs helper node)
8+
* Static IPs for nodes (lack of isolated network to let helper run DHCP server)
9+
* w/o Cluster-wide Proxy (HTTP and SSL/TLS with certs supported)
10+
* Restricted network
411

512
> This repo is most ideal for Home Lab and Proof-of-Concept scenarios. Having said that, if prerequistes (below) can be met and if the vCenter service account can be locked down to access only certain resources and perform only certain actions, the same repo can then be used for DEV or higher environments. Refer to this [link](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) for more details on required permissions for a vCenter service account.
613
@@ -9,19 +16,16 @@ The goal of this repo is to make deploying and redeploying a new OpenShift v4 cl
916
1. vSphere ESXi and vCenter 6.7 installed. For vCenter 6.5 please see a cautionary note below:
1017
2. A datacenter created with a vSphere host added to it, a datastore exists and has adequate capacity
1118
3. The playbook(s) assumes you are running a [helper node](https://github.com/RedHatOfficial/ocp4-helpernode) running in the same network to provide all the necessary services such as [DHCP/DNS/HAProxy as LB]. Also, the MAC addresses for the machines should match between helper repo and this. If not using the helper node, the minimum expectation is that the webserver and tftp server (for PXE boot) are running on the same external host, which we will then treat as a helper node.
12-
* The necessary services such as [DNS/LB(Load Balancer] must be up and running before this repo can be used
13-
* This repo works in environments where :
14-
* DHCP is enabled: Use vSphere OVA template or use PXE boot
15-
* DHCP is disabled: Use Static IPs with CoreOS ISO file
16-
4. Ansible (preferably latest) with **Python 3** on the machine where this repo is cloned
19+
4. The necessary services such as [DNS/LB(Load Balancer] must be up and running before this repo can be used
20+
5. Ansible (preferably latest) with **Python 3** on the machine where this repo is cloned. Before you install Ansible, install the `epel-release`, run `yum -y install epel-release`
1721

18-
> For vSphere 6.5, the files relating to interaction with VMware/vCenter such as [this](roles/dhcp_ova/tasks/main.yml) ***may*** need to have `vmware_deploy_ovf` module to include [`cluster`](https://docs.ansible.com/ansible/latest/modules/vmware_deploy_ovf_module.html#parameter-cluster) and [`resource-pool`](https://docs.ansible.com/ansible/latest/modules/vmware_deploy_ovf_module.html#parameter-resource_pool) parameters and their values set to work correctly.
22+
> For vSphere 6.5, the files relating to interaction with VMware/vCenter such as [this](roles/dhcp_ova/tasks/main.yml) ***may*** need to have `vmware_deploy_ovf` module to include [`cluster`](https://docs.ansible.com/ansible/latest/modules/vmware_deploy_ovf_module.html#parameter-cluster), [`resource-pool`](https://docs.ansible.com/ansible/latest/modules/vmware_deploy_ovf_module.html#parameter-resource_pool) parameters and their values set to work correctly.
1923
2024
## Automatic generation of ignition and other supporting files
2125

2226
### Prerequisites
23-
> Pre-populated entries in **group_vars/all.yml** are ready to be used unless you need to customize further
24-
1. Get the ***pull secret*** from [here](https://cloud.redhat.com/OpenShift/install/vsphere/user-provisioned). Update [group_vars/all.yml](group_vars/all.yml) file on the line with `pull_secret` by providing the entire pull secret as a single line replacing the provided/incomplete pull secret
27+
> Pre-populated entries in **group_vars/all.yml** are ready to be used unless you need to customize further. Any updates described below refer to [group_vars/all.yml](group_vars/all.yml) unless otherwise specified.
28+
1. Get the ***pull secret*** from [here](https://cloud.redhat.com/OpenShift/install/vsphere/user-provisioned). Update the file on the line with `pull_secret` by providing the entire pull secret as a single line replacing the provided/incomplete pull secret
2529
2. Get the vCenter details:
2630
1. IP address
2731
2. Service account username (can be the same as admin)
@@ -35,9 +39,36 @@ The goal of this repo is to make deploying and redeploying a new OpenShift v4 cl
3539
1. base domain *(pre-populated with **example.com**)*
3640
2. cluster name *(pre-populated with **ocp4**)*
3741
5. HTTP URL of the ***bootstrap.ign*** file *(pre-populated with a example config pointing to helper node)*
38-
6. Update the inventory file: **staging** and under the `webservers.hosts` entry, use one of two options below :
42+
6. Update the inventory file: **staging** under the `webservers.hosts` entry, use one of two options below :
3943
1. **localhost** : if the `ansible-playbook` is being run on the same host as the webserver that would eventually host bootstrap.ign file
4044
2. the IP address or FQDN of the machine that would run the webserver.
45+
7. Furnish any proxy details with the section like below. If `proxy.enabled` is set to `False` anything related to the proxy wouldn't be picked up.
46+
```
47+
proxy:
48+
enabled: true
49+
http_proxy: http://helper.ocp4.example.com:3129
50+
https_proxy: http://helper.ocp4.example.com:3129
51+
no_proxy: example.com
52+
cert_content: |
53+
-----BEGIN CERTIFICATE-----
54+
<certficate content>
55+
-----END CERTIFICATE-----
56+
```
57+
8. When doing the restrcited network install and following instructions from [restricted.md](restricted.md), furnish details related to the registry with a section like below. If `registry.enabled` is set to `False` anything related to the `registry` wouldn't be picked up.
58+
```
59+
registry:
60+
enabled: true
61+
product_repo: openshift-release-dev
62+
product_release_name: ocp-release
63+
product_release_version: 4.4.0-x86_64
64+
username: ansible
65+
password: ansible
66+
67+
cert_content:
68+
host: helper.ocp4.example.com
69+
port: 5000
70+
repo: ocp4/openshift4
71+
```
4172

4273
> The step **#5** needn't exist at the time of running the setup/installation step, so provide an accurate guess of where and at what context path **bootstrap.ign** will eventually be served
4374
@@ -100,6 +131,11 @@ ansible-playbook -i staging dhcp_pxe.yml
100131
```sh
101132
ansible-playbook -i staging static_ips.yml
102133
```
134+
#### Option 4: DHCP + use of OVA template in a Restricted Network
135+
```sh
136+
# Refer to restricted.md file for more details
137+
ansible-playbook -i staging restricted_ova.yml
138+
```
103139

104140
#### Miscellaneous
105141
* If vCenter folder already exists with the template because you set the vCenter the last time you ran the ansible playbook but want a fresh deployment of VMs **after** you have erased all the existing VMs in the folder, append the following to the command you chose in the above step
@@ -132,7 +168,8 @@ ansible-playbook -i staging static_ips.yml
132168
If everything goes well you should be able to log into all of the machines using the following command:
133169

134170
```sh
135-
ssh -i ~/.ssh/ocp4 core@<IP_ADDRESS_OF_BOOTSTRAP_NODE>
171+
# Assuming you are able to resolve bootstrap.ocp4.example.com on this machine
172+
ssh -i ~/.ssh/ocp4 [email protected]
136173
```
137174

138175
Once logged in, on **bootstrap** node run the following command to understand if/how the masters are (being) setup:
@@ -152,3 +189,26 @@ export PATH=$(pwd)/bin:$PATH
152189
oc whoami
153190
oc get co
154191
```
192+
### Debugging
193+
194+
To check if the proxy information has been picked up:
195+
```sh
196+
# On Master
197+
cat /etc/systemd/system/machine-config-daemon-host.service.d/10-default-env.conf
198+
199+
# On Bootstrap
200+
cat /etc/systemd/system.conf.d/10-default-env.conf
201+
```
202+
To check if the registry information has been picked up:
203+
```sh
204+
# On Master or Bootstrap
205+
cat /etc/containers/registries.conf
206+
```
207+
To check if your certs have been picked up:
208+
```sh
209+
# On Master
210+
cat /etc/pki/ca-trust/source/anchors/openshift-config-user-ca-bundle.crt
211+
212+
# On Bootstrap
213+
cat /etc/pki/ca-trust/source/anchors/ca.crt
214+
```

ansible.cfg

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,3 @@
22
fact_caching = jsonfile
33
fact_caching_connection = /tmp
44
host_key_checking = False
5-
remote_user = root
6-
ask_pass = True
7-
8-
[privilege_escalation]
9-
become_ask_pass = True

group_vars/all.yml

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
---
21
helper_vm_ip: 192.168.86.180
32
bootstrap_ignition_url: "http://{{helper_vm_ip}}:8080/ignition/bootstrap.ign"
43
config:
@@ -18,8 +17,8 @@ vcenter:
1817
vm_power_state: poweredon
1918
templateName: rhcos-vmware
2019
download:
21-
clients_url: https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest
22-
dependencies_url: https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/latest/latest
20+
clients_url: https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/4.4.3/
21+
dependencies_url: https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.4/latest/
2322
govc: https://github.com/vmware/govmomi/releases/download/v0.22.1/govc_linux_amd64.gz
2423
bootstrap_vms:
2524
- { name : "bootstrap", mac: "00:50:56:a8:aa:a1", ip: "192.168.86.181"}
@@ -34,4 +33,25 @@ worker_vms:
3433
static_ip:
3534
gateway: 192.168.86.1
3635
netmask: 255.255.255.0
37-
network_interface_name: ens192
36+
network_interface_name: ens192
37+
proxy:
38+
enabled: true
39+
http_proxy: http://helper.ocp4.example.com:3129
40+
https_proxy: http://helper.ocp4.example.com:3129
41+
no_proxy: example.com
42+
cert_content: |
43+
-----BEGIN CERTIFICATE-----
44+
<certficate content>
45+
-----END CERTIFICATE-----
46+
registry:
47+
enabled: true
48+
product_repo: openshift-release-dev
49+
product_release_name: ocp-release
50+
product_release_version: 4.4.0-x86_64
51+
username: ansible
52+
password: ansible
53+
54+
cert_content:
55+
host: registry.ocp4.example.com
56+
port: 5000
57+
repo: ocp4/openshift4

restricted.md

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Installation in a Restricted Network
2+
3+
Installation in an restricted (network) environment is going to be different. In such a setting, the base cluster (bootstrap, masters[0,3], workers[0,3]) won't have open access to the internet. The only access this core infrastucture will be allowed to have, is to a registry on a node/VM that will mirror the contents of the installation repos hosted on quay.io.
4+
5+
This documentation will guide you in using this repo to setup this registry for installation in such as restricted network.
6+
7+
## Prerequisites
8+
0. Familiarity with this repo and a thorough reading of [README](README.md)
9+
1. Prepare a RHEL 8/Fedora VM or reuse `helper` node as registry host
10+
* Run `yum install -y podman httpd httpd-tools` when the VM is connected to internet
11+
2. The `helper` is the `bastion` host and as such the installation msut be run on the `helper`
12+
13+
## (Optional) Network isolation for OpenShift VMs + registry in vCenter
14+
> This section is meant for a lab environment, to practice a disconnected install. The subnets and IP addresses used below are shown only as an illustration.
15+
16+
### [Step 1] Create a Standard Network Port Group
17+
1. Right click on vSphere host 🠪 Configure 🠪 Networking 🠪 Virtual Switches
18+
2. Click on `ADD NETWORKING` button on the page (top right hand corner)
19+
3. Select `Virtual Machine Port Group for a Standard Switch` and click `NEXT`
20+
4. Select `New standard switch` (with defaults) and click `NEXT`
21+
5. Click `NEXT` for Step 3
22+
6. Click `OK` for the warning that there are no active physical network adapters
23+
7. Give a name for the port-group and choose a number between 0-4095 for VLAN ID and click `NEXT`
24+
8. Click `FINISH` on the final screen
25+
26+
When all done your setting should resemble somewhat like this image with the new `default` virtual switch.
27+
28+
[![](.images/virtual-switch.png)](.images/virtual-switch.png)
29+
30+
### [Step 2] Convert helper into a bastion host
31+
1. Right click on the `helper` VM and click on `Edit Settings`
32+
2. Click on the `ADD NEW DEVICE` (top right hand corner) when in the tab `Virtual Hardware`
33+
3. Choose `Network Adapter` and when its added, click on `Browse` under the drop-down for network, choose the newly added port-group and then click on `OK`
34+
4. SSH'ing into helper and using `ifconfig` determine the name of the new NIC. In my homelab, its `ens224`.
35+
* Assuming you assigned a static IP address to the first NIC `ens192`, copy `ifcfg-ens192` in `/etc/sysconfig/network-scripts` and save it as `ifcfg-ens224` in the same folder.
36+
* Edit the file `ifcfg-ens224` and ensure that the IP assigned is on a different subnet
37+
> In my homelab, `ens192` was in `192.168.86.0/24` subnet with GATEWAY pointing to 192.168.86.1 and `ens224` was in `192.168.87.0/24` subnet with GATWAY pointing at 192.168.87.1
38+
5. Restart the network with `systemctl restart NetworkManager`, a quick `ifconfig` or `nmcli device show ens224` should show the IP address picked up by the new NIC.
39+
40+
### [Step 3] Create a new VM for registry or reuse helper
41+
42+
#### If creating a new VM for registry (not re-using helper):
43+
1. Ensure that VM is setup, *connected to internet* and #2 of prerequisites above is run
44+
2. Assign it as hostname similar to `registry.ocp4.example.com`
45+
3. Create a `ifcfg-ens192` file under `/etc/sysconfig/network-scripts`, for reference my file looks like this :
46+
```sh
47+
TYPE="Ethernet"
48+
PROXY_METHOD="none"
49+
BROWSER_ONLY="no"
50+
BOOTPROTO="dhcp"
51+
DEFROUTE="yes"
52+
IPV4_FAILURE_FATAL="no"
53+
IPV6INIT="yes"
54+
IPV6_AUTOCONF="yes"
55+
IPV6_DEFROUTE="yes"
56+
IPV6_FAILURE_FATAL="no"
57+
IPV6_ADDR_GEN_MODE="stable-privacy"
58+
NAME="ens192"
59+
DEVICE="ens192"
60+
ONBOOT="yes"
61+
IPV6_PRIVACY="no"
62+
```
63+
64+
### [Step 4] Re-run helper playbook
65+
66+
In the helper `vars.yml` file ensure that all IP addresses (helper + bootstrap+ masters + workers) now belong to the new subnet `192.168.87.0/24`, that includes changing `helper.ipaddr` and `helper.networkifacename` to the new network adpater settings.
67+
68+
#### If creating a new VM for registry (not re-using helper)
69+
Make accomdations for registry node: `registry.ocp4.example.com` by changing the helper's DNS and DHCP config files as shown:
70+
1. Add a section for registry in helper's `vars.yml` file, as shown below. The `macaddr` should reflect the MAC address assigned to `ens192` adapter:
71+
```
72+
registry:
73+
name: "registry"
74+
ipaddr: "192.168.87.188"
75+
macaddr: "00:50:56:a8:4b:4f"
76+
```
77+
2. Add the following line to `templates/dhcpd.conf.j2` under the Static entries (for example, below the line for bootstrap)
78+
```
79+
host {{ registry.name }} { hardware ethernet {{ registry.macaddr }}; fixed-address {{ registry.ipaddr }}; }
80+
```
81+
3. Add the following line to `templates/zonefile.j2` (for example, below the line for bootstrap)
82+
```
83+
; Create entry for the registry host
84+
{{ registry.name }} IN A {{ registry.ipaddr }}
85+
;
86+
```
87+
4. Add the following line to `templates/reverse.j2` (for example, below the line for bootstrap)
88+
```
89+
{{ registry.ipaddr.split('.')[3] }} IN PTR {{ registry.name }}.{{ dns.clusterid }}.{{ dns.domain }}.
90+
;
91+
```
92+
93+
Now that helper is all set with is configuration, lets re-run the playbook and when it goes to success, reboot `registry.ocp4.example.com` so that it could pickup its IP address via DHCP.
94+
95+
## Run Ansible Automation
96+
97+
### Configurations
98+
99+
Modify `staging` file to look like below:
100+
```
101+
all:
102+
hosts:
103+
localhost:
104+
ansible_connection: local
105+
children:
106+
webservers:
107+
hosts:
108+
localhost:
109+
registries:
110+
hosts:
111+
registry.ocp4.example.com:
112+
ansible_ssh_user: root
113+
ansible_ssh_pass: <password for ease of installation>
114+
```
115+
> If reusing the helper the hostname under registries would be `localhost` and the credentials underneath removed as this repo is intented to be run on helper node
116+
117+
In `ansible.cfg` have the following as the content, as we will be running this as `root` user on helper node.
118+
```
119+
[defaults]
120+
fact_caching = jsonfile
121+
fact_caching_connection = /tmp
122+
host_key_checking = False
123+
remote_user = root
124+
```
125+
In [group_vars/all.yml](group_vars/all.yml)'s registry dict, with rest being optional, the following must be changed:
126+
* All IPs should now reflect the new subnet including
127+
* helper_vm_ip (the new IP obtained under the new subnet)
128+
* All IPs for bootstrap, masters, workers
129+
* static_ip.gateway
130+
* `registry.host` should be pointed to the IP or FQDN of the host mentioned in the previous step. If reusing the helper then use `helper.ocp4.example.com` else use (for example) `registry.ocp4.example.com`
131+
* `registry.product_release_version` must be updated to the latest version of the container image. _(Use [documentation links](#documentation-links))_
132+
* `vcenter.network` with the name of the new virtual switch port-group as we want all the new VMs land on the newly created virtual switch
133+
134+
### Installation in a restricted network
135+
136+
Now that helper, registry and the automation configs are all set, lets run the installation with the command:
137+
138+
```sh
139+
# If vCenter folders exist
140+
ansible-playbook --flush-cache -i staging restricted_ova.yml -e vcenter_preqs_met=true
141+
142+
# If vCenter folders DONT exist
143+
ansible-playbook --flush-cache -i staging restricted_ova.yml
144+
```
145+
146+
The final network topology should somewhat like the image below:
147+
[![](.images/virtual-switch-final.png)](.images/virtual-switch-final.png)
148+
149+
## Final Check
150+
151+
To check if the registry information has been picked up run and command below on either kind of nodes or check the decoded contents of secret `pull-secret` in `openshift-config` when the cluster is operational
152+
```sh
153+
# On Master or Bootstrap
154+
cat /etc/containers/registries.conf
155+
```
156+
157+
### Things to watch out for
158+
1. The OLM is broken on the restricted install, see #4 link below
159+
2. You have to figure out how to get traffic into the cluster, relying on the DNS of helper won't help as it is on a different subnet with no internet access. I use `dnsmasq` to route any traffic to `example.com` domain to public/accessible IP of the helper node
160+
161+
162+
## Documentation Links
163+
1. [Create a mirror registry for installation in a restricted network](https://docs.openshift.com/container-platform/4.4/installing/install_config/installing-restricted-networks-preparations.html)
164+
2. [Installing a cluster on vSphere in a restricted network](https://docs.openshift.com/container-platform/4.4/installing/installing_vsphere/installing-restricted-networks-vsphere.html)
165+
3. https://www.openshift.com/blog/openshift-4-2-disconnected-install
166+
4. [Using Operator Lifecycle Manager on restricted networks](https://docs.openshift.com/container-platform/4.4/operators/olm-restricted-networks.html)

0 commit comments

Comments
 (0)