Table of Contents
Introduction
If you’re looking to get started with the upstream version of OpenShift, also known as OKD, and you’re using Proxmox as your virtualization platform, this step-by-step guide will walk you through the process. From setting up your virtual environment to installing OKD, we’ll cover each phase to ensure that you have a smooth and successful deployment. Whether you’re new to OKD or Proxmox, this guide will provide you with clear instructions to get your OpenShift cluster up and running efficiently.
System Requirement
Number of Servers required to have a functional OKD cluster as follows
- Single node Cluster – 1 Server | 8 CPU | 16 GB Memory | 500 GB Disk
- Bastion node, just a gateway server | 2 CPU | 4 GB Memory | 50 GB Disk
- Infra node to host the DNS, DHCP and NTP | 4 CPU | 4 GB Memory | 50 GB Disk
- OKD Single node is Fedora CoreOS, all other servers are Rocky Linux 9.4
Before moving forward, we need to have a functional DNS, DHCP and NTP server.
DNS Requirements
A complete guide on setting up Bind DNS Setup for OpenShift Single Node
Only below DNS forward entries are required to setup a single node OKD cluster
ns1.okd.linuxsysadmins.lan. IN A 192.168.11.100
api.okd.linuxsysadmins.lan. IN A 192.168.11.101
api-int.okd.linuxsysadmins.lan. IN A 192.168.11.101
*.apps.okd.linuxsysadmins.lan. IN A 192.168.11.101
control-plane.okd.linuxsysadmins.lan. IN A 192.168.11.101
bastion.okd.linuxsysadmins.lan. IN A 192.168.11.11DNS reverse entries
100.11.168.192.in-addr.arpa. IN PTR ns1.okd.linuxsysadmins.lan.
101.11.168.192.in-addr.arpa. IN PTR api.okd.linuxsysadmins.lan.
101.11.168.192.in-addr.arpa. IN PTR api-int.okd.linuxsysadmins.lan.
101.11.168.192.in-addr.arpa. IN PTR control-plane.okd.linuxsysadmins.lan.
11.11.168.192.in-addr.arpa. IN PTR bastion.okd.linuxsysadmins.lan.DHCP Requirements
A complete guide on up running with a DHCP server as follows
Configuring DHCP Server for OpenShift Nodes
A fixed address for control plane from anyone of DHCP server.
# vim /etc/dhcp/dhcpd.conf
option domain-name "okd.linuxsysadmins.lan";
option domain-name-servers ns1.okd.linuxsysadmins.lan;
subnet 192.168.11.0 netmask 255.255.255.0 {
option ntp-servers 192.168.11.100;
}
host control-plane {
hardware ethernet BC:24:11:EC:84:07;
fixed-address 192.168.11.101;
option host-name "control-plane.okd.linuxsysadmins.lan";
option routers 192.168.11.1;
}Time Synchronization
Configure chrony as time server to provide time for control plane
# $ vim /etc/chrony.conf
server 192.168.0.1 iburst
sourcedir /run/chrony-dhcp
driftfile /var/lib/chrony/drift
makestep 1.0 3
allow 192.168.11.0/24
keyfile /etc/chrony.keys
logdir /var/log/chrony
log measurements statistics trackingThe Control Plane IP will be assigned via DHCP. If your DHCP server does not provide time information, the control plane bootstrap process may fail. In this case, you will need to manually configure chrony before initiating the bootstrap process to ensure the localhost API functions correctly on the control plane node.
bootkube.sh[99818]: Error: Post "https://localhost:6443/api/v1/namespaces/kube-system/events": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current timeTo avoid this share the time through DHCP
Preparing to Install OKD
Going with N-1 stable version 4.15.0-0.okd-2024-03-10-010116
Pick your right version from official GitHub Repo.
To easy the download set the variables
$ OKD_VERSION=4.15.0-0.okd-2024-03-10-010116
$ ARCH=x86_64Download the client and installer tar files, or manually obtain them by selecting matching versions from the official GitHub URL mentioned above.
$ curl -L https://github.com/okd-project/okd/releases/download/$OKD_VERSION/openshift-client-linux-$OKD_VERSION.tar.gz -o oc.tar.gz
$ curl -L https://github.com/okd-project/okd/releases/download/$OKD_VERSION/openshift-install-linux-$OKD_VERSION.tar.gz -o openshift-install-linux.tar.gzExtract the client packages and move under required location.
$ tar xvf openshift-client-linux-4.15.0-0.okd-2024-03-10-010116.tar.gz
$ mv oc kubectl /usr/local/bin/
$ restorecon -RFv /usr/local/bin/*Extract the Openshift Installer under the same location
$ tar xvf openshift-install-linux-4.15.0-0.okd-2024-03-10-010116.tar.gzDownload the Fedora CoreOS using below commands.
$ ISO_URL=$(./openshift-install coreos print-stream-json | grep location | grep $ARCH | grep iso | cut -d\" -f4)
$ curl -L $ISO_URL -o fcos-live.isoPreparing Installer configuration
Prior to preparing the ignition config files, copy or download the pull secret from Red Hat OpenShift Cluster Manager
Generate a SSH Key pair for authenticating with the nodes
$ ssh-keygenPrepare the install-config.yaml appropriate to your environment requirement.
- Here the my base domain is linuxsysadmins.lan
- Since we are going to have a all in one node, worker node replica should be 0
- In my case okd is the cluster name.
- My one and only control plane will be reside under 192.168.11.0/24
[ansible@bastion ~]$ cat install-config.yaml
apiVersion: v1
baseDomain: linuxsysadmins.lan
compute:
- name: worker
replicas: 0
controlPlane:
name: master
replicas: 1
metadata:
name: okd
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 192.168.11.0/24
networkType: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16
platform:
none: {}
bootstrapInPlace:
installationDisk: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0
pullSecret: '{"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoLXxxxxxxxxxxRkZZ","email":"baxxxxxxxx.com"}}}'
sshKey: |
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAGkrSprpi7Dc= ansible@bastion.okd.linuxsysadmins.lan
[ansible@bastion ~]$Generate the ignition files by running
$ mkdir sno
$ cp install-config.yaml sno
$ ./openshift-install --dir=sno create single-node-ignition-configOptional: Curious what inside the ignition file?
$ dnf install jq -y
$ cat sno/bootstrap-in-place-for-live-iso.ign | jqEmbed the ignition data into the ISO
$ alias coreos-installer='podman run --privileged --pull always --rm \
-v /dev:/dev -v /run/udev:/run/udev -v $PWD:/data \
-w /data quay.io/coreos/coreos-installer:release'$ coreos-installer iso ignition embed -fi sno/bootstrap-in-place-for-live-iso.ign fcos-live.isoTo streamline the setup of a virtual machine in Proxmox, first copy the ISO image from the Bastion node and upload it to the Proxmox ISO data-store.
$ scp ansible@192.168.11.11:/home/ansible/fcos-live.iso .Starting with Bootstrapping
Next, adjust the VM’s boot order to prioritize the CD-ROM drive. Attach the uploaded ISO to the virtual machine and power it on to begin the bootstrap process. Since a fixed DHCP address is defined for the Control-plane, it will automatically assign the correct IP address and proceed with the remaining bootstrap stages. This process ensures a smooth and efficient VM setup.
During the setup multiple reboots will be performed, almost after 30+ minutes we are done with the setup.
DEBUG Cluster Operator openshift-samples is stable INFO All cluster operators have completed progressing INFO Checking to see if there is a route at openshift-console/console... DEBUG Route found in openshift-console namespace: console DEBUG OpenShift console route is admitted INFO Install complete! INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/ansible/sno/auth/kubeconfig' INFO Access the OpenShift web-console here: https://console-openshift-console.apps.okd.linuxsysadmins.lan INFO Login to the console with user: "kubeadmin", and password: "rwzFQ-cY2i2-2etMP-J6xXA" DEBUG Time elapsed per stage: DEBUG Cluster Operators Available: 29m54s DEBUG Cluster Operators Stable: 1m20s INFO Time elapsed: 31m14s [ansible@bastion ~]$
Verify the Cluster
The credential for web console can be seen at the end of the setup output.
From the cli, use the kubeconfig file to list the nodes.
[ansible@bastion ~]$ oc get nodes --kubeconfig sno/auth/kubeconfig
NAME STATUS ROLES AGE VERSION
control-plane.okd.linuxsysadmins.lan Ready control-plane,master,worker 20m v1.28.7+6e2789b
[ansible@bastion ~]$
[ansible@bastion ~]$ oc get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
control-plane.okd.linuxsysadmins.lan Ready control-plane,master,worker 21m v1.28.7+6e2789b 192.168.11.101 <none> Fedora CoreOS 39.20240210.3.0 6.7.4-200.fc39.x86_64 cri-o://1.28.2
[ansible@bastion ~]$Login to the API from the CLI
$ oc login -u kubeadmin -p rwzFQ-cY2i2-2etMP-J6xXA https://console-openshift-console.apps.okd.linuxsysadmins.lan:6443List all the pods across all namespaces.
[ansible@bastion ~]$ oc get pods -A --kubeconfig sno/auth/kubeconfig
NAMESPACE NAME READY STATUS RESTARTS AGE
openshift-apiserver-operator openshift-apiserver-operator-66d44b7798-ltbd4 1/1 Running 2 11h
openshift-apiserver apiserver-8b96c9cfb-2gb5l 2/2 Running 3 11h
openshift-authentication-operator authentication-operator-766d9779d4-jxnzv 1/1 Running 2 11h
openshift-authentication oauth-openshift-66896c5b87-7c4gl 1/1 Running 1 11h
openshift-cloud-controller-manager-operator cluster-cloud-controller-manager-operator-66b4887675-dcsjc 3/3 Running 3 11h
openshift-cloud-credential-operator cloud-credential-operator-5db944c458-jtwd9 2/2 Running 3 11h
openshift-cluster-machine-approver machine-approver-77b9fcf5c7-tfkfv 2/2 Running 3 11h
openshift-cluster-node-tuning-operator cluster-node-tuning-operator-64c9946fd-cv57x 1/1 Running 1 11h
openshift-cluster-node-tuning-operator tuned-79crv 1/1 Running 1 11h
openshift-cluster-samples-operator cluster-samples-operator-54985c54c9-lzb6p 2/2 Running 2 11h
openshift-cluster-storage-operator cluster-storage-operator-7665d759d6-qbqbb 1/1 Running 1 11h
openshift-cluster-storage-operator csi-snapshot-controller-585dd5b969-7c7nj 1/1 Running 1 11h
openshift-cluster-storage-operator csi-snapshot-controller-operator-7ccf586476-9w2kt 1/1 Running 1 11h
openshift-cluster-storage-operator csi-snapshot-webhook-7c7b6d48f8-jwrbj 1/1 Running 1 11h
openshift-cluster-version cluster-version-operator-7944dbcf8b-x5t6k 1/1 Running 1 11h
openshift-config-operator openshift-config-operator-c645b84c6-ts267 1/1 Running 2 11h
openshift-console-operator console-operator-7b9c78dccc-t42vl 2/2 Running 2 11h
openshift-console console-58d76b78f7-wcqfs 1/1 Running 3 11h
openshift-console downloads-7cb97b5877-v5jbq 1/1 Running 1 11h
openshift-controller-manager-operator openshift-controller-manager-operator-9f84455b9-mhskx 1/1 Running 2 11h
openshift-controller-manager controller-manager-5999b8bfbb-xxnbh 1/1 Running 1 11h
openshift-dns-operator dns-operator-668f779d47-kwzs7 2/2 Running 2 11h
openshift-dns dns-default-br4dv 2/2 Running 2 11h
openshift-dns node-resolver-j6t6t 1/1 Running 1 11h
openshift-etcd-operator etcd-operator-64f9b89996-fbfw2 1/1 Running 2 11h
openshift-etcd etcd-control-plane.okd.linuxsysadmins.lan 4/4 Running 4 11h
openshift-etcd installer-3-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-image-registry cluster-image-registry-operator-58ff44549-5tpsq 1/1 Running 1 11h
openshift-image-registry node-ca-65hb6 1/1 Running 1 11h
openshift-ingress-canary ingress-canary-4xjlz 1/1 Running 1 11h
openshift-ingress-operator ingress-operator-6d9c9b9d7c-jcvjw 2/2 Running 5 (10h ago) 11h
openshift-ingress router-default-65c5bbf476-4mlcv 1/1 Running 3 (11h ago) 11h
openshift-insights insights-operator-79458b87cc-984gz 1/1 Running 3 11h
openshift-kube-apiserver-operator kube-apiserver-operator-5c5b686ff9-j79ln 1/1 Running 2 11h
openshift-kube-apiserver installer-3-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-apiserver installer-5-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-apiserver kube-apiserver-control-plane.okd.linuxsysadmins.lan 5/5 Running 5 11h
openshift-kube-controller-manager-operator kube-controller-manager-operator-58f696cbbf-8swbz 1/1 Running 2 11h
openshift-kube-controller-manager installer-3-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-controller-manager installer-4-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-controller-manager installer-5-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-controller-manager kube-controller-manager-control-plane.okd.linuxsysadmins.lan 4/4 Running 4 11h
openshift-kube-scheduler-operator openshift-kube-scheduler-operator-dc89c89fc-kp9lf 1/1 Running 2 11h
openshift-kube-scheduler installer-5-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-scheduler installer-6-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-scheduler openshift-kube-scheduler-control-plane.okd.linuxsysadmins.lan 3/3 Running 3 11h
openshift-kube-scheduler revision-pruner-6-control-plane.okd.linuxsysadmins.lan 0/1 Completed 0 11h
openshift-kube-storage-version-migrator-operator kube-storage-version-migrator-operator-7df68c455f-srj8g 1/1 Running 2 11h
openshift-kube-storage-version-migrator migrator-7dc4b56b9c-cg4jd 1/1 Running 1 11h
openshift-machine-api cluster-autoscaler-operator-64759f8bc6-hs4t9 2/2 Running 2 11h
openshift-machine-api cluster-baremetal-operator-5785c4dd4d-r2gjv 2/2 Running 3 11h
openshift-machine-api control-plane-machine-set-operator-7fbb586545-rfrxr 1/1 Running 2 11h
openshift-machine-api machine-api-operator-6599bd4cbd-4k4pj 2/2 Running 2 11h
openshift-machine-config-operator machine-config-controller-84c8d5686b-5zmhj 2/2 Running 2 11h
openshift-machine-config-operator machine-config-daemon-pjp76 2/2 Running 2 11h
openshift-machine-config-operator machine-config-operator-5974f7756-2k6jh 2/2 Running 2 11h
openshift-machine-config-operator machine-config-server-68v4t 1/1 Running 1 11h
openshift-marketplace community-operators-c5djb 1/1 Running 1 11h
openshift-marketplace marketplace-operator-6cccff5dfc-px98g 1/1 Running 4 11h
openshift-monitoring alertmanager-main-0 6/6 Running 6 11h
openshift-monitoring cluster-monitoring-operator-76dd7766d8-thsld 1/1 Running 1 11h
openshift-monitoring kube-state-metrics-64d7486765-8gmh6 3/3 Running 3 11h
openshift-monitoring monitoring-plugin-67cf9fcf4d-5tf8k 1/1 Running 1 11h
openshift-monitoring node-exporter-sd6rv 2/2 Running 2 11h
openshift-monitoring openshift-state-metrics-b7f48f7cf-56f65 3/3 Running 3 11h
openshift-monitoring prometheus-adapter-74c589c598-8d2nc 1/1 Running 1 11h
openshift-monitoring prometheus-k8s-0 6/6 Running 6 11h
openshift-monitoring prometheus-operator-7c5d9bbff5-rrzr2 2/2 Running 2 11h
openshift-monitoring prometheus-operator-admission-webhook-8574c9c8b-8b7fk 1/1 Running 1 11h
openshift-monitoring telemeter-client-6d5694d9df-wx8cs 3/3 Running 3 11h
openshift-monitoring thanos-querier-5bd55ff96-bns7q 6/6 Running 6 11h
openshift-multus multus-additional-cni-plugins-wsdqc 1/1 Running 1 11h
openshift-multus multus-admission-controller-64d6895c6c-v9skl 2/2 Running 0 10h
openshift-multus multus-n7ml5 1/1 Running 2 11h
openshift-multus network-metrics-daemon-28ljx 2/2 Running 2 11h
openshift-network-diagnostics network-check-source-5f9b68d7f8-rfhcx 1/1 Running 1 11h
openshift-network-diagnostics network-check-target-bvjtw 1/1 Running 1 11h
openshift-network-node-identity network-node-identity-m4n4h 2/2 Running 3 11h
openshift-network-operator network-operator-656d4d696f-bk58z 1/1 Running 2 11h
openshift-oauth-apiserver apiserver-df468d8c-dgpwr 1/1 Running 3 11h
openshift-operator-lifecycle-manager catalog-operator-5db8cc987c-7sjhz 1/1 Running 1 11h
openshift-operator-lifecycle-manager collect-profiles-28711410-nsjsd 0/1 Completed 0 37m
openshift-operator-lifecycle-manager collect-profiles-28711425-tnh5d 0/1 Completed 0 22m
openshift-operator-lifecycle-manager collect-profiles-28711440-kfw69 0/1 Completed 0 7m
openshift-operator-lifecycle-manager olm-operator-65b47cf747-26r8v 1/1 Running 1 11h
openshift-operator-lifecycle-manager package-server-manager-7d888667b7-lqz4q 2/2 Running 2 11h
openshift-operator-lifecycle-manager packageserver-69b5f7c498-thwgg 1/1 Running 1 11h
openshift-ovn-kubernetes ovnkube-control-plane-68798d4445-p4kps 2/2 Running 3 11h
openshift-ovn-kubernetes ovnkube-node-sgfpb 8/8 Running 8 11h
openshift-route-controller-manager route-controller-manager-7c7fd897bc-mmsl2 1/1 Running 1 11h
openshift-service-ca-operator service-ca-operator-5c5d464f8-2jfv6 1/1 Running 2 11h
openshift-service-ca service-ca-6ff7684868-8fvnr 1/1 Running 1 11h
[ansible@bastion ~]$To access the UI, add below host entries from where you are trying to access.
# vim /etc/hosts
192.168.11.101 console-openshift-console.apps.okd.linuxsysadmins.lan
192.168.11.101 oauth-openshift.apps.okd.linuxsysadmins.lanAccessing Console from the Browser, the password for UI can be found here.
[ansible@bastion ~]$ ls -lthr sno/auth/kubeadmin-password
-rw-r-----. 1 ansible ansible 23 Aug 3 04:24 sno/auth/kubeadmin-password
[ansible@bastion ~]$
[ansible@bastion ~]$ cat sno/auth/kubeadmin-password
rwzFQ-cY2i2-2etMP-J6xXA
[ansible@bastion ~]$https://console-openshift-console.apps.okd.linuxsysadmins.lan
That’s it, we are done with setting up a single node OKD cluster.