How to deploy OpenShift Origin 10 (OKD) on one node with GlusterFS

Question

I am able to install OKD on one node and scaleup on multiple node accordngly. But now i want to install OKD with GlusterFS on one node and then extend this on multiple nodes. Currently i am getting error that at least three nodes required. How i can bypass this check in ansible?

As per github documentations i have three options

Configuring a new, natively-hosted GlusterFS cluster. In this scenario, GlusterFS pods are deployed on nodes in the OpenShift cluster which are configured to provide storage.
Configuring a new, external GlusterFS cluster. In this scenario, the cluster nodes have the GlusterFS software pre-installed but have not been configured yet. The installer will take care of configuring the cluster(s) for use by OpenShift applications.
Using existing GlusterFS clusters. In this scenario, one or more GlusterFS clusters are assumed to be already setup. These clusters can be either natively-hosted or external, but must be managed by a heketi service.

Can option 2 or 3 be used to start with one node and extend accordingly? I have install glusterfs cluster on one node and extend it to second node but how to introduce in openshift?

https://imranrazakh.blogspot.com/2018/08/

ImranRazaKhan ImranRazaKhan · Accepted Answer · 2018-10-02T08:38:48

I found one way to install glusterfs on one node, Find below all in one installation with glusterfs

Changed inventory file like below

    [OSEv3:children]
    masters
    nodes
    etcd
    glusterfs

    [OSEv3:vars]
    ansible_ssh_common_args='-o StrictHostKeyChecking=no'
    ansible_ssh_user=root
    openshift_deployment_type=origin
    openshift_enable_origin_repo=false
    openshift_disable_check=disk_availability,memory_availability

    os_firewall_use_firewalld=true

    openshift_public_hostname=console.1.1.0.1.nip.io
    openshift_master_default_subdomain=apps.1.1.0.1.nip.io

    openshift_storage_glusterfs_is_native=false
    openshift_storage_glusterfs_storageclass=true
    openshift_storage_glusterfs_heketi_is_native=true
    openshift_storage_glusterfs_heketi_executor=ssh
    openshift_storage_glusterfs_heketi_ssh_port=22
    openshift_storage_glusterfs_heketi_ssh_user=root
    openshift_storage_glusterfs_heketi_ssh_sudo=false
    openshift_storage_glusterfs_heketi_ssh_keyfile="/root/.ssh/id_rsa

    [masters]
    1.1.0.1 openshift_ip=1.1.0.1 openshift_schedulable=true

    [etcd]
    1.1.0.1  openshift_ip=1.1.0.1    

    [nodes]
    1.1.0.1  openshift_ip=1.1.0.1  openshift_node_group_name="node-config-all-in-one"  openshift_schedulable=true

    [glusterfs]
    1.1.0.1  glusterfs_devices='[ "/dev/vdb" ]'

Now we have to hack ansible script as it expect three nodes by adding --durability none in following ansible script

openshift-ansible/roles/openshift_storage_glusterfs/tasks/heketi_init_db.yml

Following is updated snippet

- name: Create heketi DB volume
  command: "{{ glusterfs_heketi_client }} setup-openshift-heketi-storage --image {{ glusterfs_heketi_image }} --listfile /tmp/heketi-storage.json --durability none"
  register: setup_storage

As by default it create StorageClass which expect replicate environment, so we have to create custom storageclass like below with "volumetype: none"

oc create -f - <<EOT
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: glusterfs-nr-storage
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
parameters:
  resturl: http://heketi-storage.glusterfs.svc:8080
  restuser: admin
  secretName: heketi-storage-admin-secret
  secretNamespace: glusterfs
  volumetype: none
provisioner: kubernetes.io/glusterfs
volumeBindingMode: Immediate
EOT

Now you can create storage dynamically from webconsole :) Any suggestions for improvement are welcome.

Next i will check how i can extend it?

How to deploy OpenShift Origin 10 (OKD) on one node with GlusterFS

1 Answers