Possible configurations of GlusterFS in Kubernetes/OpenShift setup

In previous blog posts we discussed, how to use GlusterFS as a persistent storage in Kubernetes and Openshift. In nutshell, the GlusterFS can be deployed/used in a kubernetes/openshift environment as :

*) Contenarized GlusterFS ( Pod )
*) GlusterFS as Openshift service and Endpoint (Service and Endpoint).
*) GlusterFS volume as Persistent Volume (PV) and using GlusterFS volume plugin
to bind this PV to a Persistent Volume Claim ( PVC)
*) GlusterFS template to deploy GlusterFS pods in an Openshift Environment.



All the configuration files that can be used to deploy GlusterFS can be found @ github.com/humblec/glusterfs-kubernetes-openshift/ or github.com/gluster/glusterfs-kubernetes-openshift. Lets see how to use these files to deploy GlusterFS in kubernetes and Openshift.

We will start with Deploying GlusterFS pods in an Openshift/Kubernetes Environment.

Deploying GlusterFS Pod:

[Update] The pod file is renamed to gluster-pod.yaml in the mentioned repo. More details about Gluster Containers can be found @http://www.slideshare.net/HumbleChirammal/gluster-containers

GlusterFS pods can be deployed in Kubernetes/Openshift, so that Gluster Nodes are deployed in containers and it can provide persistent storage for Openshift/Kubernetes setup.

The examples files in this repo are used for this demo.

Step 1: Create GlusterFS pod

[root@atomic-node2 gluster_pod]# oc create -f gluster-1.yaml

Step 2: Get details about the GlusterFS pod.


[root@atomic-node2 gluster_pod]# oc describe pod gluster-1
Name: gluster-1
Namespace: default
Image(s): gluster/gluster-centos
Node: atomic-node1/10.70.43.174
Start Time: Tue, 17 May 2016 10:19:17 +0530
Labels: name=gluster-1
Status: Running
Reason:
Message:
IP: 10.70.43.174
Replication Controllers:
Containers:
glusterfs:
Container ID: docker://ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742
Image: gluster/gluster-centos
Image ID: docker://033de9c44a8aabde55ce8a2b751ccf5bc345fdb534ea30e79a8fa70b82dc7761
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Running
Started: Tue, 17 May 2016 10:20:35 +0530
Ready: True
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Ready True
Volumes:
brickpath:
Type: HostPath (bare host directory volume)
Path: /mnt/brick1
default-token-72d89:
Type: Secret (a secret that should populate this volume)
SecretName: default-token-72d89
Events:
FirstSeen LastSeen Count From SubobjectPath Reason Message
───────── ──────── ───── ──── ───────────── ────── ───────
1m 1m 1 {scheduler } Scheduled Successfully assigned gluster-1 to atomic-node1
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Pulled Container image "openshift3/ose-pod:v3.1.1.6" already present on machine
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Created Created with docker id f55ce55e6ea3
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Started Started with docker id f55ce55e6ea3
1m 1m 1 {kubelet atomic-node1} spec.containers{glusterfs} Pulling pulling image "gluster/gluster-centos"
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Pulled Successfully pulled image "gluster/gluster-centos"
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Created Created with docker id ff8f4af700d7
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Started Started with docker id ff8f4af700d7


From above logs, you can see it pulled `gluster/gluster-centos` container image and deployed containers from it.


[root@atomic-node2 gluster_pod]# oc get pods
NAME READY STATUS RESTARTS AGE
gluster-1 1/1 Running 0 1m


Examine the container and make sure it has a running GlusterFS daemon.


[root@atomic-node2 gluster_pod]# oc exec -ti gluster-1 /bin/bash


Examine the processes running in this container and the `glusterd` service information.


[root@atomic-node1 /]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.4 0.0 40780 2920 ? Ss 04:50 0:00 /usr/sbin/init
root 20 0.3 0.0 36816 4272 ? Ss 04:50 0:00 /usr/lib/syste
root 21 0.0 0.0 118476 1332 ? Ss 04:50 0:00 /usr/sbin/lvme
root 37 0.0 0.0 101344 1228 ? Ssl 04:50 0:00 /usr/sbin/gssp
rpc 44 0.1 0.0 64904 1052 ? Ss 04:50 0:00 /sbin/rpcbind
root 209 0.1 0.1 364716 13444 ? Ssl 04:50 0:00 /usr/sbin/glus
root 341 1.1 0.0 13368 1964 ? Ss 04:51 0:00 /bin/bash
root 354 0.0 0.0 49020 1820 ? R+ 04:51 0:00 ps aux
[root@atomic-node1 /]# service glusterd status
Redirecting to /bin/systemctl status glusterd.service
● glusterd.service - GlusterFS, a clustered file-system server
Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2016-05-17 04:50:41 UTC; 35s ago
Process: 208 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 209 (glusterd)
CGroup: /system.slice/docker-ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742.scope/system.slice/glusterd.service
└─209 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO...
‣ 209 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO...
May 17 04:50:36 atomic-node1 systemd[1]: Starting Gluste...
May 17 04:50:41 atomic-node1 systemd[1]: Started Gluster...
Hint: Some lines were ellipsized, use -l to show in full.


Let’s fetch some more details about GlusterFS in this container.


[root@atomic-node1 /]# gluster --version
glusterfs 3.7.9 built on Mar 20 2016 03:19:49
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@atomic-node1 /]#
[root@atomic-node1 /]# mount |grep mnt
/dev/mapper/atomic-node1-root on /mnt/brick1 type xfs (rw,relatime,seclabel,attr2,inode64,noquota)


This container is built on top of CentOS base image as shown below.


[root@atomic-node1 /]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
[root@atomic-node1 /]#


In this article we discussed, how to run GlusterFS as a pod in Kubernetes or Openshift setup.

[Part 2] covers `how to use GlusterFS as a service, Persistent Volume for a Persistent Volume Claim`.

[Part 3] covers `how to use GlusterFS template to deploy GlusterFS pods in an Openshift/kubernetes setup`.