all posts tagged volumes


by on June 14, 2016

GlusterFS Snapshots And Their Prerequisites

Long time, no see huh!!! This post has been pending on my part for a while now, partly because I was busy and partly because I am that lazy. But it's a fairly important post as it talks about snapshotting the GlusterFS volumes. So what are these snapshots and why are they so darn important. Let's find out...

Wikipedia says, 'a snapshot is the state of a system at a particular point in time'. In filesystems specifically, a snapshot is a 'backup' (a read only copy of the data set frozen at a point in time). Obviously, it's not a full backup of the entire dataset, but it's a backup nonetheless, which makes it pretty important. Now moving on to GlusterFS snapshots. GlusterFS snapshots, are point-in-time, read-only, crash consistent copies, of GlusterFS volumes. They are also online snapshots, and hence the volume and it's data continue to be available to the clients, while the snapshot is being taken.

GlusterFS snapshots are thinly provisioned LVM based snapshots, and hence they have certain pre-requisites. A quick look at the product documentation tells us what those pre-requisites. For a GlusterFS volume, to be able to support snapshots, it needs to meet the following pre-requisites:
  • Each brick of the GlusterFS volume, should be on an independent, thinly-provisioned LVM.
  • A brick's lvm should not contain any data other than the brick's.
  • None of the bricks should be on a thick LVM
  • gluster version should be 3.6 and above (duh!!)
  • The volume should be started.
  • All brick processes must be up and running.

Now that I have laid out the rules above, let me give you their origin story as well. As in, how do the GlusterFS snapshots internally enable you to take a crash-consistent backup using thinly-provisioned LVM in a space efficient manner. We start by having a look at a GlusterFS volume, whose bricks are on independent, thinly-provisioned LVMs.


In the above diagram, we can see that GlusterFS volume test_vol comprises of two bricks, Brick1 and Brick2. Both the bricks are mounted on independent, thinly-provisioned LVMs. When the volume is mounted, the client process maintains a connection to both the bricks. This is as much summary of GlusterFS volumes, as is needed for this post. A GlusterFS snapshot, is also internally a GlusterFS volume with the exception that, it is a read-only volume and it is treated differently than a regular volume in certain aspects.

When we take a snapshot (say snap1) of the GlusterFS volume test_vol, following set of things happen in the background:
  •  It is checked if the volume is in started state, and if so are all the brick processes up and running.
  • At this point in time, we barrier certain fops, in order to make the snapshots crash-consistent. What it means is even though it is an online snapshot, certain write fops will be barriered for the duration of the snapshot. The fops that are on the fly when the barrier is initiated will be allowed to complete, but the acknowledgement to the client will be pending till the snapshot creation is complete. The barriering has a default time-out window of 2 mins, within which if the snapshot is not complete, the fops are unbarriered, and we fail that particular snapshot.
  • After successfully barriering fops on all brick processes, we proceed to take individual copy-on-write LVM snapshots of each brick. A copy-on-write snapshot LVM snapshot ensures a fast, space-efficient backup of the data currently on the brick. These LVM snapshots reside in the same LVM thinpool as the GLusterFS brick LVMs.
  • Once this snapshot is taken, we carve bricks out of these LVM snapshots, and create a snapshot volume out of those bricks.
  • Once the snapshot creation is complete, we unbarrier the GlusterFS volume.

As can be seen in the above diagram, the snapshot creation process has created a LVM snapshot for each LVM, and these snapshots lie in the same thinpool as the LVM. Then we carve bricks (Brick1" and Brick2") out of these snapshots, and create a snapshot volume called snap1.

This snapshot, snap1 is a read-only snapshot volume which can be:
  • Restored to the original  volume test_vol.
  • Mounted as a read-only volume and accessed.
  • Cloned to create a writeable snapshot.
  • Can be accessed via User-Servicable-Snapshots.
All these functionalities will be discussed in future posts, starting with the command line tools to create, delete and restore GlusterFS snapshots.
by on January 31, 2013

Volume Files and A Sneak Peak At Translators

In my last post, we went through the three vanilla types of volumes : Distribute, Replicate and Stripe, and now we have a basic understanding of what each of them does. I also mentioned we would be creating volumes, which are a mix of these three types. But before we do so, let's have a look at volume files.

Whenever a volume is created, subsequent volume files are also created. Ideally volume files are located in a directory(bearing the same name as the volume name) inside /var/lib/glusterd/vols/. Let's have a look at the volume files, for the distribute volume(test-vol) we created last time.
# gluster volume info

Volume Name: test-vol
Type: Distribute
Volume ID: 5d28ca28-9363-4b79-b922-5f28d0c0db65
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: Gotham:/home/asengupt/node1
Brick2: Gotham:/home/asengupt/node2

# cd /var/lib/glusterd/vols/
# ls -lrt
total 4
drwxr-xr-x. 4 root root 4096 Jan 31 14:20 test-vol
# cd test-vol/
# ls -lrt
total 32
-rw-------. 1 root root 1406 Jan 31 14:19 test-vol.Gotham.home-asengupt-node1.vol
-rw-------. 1 root root 1406 Jan 31 14:19 test-vol.Gotham.home-asengupt-node2.vol
-rw-------. 1 root root 1349 Jan 31 14:19 trusted-test-vol-fuse.vol
-rw-------. 1 root root 1121 Jan 31 14:20 test-vol-fuse.vol
drwxr-xr-x. 2 root root   80 Jan 31 14:20 run
-rw-------. 1 root root  305 Jan 31 14:20 info
drwxr-xr-x. 2 root root   74 Jan 31 14:20 bricks
-rw-------. 1 root root   12 Jan 31 14:20 rbstate
-rw-------. 1 root root   34 Jan 31 14:20 node_state.info
-rw-------. 1 root root   16 Jan 31 14:20 cksum
#
As I have the bricks in the same machine as the mount, hence we are seeing all the volume files here. The test-vol.Gotham.home-asengupt-node1.vol and test-vol.Gotham.home-asengupt-node2.vol are the volume files for Brick1 and Brick2 respectively. The volume file for test-vol volume is trusted-test-vol-fuse.vol. Let's have a look inside :
# cat trusted-test-vol-fuse.vol
volume test-vol-client-0
    type protocol/client
    option password 010f5d80-9d99-4b7c-a39e-1f964764213e
    option username 6969e53a-438a-4b92-a113-de5e5b7b5464
    option transport-type tcp
    option remote-subvolume /home/asengupt/node1
    option remote-host Gotham
end-volume

volume test-vol-client-1
    type protocol/client
    option password 010f5d80-9d99-4b7c-a39e-1f964764213e
    option username 6969e53a-438a-4b92-a113-de5e5b7b5464
    option transport-type tcp
    option remote-subvolume /home/asengupt/node2
    option remote-host Gotham
end-volume

volume test-vol-dht
    type cluster/distribute
    subvolumes test-vol-client-0 test-vol-client-1
end-volume

volume test-vol-write-behind
    type performance/write-behind
    subvolumes test-vol-dht
end-volume

volume test-vol-read-ahead
    type performance/read-ahead
    subvolumes test-vol-write-behind
end-volume

volume test-vol-io-cache
    type performance/io-cache
    subvolumes test-vol-read-ahead
end-volume

volume test-vol-quick-read
    type performance/quick-read
    subvolumes test-vol-io-cache
end-volume

volume test-vol-md-cache
    type performance/md-cache
    subvolumes test-vol-quick-read
end-volume

volume test-vol
    type debug/io-stats
    option count-fop-hits off
    option latency-measurement off
    subvolumes test-vol-md-cache
end-volume
#
This is what a volume file looks like. It's actually an inverted graph, of the path that the data from the mount point is supposed to follow. Savvy? No? Ok, Let's have a closer look at it. It is made up of a number of sections, each of which begin with "volume test-vol-xxxxx" and end with "end-volume". Each section stores information (type, options, subvolumes etc) for the respective translators(we will come back to what they are in a minute).

For example : Let's say a read fop(file-operation) was attempted at the mount point. The request information (type of fop:read, filename:the file the user tried to read etc.) will be passed on from one translator to another, starting from the io-stats translator at the bottom to either of the client translators at the top. Similarly the response will be transferred back from the client translator, all the way down to the io-stats and finally to the user.

So what are these translators. A translator is a module, which has a very specific purpose. It is to receive data, perform the necessary operations, and pass the data down to the next translator. That's about it in a nutshell. For example let's look at the dht-translator in the above volume file.
volume test-vol-dht
    type cluster/distribute
    subvolumes test-vol-client-0 test-vol-client-1
end-volume
"test-vol" is a distribute type volume, and hence has a dht translator. DHT a cluster translator, as is visible in it's "type". We know that in a distribute volume, a hashing algorithm, decides in which of the "subvolumes" is the data actually present. DHT translator is the one who does that for us.

The dht-translator will receive our read-fop, along with the filename. Based on the filename, the hashing algorithm, finds out the correct subvolume (among the two "test-vol-client-0" and "test-vol-client-1"), and passes the read-fop to the subsequent translator. That is how every other translator in the graph also works(recieve the data, perform it's part of the processing, and pass the data on to the next translator). As is quite visible here, the concept of translators provides us with a lot of modularity.

The volume files are created by the volume-create command, and based on the type of the volume and it's options, a graph is build with the appropriate translators. But we can edit an existing volume-file (add, remove, or modify a couple of translators), and the volume will change behaviour accordingly. Let's try that. Currently "test-vol" is a distribute volume. As per the behaviour, any file that is created, will be present in one of the bricks.
# mount -t glusterfs Gotham:/test-vol /mnt/test-vol-mnt/
# cd /mnt/test-vol-mnt/
# ls -lrt
total 0
# touch file1
# ls -lrt
total 0
-rw-r--r--. 1 root root 0 Jan 31 15:57 file1
# ls -lrt /home/asengupt/node1/
total 0
# ls -lrt /home/asengupt/node2/
total 0
-rw-r--r--. 2 root root 0 Jan 31 15:57 file1
#
The dht-translator, created the file in node2 only. Let's edit the volume-file for test-vol(/var/lib/glusterd/vols/test-vol/trusted-test-vol-fuse.vol). But before that we need to stop the volume and unmount it.
# gluster volume stop test-vol
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: test-vol: success
# umount /mnt/test-vol-mnt 
Then let's edit the volume file, and replace the dht-translator with a replicate translator.
# vi /var/lib/glusterd/vols/test-vol/trusted-test-vol-fuse.vol 
**********Necessary Changes in the volume file************
volume test-vol-afr
    type cluster/replicate
    subvolumes test-vol-client-0 test-vol-client-1
end-volume

volume test-vol-write-behind
    type performance/write-behind
    subvolumes test-vol-afr
end-volume
**********************************************************
Now start the volume, and mount it again.
# gluster volume start test-vol
volume start: test-vol: success
# mount -t glusterfs Gotham:/test-vol /mnt/test-vol-mnt/
Let's create a file at this mount-point and check the behaviour of "test-vol".
# cd /mnt/test-vol-mnt/
# ls -lrt
total 0
-rw-r--r--. 1 root root 0 Jan 31 15:57 file1
# touch file2
# ls -lrt /home/asengupt/node1/
total 4
-rw-r--r--. 2 root root 0 Jan 31 16:06 file1
-rw-r--r--. 2 root root 0 Jan 31 16:06 file2
# ls -lrt /home/asengupt/node2/
total 4
-rw-r--r--. 2 root root 0 Jan 31 16:06 file1
-rw-r--r--. 2 root root 0 Jan 31 16:06 file2
#
Now we have, the same set of files in all the bricks, as is the behaviour of a replicate volume. We also observe that not only did it create the new file(file2) in all the bricks, but when we re-started the volume after changing the volume file, it also created a copy of the existing file(file1) in all the bricks. Let's check the volume info.
# gluster volume info 
  
Volume Name: test-vol 
Type: Distribute 
Volume ID: 5d28ca28-9363-4b79-b922-5f28d0c0db65 
Status: Started 
Number of Bricks: 2 
Transport-type: tcp 
Bricks: 
Brick1: Gotham:/home/asengupt/node1 
Brick2: Gotham:/home/asengupt/node2
It should be noted that, because of the changes we made in the volume file, though the behaviour of the volume changes, the volume-info still reflects the original details of the volume.
As promised we will create a mix of the vanilla volume-types, i.e a distributed-replicate volume.
# gluster volume create mix-vol replica 2 Gotham:/home/asengupt/node1 Gotham:/home/asengupt/node2 Gotham:/home/asengupt/node3 Gotham:/home/asengupt/node4
Multiple bricks of a replicate volume are present on the same server. This setup is not optimal.
Do you still want to continue creating the volume?  (y/n) y
volume create: mix-vol: success: please start the volume to access data
# gluster volume start mix-vol;
volume start: mix-vol: success
# gluster volume info

Volume Name: mix-vol
Type: Distributed-Replicate
Volume ID: 2fc6f11e-254e-444a-8179-43da62cc56e9
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: Gotham:/home/asengupt/node1
Brick2: Gotham:/home/asengupt/node2
Brick3: Gotham:/home/asengupt/node3
Brick4: Gotham:/home/asengupt/node4
#
As it is a distributed-replicate volume, the distribute translator should have two sub-volumes, each of which in turn is a replicate translator, having two bricks each as it's sub-volumes. A quick look at the volume file(/var/lib/glusterd/vols/mix-vol/trusted-mix-vol-fuse.vol) will give us more clarity. This is what the cluster translators in the volume file will look like :
volume mix-vol-replicate-0
    type cluster/replicate
    subvolumes mix-vol-client-0 mix-vol-client-1
end-volume

volume mix-vol-replicate-1
    type cluster/replicate
    subvolumes mix-vol-client-2 mix-vol-client-3
end-volume

volume mix-vol-dht
    type cluster/distribute
    subvolumes mix-vol-replicate-0 mix-vol-replicate-1
end-volume
As we can see the dht-translator has two replicate translators as its sub-volumes : "mix-vol-replicate-0" and "mix-vol-replicate-1". So every file created at the mount-point will be sent to either one of the replicate sub-volumes, by the dht-translator. Each replicate-subvolume has two bricks as it's own sub-volumes. After the write-fop is sent to the appropriate replicate-subvolume, the replicate translator will create a copy each, in both the bricks listed as it's sub-volumes. Let's check this behaviour :
# mount -t glusterfs Gotham:/mix-vol /mnt/test-vol-mnt/
# cd /mnt/test-vol-mnt/
# touch file1
# ls -lrt /home/asengupt/node1/
total 0
# ls -lrt /home/asengupt/node2/
total 0
# ls -lrt /home/asengupt/node3/
total 0
-rw-r--r--. 2 root root 0 Jan 31 16:31 file1
# ls -lrt /home/asengupt/node4/
total 0
-rw-r--r--. 2 root root 0 Jan 31 16:31 file1
#
Similarly, a distributed-stripe or a replicated-stripe volume can also be created.  Jeff Darcy's blog has an awesome set of articles on translators. It's  a great read :
EDIT : The above links are broken, but the same information is provided in glusterfs/doc/developer-guide/translator-development.md in the source code. Thanks to Jo for pointing out the same.
    by on January 11, 2013

    Volumes

    I feel it's safe to say, that we now have a fair idea of what GlusterFS is, and we are pretty comfortable installing GlusterFS, and creating a volume.
    Let's create a volume with two local directories as two bricks.
    # gluster volume create test-vol Gotham:/home/asengupt/node1 Gotham:/home/asengupt/node2
    volume create: test-vol: success: please start the volume to access data
    # gluster volume start test-vol;
    volume start: test-vol: success
    Let's mount this volume, and create a file in that volume.
    # mount -t glusterfs Gotham:/test-vol /mnt/test-vol-mnt/
    # touch /mnt/test-vol-mnt/file1
    # cd /mnt/test-vol-mnt/
    # ls -lrt
    total 1
    -rw-r--r--. 1 root root   0 Jan 10 14:40 file1
    Now where does this file really get created in the backend. Let's have a look at the two directories we used as bricks(subvolumes):
    # cd /home/asengupt/node1
    # ls -lrt
    total 0
    # cd ../node2/
    # ls -lrt
    total 1
    -rw-r--r--. 1 root root   0 Jan 10 14:40 file1
    So the file we created at the mount-point(/mnt/test-vol-mnt), got created in one of the bricks. But why in this particular brick, why not the other one? The answer to that question lies in the volume information.
    # gluster volume info
     
    Volume Name: test-vol
    Type: Distribute
    Volume ID: 5d28ca28-9363-4b79-b922-5f28d0c0db65
    Status: Started
    Number of Bricks: 2
    Transport-type: tcp
    Bricks:
    Brick1: Gotham:/home/asengupt/node1
    Brick2: Gotham:/home/asengupt/node2
    It gives us a lot of info. While creating a volume we have the liberty of providing a number of options like the transport-type, the volume-type, etc. which eventually decides the behaviour of the volume. But at this moment what most interests us is the type. It says that our volume "test-vol" is a distributed volume. What does that mean?

    The type of a volume decides, how exactly the volume stores the data in the bricks. A volume can be of the following types :
    • Distribute : A distribute volume is one, in which all the data of the volume, is distributed throughout the bricks. Based on an algorithm, that takes into account the size available in each brick, the data will be stored in any one of the available bricks. As our "test-vol" volume is a distributed volume, so based on the algorithm "file1" was created in node2. The default volume type is distribute, hence test-vol is distribute.
    • Replicate : In a replicate volume, the data is replicated(duplicated) over every brick, based on the brick number. The number of bricks must be a multiple of the replica count. So when "file1" is created in a replicate volume, having two bricks, it will be stored in brick1, and then replicated to brick2. So the file will be present in both the bricks. Let's create one and see for ourselves.
      # gluster volume create test-vol replica 2 Gotham:/home/asengupt/node1 Gotham:/home/asengupt/node2
      Multiple bricks of a replicate volume are present on the same server. This setup is not optimal.
      Do you still want to continue creating the volume?  (y/n) y
      volume create: test-vol: success: please start the volume to access data
      # gluster volume start test-vol
      volume start: test-vol: success
      # gluster volume info
       
      Volume Name: test-vol
      Type: Replicate
      Volume ID: bfb685e9-d30d-484c-beaf-e5fd3b6e66c7
      Status: Started
      Number of Bricks: 1 x 2 = 2
      Transport-type: tcp
      Bricks:
      Brick1: Gotham:/home/asengupt/node1
      Brick2: Gotham:/home/asengupt/node2
      # mount -t glusterfs Gotham:/test-vol /mnt/test-vol-mnt/
      # touch /mnt/test-vol-mnt/file1
      # cd /mnt/test-vol-mnt/
      # ls -lrt
      total 0
      -rw-r--r--. 1 root root 0 Jan 10 14:58 file1
      # ls -lrt /home/asengupt/node1/
      total 0
      -rw-r--r--. 2 root root 0 Jan 10 14:58 file1
      # ls -lrt /home/asengupt/node2/
      total 0
      -rw-r--r--. 2 root root 0 Jan 10 14:58 file1
    • Stripe : A stripe volume is one, in which the data being stored in the backend is striped into units of a particular size, among the bricks. The default unit size is 128KB, but it's configurable. If we create a striped volume of stripe count 3, and then create a 300 KB file at the mount point, the first 128KB will be stored in the first sub-volume(brick in our case), the next 128KB in the second, and the remaining 56KB in the third. The number of bricks should be a multiple of the stripe count.
      # gluster volume create test-vol stripe 3 Gotham:/home/asengupt/node1 Gotham:/home/asengupt/node2 Gotham:/home/asengupt/node3
      volume create: test-vol: success: please start the volume to access data
      # gluster volume start test-vol
      volume start: test-vol: success
      # gluster volume info
       
      Volume Name: test-vol
      Type: Stripe
      Volume ID: c5aa1590-2f6e-464d-a783-cd9bc222db30
      Status: Started
      Number of Bricks: 1 x 3 = 3
      Transport-type: tcp
      Bricks:
      Brick1: Gotham:/home/asengupt/node1
      Brick2: Gotham:/home/asengupt/node2
      Brick3: Gotham:/home/asengupt/node3
      # mount -t glusterfs Gotham:/test-vol /mnt/test-vol-mnt/
      # cd /mnt/test-vol-mnt/
      # ls -lrt
      total 0
      # cp /home/asengupt/300KB_File .
      # ls -lrt
      total 308
      -rwxr-xr-x. 1 root root 307200 Jan 11 12:46 300KB_File
      # ls -lrt /home/asengupt/node1/
      total 132
      -rwxr-xr-x. 2 root root 131072 Jan 11 12:46 300KB_File
      # ls -lrt /home/asengupt/node2/
      total 132
      -rwxr-xr-x. 2 root root 262144 Jan 11 12:46 300KB_File
      # ls -lrt /home/asengupt/node3/
      total 48
      -rwxr-xr-x. 2 root root 307200 Jan 11 12:46 300KB_File
      Why do we see that the first sub-volume indeed has 128kb of data, but the second and third sub-volumes contain 256KB, and 300KB respectively?
      That's because of holes. It means that the filesystem just pretends that at a particular place in the file there is just zero bytes, but no actual disk sectors are reserved for that place in the file. To proof this let's check the disk usage.
      # cd /home/asengupt
      # du | grep node.$
      136    ./node2
      136    ./node1
      52    ./node3
      0    ./node4
      Here we observe, that node1 and node2 indeed have 128KB of data, while node3 has 44KB. The additional 8KB present in these directories, are glusterfs system files.

    Apart from these three vanilla types of volume, we can also create a volume which is a mix of these types. We will go through these and the respective volume files in the next post.
    by on December 13, 2012

    GlusterFS – The Product


    I can only think of three reasons why you would be reading this :
    • You are a developer, interested in contributing to GlusterFS : If this is where you belong, you are in the right place. View this space, not as a guide, but as a peer's journal, and familiarize yourself with GlusterFS as I too continue to do the same.
    • You are an existing contributor to GlusterFS, and you are still reading this, wondering what am I rambling about : If you are one of the elites, please keep in mind that I too am in the process of understanding the product right now, and the purpose of this communication is only to pen down my understanding of the product as I continue to work on it. So while I am at it, I am sure there will be times when the information I put down here, might be inadequate. In such cases, I request you to kindly provide comments below, and help make this forum more valuable.
    • I have become so incredibly famous that you googled me, and landed up here : In case that this is true, cheers to me :p, If not, I guess it's pretty safe to assume, that you belong to either of the above two groups.
    Before one starts to work on and contribute to a product, what's most essential is a fundamental overview of the product. So in this post, we will go through a very brief introduction of GlusterFS, followed by a hands-on of pulling in the source code, building and installing the product, and finally creating a volume.

    So what is GlusterFS? Well, when we Google GlusterFS, this is what we find..
    Wikipedia says "GlusterFS is a scale-out NAS file system.". The gluster-community says "GlusterFS is a powerful network/cluster file-system written in user space which uses FUSE to hook itself with VFS layer. GlusterFS takes a layered approach to the file system, where features are added/removed as per the requirement. Though GlusterFS is a File System, it uses already tried and tested disk file systems like ext3, ext4, xfs, etc. to store the data. It can easily scale up to petabytes of storage which is available to user under a single mount point."

    Kudos to you if you understood all of it. If not let's start from the start. In the simplest terms we can say GlusterFS is a file system. So when you hear file system, you start thinking ext3, ext4, xfs right? Well they are disk file systems, which basically manage every file-system related operation(fops) in their own approach, and each of them have evolved over the years, to become quite stable and mature. But GlusterFS is not a disk file system. Instead of re-inventing the wheel, GlusterFS uses any of these disk files systems(xfs is recommended), as the back-end file system to perform all file related operations, while it itself sits on top, and that too in userspace. So the obvious question that's haunting you now, is what does it do at the top, that these fellows at the back-end are not doing. Well the answer lies in the one line that Wikipedia gave us : scale-out

    What does scale-out really mean? Traditional storage systems, are single storage controller systems, which are connected to ideally a rack of disk drives. The processing power here will be fixed, and cannot be scaled as the storage capacity increases. A scale-out storage system, can consist of multiple modules. In this architecture, each module in the system, has processing as well as storage capacity, which also means, as the system scales, so does it's processing power. GlusterFS is scalable to petabytes and beyond, and provides a clustered scale-out solution.

    Every module in GlusterFS is treated as a volume. A volume can consist of several bricks. A brick is a node or storage file-system, assigned to a volume. Now that you have a picture of bricks and volumes(doesn't matter even if it's a vague picture), try creating a volume of your own. Please go through this link to understand the developer's work-flow. If you are not familiar to git, you can have a look at this.

    Now that you have gone through the developer's work-flow, registered yourself at Gerrit, and added your machine's public key to Gerrit, let's pull in the source code :
    git clone ssh://[username@]git.gluster.org/glusterfs.git ~/Source/
    Now you have a directory called Source in your home directory, which contains the source code. Before we begin to build the source code, please ensure you have the following libraries installed :
    yum install libtool autoconf automake flex bison openssl openssl-devel libibverbs-devel readline-devel libxml2-devel libacl-devel sqlite-devel python-devel userspace-rcu-devel dbench nfs-utils yajl attr psmisc
    Once these libraries are installed, you can build GlusterFS.
    # ./autogen.sh
    # ./configure
    # make
    # make install
    To verify which version of GlusterFS is installed
    # gluster --version
    Congratulations you have now successfully built GlusterFS from the source code and installed it. Let's create a volume. Please do make sure, you have a hostname assigned to your machine. This avoids any hassles from dynamic IPs. For the moment, you don't need actual servers to create a volume.
    Create two directories node1 and node2 in your home directory to be used as bricks. The command to create a new volume is
    gluster volume create NEW-VOLNAME [stripe COUNT] [replica COUNT] [transport <tcp | rdma>] NEW-BRICK ...
    For example :
    # gluster volume create test-vol Hostname:/~/node1 Hostname:/~/node2
    Start the volume:
    # gluster volume start test-vol
    To display the volume information :
    # gluster volume info
    In Association with Amazon.in
    by on March 27, 2011

    Finding Gluster volumes from a client machine

    One of the questions that I come across in IRC and other places often is how to obtain a list of gluster volumes that can be mounted from a client machine. NFS provides showmount which helps in figuring out the list of exports from a server amongst other things. GlusterFS currently does not have an equivalent of showmount and this is one of the nifty utilities that we intend to develop in the near term.

    Right now, there is a way of figuring out the list of volumes on a client machine. Gluster 3.1 exports volumes through NFS by default. Owing to this, showmount itself can be leveraged to figure out the volumes that are exported by Gluster servers. The output of  ‘showmount -e <server>‘ will be the list of volumes that can be mounted from the client. For e.g.

    # showmount -e deepthought
    Export list for deepthought:
    /dr    *
    /music *

    If you just need a list of volumes, you can clear the extra output too by making use of your favorite filters. Something like:

    #showmount -e  <server> | grep /  | tr -d ‘/’ | cut -d ‘ ‘ -f1

    would yield you the list.

    Once you have the list on your client, you can mount it through NFS and/or GlusterFS protocols, based on your choice of access protocol.