all posts by

by on February 28, 2017

Further notes on Gluster 3.10 and the direction for Gluster

This release of Gluster ushers in improvements for container storage, hyperconverged storage and scale-out Network Attached Storage (NAS) use cases. These use cases have been the primary focus areas for previous releases over the last 12-18 months and will continue to be the primary focus for the next three planned releases.

One of the things we’re really focused on as a project is persistent storage for containerized microservices. Part of this effort has been working with heketi and gluster-kubernetes to enhance our integration with containers. Continuing in the same vein, 3.10 brings about the following key improvements for container storage:


  • Brick multiplexing: Provides the ability to scale the number of exports and volumes per node. This is useful in container storage where there is a need for a large number of shared storage (Read Write Many) volumes. Brick Multiplexing also provides the infrastructure needed to implement Quality of Service in Gluster for a multi-tenant container deployment.
  • gluster-block: Along with 3.10, we are also releasing gluster-block v0.1.  gluster-block provides a very intuitive lifecycle management interface for block devices in a Gluster volume. This release of gluster-block configures block devices to be accessed from initiators through iSCSI.  Work on integrating gluster-block with Heketi for supporting Read Write Once volumes in Kubernetes is in progress.
  • S3 access for Gluster: We are also releasing an Amazon S3 compatible object storage container based on Swift3 and gluster-swift in Gluster’s docker hub. S3 access for Gluster will be useful for application developers who leverage S3 API for storage.


Deployment of hyperconverged storage for containers and virtualization is also a focus area for 3.10. gdeploy provides an improved ansible playbook for deploying hyperconvergence with oVirt and cockpit-gluster provides a wizard to make deployment using this playbook easy with oVirt. gk-deploy makes it easy to deploy Heketi and Gluster in hyperconverged container deployments.


There have been various improvements for scale-out NAS deployments in 3.10. Some of them include:


  • Parallelized readdirp: Improves performance of operations that perform directory crawl. This in conjunction with meta-data caching introduced in 3.9 provides a nice performance boost for small file operations.
  • Statedump support for libgfapi:  Debuggability and supportability improvement for projects like NFS Ganesha and Samba that have been integrated with libgfapi.
  • Estimate for rebalance completion: Helps administrators understand when rebalancing would complete.


Needless to say, none of this would have been possible without the support of our contributors and maintainers. Thank you to all those who made it happen! We  are excited at this juncture to deliver features that enhance the user experience on our key focus areas of container storage, hyperconvergence & scale-out NAS. We intend building on this momentum for further improvements on our key focus areas. Stay tuned and get involved as we progress along this journey with further Gluster releases!


by on October 2, 2015

Gluster News: September 2015

Since we did not have any weekly Gluster news go out in September, this post tries to capture and summarize action from the entire month of September 2015.

==  General News ==

GlusterFS won yet another Bossie in the open source platforms, infrastructure, management, and orchestration software category. Long time users of the project might remember the first Bossie win in 2011.

GlusterFS had three bug fix releases happen in September. 3.7.4, 3.6.6 and 3.5.6 releases happened over the course of this month.

GlusterFS 3.7.5 is expected to be released in the first week of October.

GlusterFS 3.7.x  – Samba libgfapi support is affected currently. Stay on GlusterFS 3.6.x if you use it.

gdeploy, an ansible based deployment tool for Gluster, was released this month. More details about gdeploy can be found in the announcement email.

Introducing georepsetup – Gluster Geo-replication  Setup Tool.

If you are interested in Containers & Gluster, learn more about Running GlusterFS inside Kubernetes.

== Technical News == design discussions happened virtually in the week of 09/28. Discussions happened around these topics:

Several new features have been proposed for Gluster 3.8 and can be found here. A release planning page on for 3.8 is expected in the next few weeks.

Getting Started with Code contributions to GlusterFS –

== Community News ==

Introduction of Amye Scavarda – new community lead for Gluster.

GlusterFS Silicon Valley Meetup group had a meetup at Facebook Campus in Menlo Park, CA. More details about the meetup can be found here.

GlusterFS India Community had a meetup in Bangalore on September 12.

Soumya Koduri & Poornima Gurusiddaiah presented about the following topics at SNIA Software Developers Conference event in Santa Clara, CA.

Achieving Coherent and Aggressive Client Caching in Gluster, a Distributed System
Introduction to Highly Available NFS Server on Scale-Out Storage Systems Based on GlusterFS

Niels de Vos gave a presentation and demo about writing backups with Bareos to Gluster at the Open Source Backup Conference.

Several talks related to Gluster are planned at the upcoming LinuxCon EU Conference in Dublin, Ireland: NFS-Ganesha and Clustered NAS on Distributed Storage Systems – Soumya Koduri, Meghana Madhusudhan Advancements in Automatic File Replication in Gluster – Ravishankar N BitRot Detection in GlusterFS – Gaurav Garg, Venky Shankar Open Storage in the Enterprise with Gluster and Ceph – Dustin Black

We plan to have bi-weekly updates from the next edition. Do stay tuned in to learn about happenings in the Gluster World!






by on January 30, 2015

GlusterFS events over the next 10 days

There is plenty of action planned out for GlusterFS in the period between Jan 30th to Feb 8th 2015. Our community enthusiasts will be attending and presenting at various conferences and meetups across the globe. Here is a breakdown of the planned action:

  1. GlusterFS QuickStart Tutorial by Lalatendu Mohanty at the CentOS Dojo in Brussels, Belgium on Jan 30th.
  2. Talk on GlusterFS Overview and Future Directions by Niels de Vos as part of FOSDEM’s IaaS Devroom in Brussels, Belgium on Jan 31st.
  3. Talk on oVirt and Gluster Hyperconvergence by Federico Simoncelli as part of FOSDEM’s IaaS Devroom on Jan 31st.
  4. Lalatendu Mohanty, Niels de Vos, and Kaleb Keithley will be holding GlusterFS Office Hours at FOSDEM in the CentOS Booth on Jan 31st.
  5. Kaleb Keithley and Lalatendu Mohanty intend talking about The CentOS Storage SIG and GlusterFS as part of FOSDEM’s Distributions Devroom on Feb 1st.
  6. Vijay Bellur will be talking about GlusterFS – Architecture & Roadmap at in Brno, Czech Republic on Feb 6th.
  7. James Shubin will be showcasing Automatically deploying GlusterFS with Puppet-Gluster + Vagrant at on Feb 7th.
  8. Following talks are scheduled as part of GlusterFS Meetup in Pune, India on Feb 7th:
    • GlusterFS & Roadmap by Atin Mukherjee
    • GlusterFS and Ovirt by Ramesh Nachimuthu
    • GlusterFS in Openstack by Prasanth Pai
    • GlusterFS for storage admins by Vikhyat Umrao
  9. Dan Lambright and Joseph Fernandes will talk about Efficient Data Maintenance in GlusterFS using databases in on Feb 8th.

As you can see, it is a busy few days ahead for the Gluster community. Do drop by to one of these events and connect with other Gluster enthusiasts from all over. If you are not attending any of these events, do watch this space for event reports from everywhere!

by on November 4, 2014

GlusterFS 3.6.0 – Looking Back and Looking Ahead..

GlusterFS 3.6.0 was released over the last weekend. As I mulled over the release cycle during the weekend, several thoughts on both on the technical and project fronts crossed my mind. This post is an attempt to capture some of those thoughts.

On the technical front, several new features and significant number of bug fixes are part of GlusterFS 3.6.0. The release notes contains a comprehensive list of what’s new in this release. My noteworthy list includes:

  •  Volume Snapshots – Distributed lvm thin-pool based snapshots for backing up volumes in a Gluster Trusted Storage Pool.  Apart from providing cluster wide co-ordination to trigger a consistent snapshot, several improvements have been  performed through the GlusterFS stack to make translators more crash consistent. Snapshotting of volumes is tightly coupled with lvm today but one could also enhance the same framework to integrate with a backend storage technology like btrfs that can perform snapshots.
  •  Erasure Coding – Xavier Hernandez from Datalab added support to perform erasure coding of data in a GlusterFS volume across nodes in a Trusted Storage Pool. Erasure Coding requires fewer nodes to provide better redundancy than a n-way replicated volume and can help in reducing the overall deployment cost. We look forward to build on this foundation and deliver more enhancememnts in upcoming releases.
  • Meta translator – This translator provides a /proc like view for examining internal state of translators on the client stack of a GlusterFS volume and certainly looks like an interface that I would be heavily consuming for introspection of  GlusterFS.
  • Automatic File Replication (AFR) v2 – A significant re-factor of the synchronous replication translator, provides granular entry self-healing and reduced resource consumption with entry self-heals.
  • NetBSD, OSX and FreeBSD ports – Lot of fixes on the portability front. The NetBSD port passes most regression tests as of 3.6.0. At this point, none of these ports are ready to be deployed in production. However, with dedicated maintainers for each of these ports, we expect to reach production readiness on these platforms in a future release.

On the project front,  there have been several things of note happening as far as this release cycle goes. Here are my personal highlights from 3.6.0:

  • 3.6.0 marks the first occurrence of two major GlusterFS releases happening in a calendar year.
  • More patches merged in the 6 month cycle leading to 3.6.0 than any other major release. Addition of more maintainers for GlusterFS and revamp of our Jenkins based CI infrastructure seems to be helping here.

Looking ahead, there are already signs of what is coming up in GlusterFS 3.7. Projects like bitrot detection, volume tiering, Trash translator, Export/Netgroup style authentication for gluster NFS are already in progress and in all probability will be part of 3.7.0 along with other features. We have also started early discussions on what we would want to build for releases and more details will be available in the community soon. If you have thoughts on how addition of a new feature can address your problem(s), do let us know and that can help us in planning out the future of GlusterFS.

Overall, it has been a gratifying time for me to be in the Gluster Community. Personally I look forward to build on this feeling of gratification as we build the next versions of GlusterFS to play a significant role in the distributed storage segment.

by on July 28, 2013

Feedback requested: Governance of GlusterFS project

We are in the process of formalizing the governance model of the GlusterFS project. Historically, the governance of the project has been loosely structured. This is an invitation to all of you to participate in this discussion and provide your feedback and suggestions on how we should evolve a formal model. Feedback from this thread will […]

by on July 15, 2013

Upgrading to GlusterFS 3.4.0

Now that GlusterFS 3.4.0 is out, here are some mechanisms to upgrade from earlier installed versions of GlusterFS.

Upgrade from GlusterFS 3.3.x:

GlusterFS 3.4.0 is compatible with 3.3.x (yes, you read it right!). You can upgrade your deployment by following one of the two procedures below.

a) Scheduling a downtime (Recommended)

For this approach, schedule a downtime and prevent all your clients from accessing the servers.

  • Stop all glusterd, glusterfsd and glusterfs processes on your server.
  • Install  GlusterFS 3.4.0
  • Start glusterd.
  • Ensure that all started volumes have processes online in “gluster volume status”.

You would need to repeat these 4 steps on all servers that form your trusted storage pool.

After upgrading the servers, it is recommended to upgrade all client installations to 3.4.0.

b) Rolling upgrades with no downtime

If you have replicated or distributed replicated volumes with bricks placed in the right fashion for redundancy, have no data to be self-healed and feel adventurous, you can perform a rolling upgrade through the following procedure:

  • Stop all glusterd, glusterfs and glusterfsd processes on your server.
  • Install GlusterFS 3.4.0.
  • Start glusterd.
  • Run “gluster volume heal <volname> info” on all volumes and ensure that there is nothing left to be self-healed on every volume. If you have pending data for self-heal, run “gluster volume heal <volname>” and wait for self-heal to complete.
  • Ensure that all started volumes have processes online in “gluster volume status”.

Repeat the above steps on all servers that are part of your trusted storage pool.

Again after upgrading the servers, it is recommended to upgrade all client installations to 3.4.0.

Upgrade from GlusterFS 3.1.x and 3.2.x:

You can follow the procedure listed here and replace 3.3.0 with 3.4.0 in the procedure.

Do report your findings on 3.4.0 in gluster-users, #gluster on Freenode and bugzilla.

Please note that this may not work for all installations & upgrades. If you notice anything amiss and would like to see it covered here, please leave a comment and I will try to incorporate your findings.

by on May 31, 2012

Upgrading to GlusterFS 3.3.0

Now that GlusterFS 3.3.0 is out, here is a quick primer on upgrading from earlier installed versions of GlusterFS. This howto covers upgrades from 3.1.x and 3.2.x versions of GlusterFS.

1) GlusterFS 3.3.0 is not compatible with any earlier released versions. Please make sure that you schedule a downtime before you upgrade.

2) Stop all glusterd, glusterfs and glusterfsd processes running in all your servers and clients.

3) Take a backup of glusterd’s  working directory on all servers – usually glusterd’s working directory is /etc/glusterd.

4) Install GlusterFS 3.3.0 on all your servers and clients. With 3.3.0, the default working directory has been changed to  /var/lib/glusterd. RPM and source installations move all content under /etc/glusterd to /var/lib/glusterd. On all servers, ensure that the directory structure remains consistent with the backup obtained in 3).  New files are created as part of the upgrade process and hence you may see some additional files. However, ensure all files backed up in 3) are present in /var/lib/glusterd.

5) If you have installed from RPM, goto 6).  Else, start glusterd in upgrade mode. glusterd terminates after it performs the necessary steps for upgrade. Re-start glusterd normally after this termination. Essentially this process boils down to:

a) killall glusterd

b) glusterd --xlator-option *.upgrade=on -N

This will re-generate volume files with the new ‘index’ translator which is needed for features like pro-active self heal in 3.3.0.

c) Start glusterd

Ensure that you repeat a), b) and c) on all servers.

6) All your gluster services on the servers should be back online now. Mount your native clients with GlusterFS 3.3.0 to access your volumes.

Enjoy your shiny new 3.3.0 and report your findings in #gluster on Freenode, bugzilla and c.g.o

Please note that this may not work for all installations & upgrades. If you notice anything amiss and would like to see it covered here, please leave a comment and I will try to incorporate your findings.

by on March 27, 2011

Finding Gluster volumes from a client machine

One of the questions that I come across in IRC and other places often is how to obtain a list of gluster volumes that can be mounted from a client machine. NFS provides showmount which helps in figuring out the list of exports from a server amongst other things. GlusterFS currently does not have an equivalent of showmount and this is one of the nifty utilities that we intend to develop in the near term.

Right now, there is a way of figuring out the list of volumes on a client machine. Gluster 3.1 exports volumes through NFS by default. Owing to this, showmount itself can be leveraged to figure out the volumes that are exported by Gluster servers. The output of  ‘showmount -e <server>‘ will be the list of volumes that can be mounted from the client. For e.g.

# showmount -e deepthought
Export list for deepthought:
/dr    *
/music *

If you just need a list of volumes, you can clear the extra output too by making use of your favorite filters. Something like:

#showmount -e  <server> | grep /  | tr -d ‘/’ | cut -d ‘ ‘ -f1

would yield you the list.

Once you have the list on your client, you can mount it through NFS and/or GlusterFS protocols, based on your choice of access protocol.