all posts tagged storage


by on April 8, 2015

GlusterFS-3.4.7 Released

The Gluster community is please to announce the release of GlusterFS-3.4.7.

The GlusterFS 3.4.7 release is focused on bug fixes:

  • 33608f5 cluster/dht: Changed log level to DEBUG
  • 076143f protocol: Log ENODATA & ENOATTR at DEBUG in removexattr_cbk
  • a0aa6fb build: argp-standalone, conditional build and build with gcc-5
  • 35fdb73 api: versioned symbols in libgfapi.so for compatibility
  • 8bc612d cluster/dht: set op_errno correctly during migration.
  • 8635805 cluster/dht: Fixed double UNWIND in lookup everywhere code

GlusterFS 3.4.7 packages for a variety of popular distributions are available from http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/

by on March 17, 2015

GlusterFS 3.4.7beta2 is now available for testing

The 2nd beta for GlusterFS 3.4.7 is now available for testing. A handful of bugs have been fixed since the 3.4.6 release, check the references below for details.

Bug reporters are encouraged to verify the fixes, and we invite others to test this beta to check for regressions. The ETA for 3.4.7 GA is not set, but will likely be before the end of March. Please note that the 3.4 release will EOL when 3.7 is released.

Packages for different distributions can be found on the main download site.

Release Notes for GlusterFS 3.4.7

GlusterFS 3.4.7 consists entirely of bug fixes. The Release Notes for 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.4.5, and 3.4.6 contain a listing of all the new features that were added and bugs fixed in the GlusterFS 3.4 stable release.

The following changes are included in 3.4.7:

  • a0aa6fb build: argp-standalone, conditional build and build with gcc-5
  • 35fdb73 api: versioned symbols in libgfapi.so for compatibility
  • 8bc612d cluster/dht: set op_errno correctly during migration.
  • 8635805 cluster/dht: Fixed double UNWIND in lookup everywhere code

Known Issues:

  • memory leak in glusterfs fuse bridge
  • File replicas differ in content even as heal info lists 0 entries in replica 2 setup

And if you’re wondering what happened to the first beta, it was made, but did not build with gcc-5.

by on November 7, 2014

Some notes on libgfapi.so symbol versions in GlusterFS 3.6.1

A little bit of background——

We started to track API/ABI changes to libgfapi.so by incrementing the SO_NAME, e.g. libgfapi.so.0(.0.0). In the master branch it was incremented to to ‘7’ or libgfapi.so.7(.0.0) for the eventual glusterfs-3.7.

I believe, but I’m not entirely certain¹, that we were supposed to reset this when we branched for release-3.6. Reset it to either ‘6’ or ‘0’, but we didn’t — apparently we forgot about it. In the 3.6.0 betas and if you build the GA release of 3.6.0 you’ll get a libgfapi.so.7(.0.0).

We didn’t hear much, if anything about this until a few days before 3.6.0 was scheduled to GA, when we were ‘reminded’ that older versions of applications like qemu, Samba, and more — linked against previous versions of libgfapi.so — no longer worked after upgrading to the new version of glusterfs.

We briefly experimented with adding a -compat package that installed a symlink: libgfapi.so.0 -> libgfapi.so.7; but some thought this was too much of a hack, and we abandoned that idea.

As a result we now have symbol versions in libgfapi.so. I’ve posted a public spreadsheet with a table of the symbols and the versions of glusterfs that they appear in at

https://docs.google.com/spreadsheets/d/1SKtzgEeVZbKFJgGGMdftf0p-AB5U7yyhVq1n2b6hBeQ/edit?usp=sharing

and also at

https://ethercalc.org/lrjvqrapzu

There are a few things to note about the symbol versions:

  1. so far all the function signatures, i.e. number of parameters and their types have not changed since libgfapi was introduced in 3.4.0. That’s a Good Thing.
  2. the symbol versions are taken from the (community) glusterfs release that they first appeared in.
  3. there are two functions declared in glfs.h that do not have an associated definition. So far it’s not clear why.
  4. there are two functions defined (in glfs-fops.c) that look like they ought to have declarations in glfs.h. Perhaps this was an oversight in the original implementation.
  5. there are several (private?) functions in libgfapi that are not declared in a public header that are used/referenced outside the library. That’s not a Bad Thing, per se, but it’s also not a Good Thing. It seems a bit strange for, e.g. glfs-heal and the nfs server xlator to have to be linked against libgfapi.so. These functions should either be made public or moved to another library, e.g. libglusterfs.so

N.B. that 3, 4, and 5 are also noted in comments in the spreadsheets.

In parallel to all this, RHEL 6.6, RHEL 7.0, and the related CentOS releases shipped updated RHS-GlusterFS client-side packages with version 3.6.0.X. This has resulted in confusion for many of the users of Community GlusterFS who are having issues with upgrading their systems. On top of that the libgfapi.so in these releases is libgfapi.so.0(.0.0), and there are applications included in those releases that are linked with it.

Earlier today (7 Nov, 2014) we released GlusterFS 3.6.1 to address and hopefully mitigate the 3.6.0 upgrade issue and fix the libgfapi.so compatibility issue by reverting to the original SO_NAME (libgfapi.so.0.0.0), now with symbol versions. The applications will not need to be recompiled or relinked.

Knowing that we were going to quickly release GlusterFS 3.6.1 to address these issues, to save our community packagers some work and to try to minimize confusion² we agreed in the community to not package GlusterFS 3.6.0 for any of the Linux distributions. We expect packages for 3.6.1 to be available soon on download.gluster.org.

And if anyone is looking for a nice Google Summer of Code project, linking libglusterfs and the xlators with link maps — with or without symbol versions — is an idea that I think has some merit.

HTH.

¹Not without slogging through a lot of old emails to reconstruct what we originally intended.

²Then we don’t have to explain why some Linux distributions have community 3.6.0 packages and others have (only) 3.6.1.

by on November 6, 2014

GlusterFS 3.4.6beta2 is now available for testing

Even though GlusterFS-3.6.0 was released last week, maintenance continues on the 3.4 stable series!

The 2nd beta for GlusterFS 3.4.6 is now available for testing. Many bugs have been fixed since the 3.4.5 release, check the references below for details.

Bug reporters are encouraged to verify the fixes, and we invite others to test this beta to check for regressions. The ETA for 3.4.6 GA is tentatively set for the week of 10 November, after testing shows the release to be stable.

Packages for different distributions can be found on the main download site.

Release Notes for GlusterFS 3.4.6

GlusterFS 3.4.6 consists entirely of bug fixes. The Release Notes for 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, and 3.4.5 contain a listing of all the new features that were added and bugs fixed in the GlusterFS 3.4 stable release.

The following changes are included in 3.4.6:

  • 049580f logrotate: gluster logrotate config should not be global
  • dc8a349 cluster/afr: Handle EAGAIN properly in inodelk
  • 4dc4325 socket: disallow CBC cipher modes
  • fe3e541 Cluster/DHT: Changing rename log severity
  • 3ade635 cluster/dht: Rename should not fail post hardlink creation
  • 49c4106 cluster/dht: Treat linkto file rename failure as non-critial error
  • c527449 cluster/dht: synchronize rename and file-migration
  • 0ebe35e libglusterfs/syncop: implement inodelk
  • fcd256f cluster/dht: introduce locking api.
  • 7a1e42e cluster/dht: Fix dht_access treating directory like files
  • 91175b3 cluster/dht: Prevent dht_access from going into a loop.
  • ebdb73d dht: fix rename race
  • fe5cf30 storage/posix: removing deleting entries in case of creation failures
  • b3387c8 cluster/dht: Fix races to avoid deletion of linkto file
  • 3042613 DHT/Create: Failing to identify a linkto file in lookup_everywhere_cbk
  • a735680 dht: fix rename race
  • bc75418 DHT/readdirp: Directory not shown/healed on mount point if exists on sin
  • 67ccd15 dht/rebalance: Do not allow rebalance when gfid mismatch found
  • f8b5bfd glusterfs.spec.in: add psmisc to -server subpackage
  • ab0547e socket: Fixed parsing RPC records containing multi fragments
  • e2a76e7 cluster/dht: Fix dict_t leaks in rebalance process’ execution path
  • c0b40b5 gNFS: Fix memory leak in setacl code path
  • 1d4ef0b mount/fuse: Handle fd resolution failures
  • f0ddba7 cluster/afr: Fix memory leak of file-path in self-heal-daemon
  • 1679b72 NFS: stripe-xlator should pass EOF at end of READDIR

Known Issues:

  • memory leak in glusterfs fuse bridge
  • data loss when rebalance + renames are in progress and bricks from replica pairs goes down and comes back
  • rebalance process crash after add-brick and `rebalance start’ operation