Jeff Darcy, Gluster developer extraordinaire with Red Hat, has written an article all about extending our favorite distributed storage system with Python, and he gets into a fair bit of detail with Glupy, his project for implementing new features in GlusterFS with Python. Glupy does this by utilizing GlusterFS’ established translator API, which you can read about on gluster.org.
I’ll let Jeff introduce the article in his own words:
Are you a Python programmer who wishes your storage could do more for you? Here’s an easy way to add functionality to a real distributed filesystem, in your favorite language.
And then gets into the good stuff:
As a case study in combining Python code with an existing compiled program or language, this article focuses on the implementation of a Python “translator” interface for GlusterFS. GlusterFS is a modern distributed filesystem based on the principle of horizontal scaling—adding capacity or performance to a system by adding more servers based on commodity hardware instead of having to pay an ever-increasing premium to make existing servers more powerful. Development is sponsored by Red Hat, but it’s completely open source, so anyone can contribute. In addition to horizontal scaling, another core principle of GlusterFS is modularity. Most of the functionality within GlusterFS actually is provided by translators—so called because they translate I/O calls (such as read or write) coming from the user into the same or other calls that are passed on toward storage. These calls are passed from one translator to another, arranged in an arbitrarily complex hierarchy, until eventually the lowest-level calls are executed on servers’ local filesystems. I call this interface TXAPI here for the sake of brevity, even though that’s not an official term. TXAPI has been used to implement internal GlusterFS functionality, such as replication and caching, and also external functionality, such as on-disk encryption.
I highly recommend the full article at LinuxJournal.com for your bedtime reading.