all posts tagged hack


by on September 22, 2015

Docker Global Hack Day #3, Bangalore Edition

We organized Docker Global Hack Day at Red Hat Office on 19th Sep’15. Though there were lots RSVPs, the turn up for the event was less than expected. We started the day by showing the recording of kick-off event.

Docker Global Hack #3The teams here worked on four different ideas, out of which two submitted to the Global Hack github page. The four ideas on which teams worked on are:-

Alan and Fayiz worked on PaaS idea, which can be used for setting up dev and qa environment.

Archit was winner of Docker Global Hack Day #2 as well for the same project. He updated the same project in this hackathon. His project is about crowd source analysis by using distributed computing  through  Docker.

  • Visualizing Docker Networking – Himanshu Roy

Himanshu was exploring the idea of visualizing multi-host Docker networking.

  • Spreading and collating containers on GlusterFS with runC -Mohamed Ashiq Liyazudeen, Hari Gowtham

By looking at runC demo in the kick-off video, we thought it would be good if we can run containers on GlusterFS and use it move containers around by saving and restoring the containers on shared volume.

I did not work on specific idea but I was helping teams and other attendees with their questions. I also worked on my upcoming tutorial at Linux Con, Europe on Data and Networking management with containers.

May be because of long weekend and other events we got less participation. Hopefully we do better next time.

by on November 3, 2014

8th Bangalore Docker meetup and Global Hackathon#2

On 1st Nov’14, Red Hat offices in Bangalore and Pune hosted Docker meetups and Hackathon.

~40 people attended Bangalore meetup. Before the hackathon we had following presentations :-

  • Docker Global Hackday opening by Avi Cavale, Co-founder and CEO, Shippable.
  • Introduction to Docker – Pranay Pareek, Shippable
  • Introduction to Project Atomic – Neependra Khare, Red Hat.

In the last meetup we looked at CoreOS and decided to to look at Project Atomic in this meetup. I used Colin Walters  slides to give Atomic presentations and then used Fedora 20 Atomic image to give Demo.

After the presentation and Lunch we had the hackathon. Four teams participated. I could not due to being host but I helped first timers with Docker handons. The four hacks we did are:-

1. Dockit – Docker GlusterFS integration @HumbleDevassy, @swordphilic, @hiSaifi

 

2. dockerComp – @arcolife,@krishnakalyan3 – Bangalore Hackathon Winner

 

3. Docker as a Load Balancer. @anandrm

Dockerize a Load Balancer to replace a Standalone Load Balancers.

4. CI with Docker @srikrishnaholla, @harishk8591106, @dilipkuki

We did not have anyone to judge, so we did the presentations and cast the vote ourselves. Local winner was dockerComp.

In Pune we did not hackathon but had great workshop. Look at the meetup page for awesome feedback. The workshop labs are available at following to anyone to use.

http://people.redhat.com/rrajaram/dockermeetup/

Overall it was a great experience. There were requests to do hands-on with Docker. We’ll try to do in next meetup.

 

by on September 28, 2013

Finite state machines in puppet

In my attempt to push puppet to its limits, (for no particular reason), to develop more powerful puppet modules, to build in a distributed lock manager, and to be more dynamic, I’m now attempting to build a Finite State Machine (FSM) in puppet.

Is this a real finite state machine, and why would you do this?

Computer science professionals might not approve of the purity level, but they will hopefully appreciate the hack value. I’ve done this to illustrate a state transition technique that will be necessary in a module that I am writing.

Can we have an example?

Sure! I’ve decided to model thermodynamic phase transitions. Here’s what we’re building:

Phase_change_-_en.svg

How does it work?

Start off with a given define that accepts an argument. It could have one argument, or many, and be of whichever type you like, such as an integer, or even a more complicated list type. To keep the example simple, let’s work with a single argument named $input.

define fsm::transition(
        $input = ''
) {
        # TODO: add amazing code here...
}

The FSM runs as follows: On first execution, the $input value is saved to a local file by means of a puppet exec type. A corresponding fact exists to read from that file and create a unique variable for the fsm::transition type. Let’s call that variable $last. This is the special part!

# ruby fact to pull in the data from the state file
found = {}
Dir.glob(transition_dir+'*').each do |d|
    n = File.basename(d)    # should be the fsm::transition name
    if n.length > 0 and regexp.match(n)
        f = d.gsub(/\/$/, '')+'/state'    # full file path
        if File.exists?(f)
            # TODO: future versions should unpickle (but with yaml)
            v = File.open(f, 'r').read.strip    # read into str
            if v.length > 0 and regexp.match(v)
                found[n] = v
            end
        end
    end
end

found.keys.each do |x|
    Facter.add('fsm_transition_'+x) do
        #confine :operatingsystem => %w{CentOS, RedHat, Fedora}
        setcode {
            found[x]
        }
    end
end

On subsequent runs, the process gets more interesting: The $input value and the $last value are used to decide what to run. They can be different because the user might have changed the $input value. Logic trees then decide what actions you’d like to perform. This lets us compare the previous state to the new desired state, and as a result, be more intelligent about which actions need to run for a successful state transition. This is the FSM part.

# logic tree modeling phase transitions
# https://en.wikipedia.org/wiki/Phase_transition
$transition = "${valid_last}" ? {
        'solid' => "${valid_input}" ? {
               'solid' => true,
               'liquid' => 'melting',
               'gas' => 'sublimation',
               'plasma' => false,
               default => '',
        },
        'liquid' => "${valid_input}" ? {
               'solid' => 'freezing',
               'liquid' => true,
               'gas' => 'vaporization',
               'plasma' => false,
               default => '',
        },
        'gas' => "${valid_input}" ? {
               'solid' => 'deposition',
               'liquid' => 'condensation',
               'gas' => true,
               'plasma' => 'ionization',
               default => '',
        },
        'plasma' => "${valid_input}" ? {
               'solid' => false,
               'liquid' => false,
               'gas' => 'recombination',
               'plasma' => true,
               default => '',
        },
        default => '',
}

Once the state transition actions have completed successfully, the exec must store the $input value in the local file for future use as the unique $last fact for the next puppet run. If there are errors during state transition execution, you may choose to not store the updated value (to cause a re-run) and/or to add an error condition fact that the subsequent puppet run will have to read in and handle accordingly. This is the important part.

$f = "${vardir}/transition/${name}/state"
$diff = "/usr/bin/test '${valid_input}' != '${valid_last}'"

# TODO: future versions should pickle (but with yaml)
exec { "/bin/echo '${valid_input}' > '${f}'":
        logoutput => on_failure,
        onlyif => "/usr/bin/test ! -e '${f}' || ${diff}",
        require => File["${vardir}/"],
        alias => "fsm-transition-${name}",
}

Can we take this further?

It might be beneficial to remember the path we took through our graph. To do this, on each transition we append the new state to a file on our local puppet client. The corresponding fact, is similar to the $last fact, except it maintains a list of values instead of just one. There is a max length variable that can be used to avoid storing unlimited old states.

Does this have a practical use?

Yes, absolutely! I realized that something like this could be useful for puppet-gluster. Stay tuned for more patches.

Hopefully you enjoyed this. By following the above guidelines, you should now have some extra tricks for building state transitions into your puppet modules. Let me know if you found this hack awesome and unique.

I’ve posted the full example module here.

Happy Hacking,

James

 

by on June 29, 2013

Unlimited PyCharm user license for GlusterFS devs

The people at JetBrains have given the GlusterFS project an “unlimited # of users” license for their cross-platform PyCharm Python IDE:

http://www.jetbrains.com/pycharm/download/

Everyone has their own preferences with programming IDE’s, and I’ve used several over the years, generally based on Eclipse.  However, I happen to like PyCharm at the moment, so applied for JetBrain’s “free license for OSS projects” for the GlusterFS project:

http://www.jetbrains.com/pycharm/buy/buy.jsp#openSource

If you’re a GlusterFS developer (with at least 1 commit on record) email me (jclift@redhat.com or justin@postgresql.org) and I’ll send the license file to you.

If you want a PyCharm license but aren’t a GlusterFS developer yet, getting started with GlusterFS development is both interesting and fairly easy:

http://www.gluster.org/community/documentation/index.php/Developers

by on April 8, 2013

Knowing when to release and deploy your code (…and a mini script)

Knowing when to release and deploy your code can turn into a complicated discussion. In general, In general, I tend to support releasing early and often, for some value of $early and $often. I’ve decided to keep this simple and introduce you to one metric that I use…

I think that I am fairly diligent in adding plenty of comments to my source code. I might even sometimes add too many. I create plenty of XXX, FIXME, or TODO tagged comments as reminders of things to work on.

To me, XXX represents an important problem that should get looked at or fixed; FIXME, reminds me that I should definitely look into something, and finally, TODO gives me homework or things to pursue when I’m in need of a new project.

I try to resolve most if not all XXX tagged comments before making a 0.1 release, FIXME’s to consider something very stable, and a lack of TODO’s mean something is completely done for now.

To count all these, I wrote a little tool that greps through the top-level directories in my ~/code/ folder, and displays the results in a table. Feel free to give it a try, and use it for your own projects.

While I don’t see this as a particularly game changing utility, it scratches my itch, and helps me keep up my bash skills. The code is available here. Let me know if you have any improvements, or if the source isn’t enough documentation for you.

Happy hacking,

James