Extending GlusterFS with Python

Although this behavior makes sense for many applications, the performance impact for many PHP applications can be severe. Without negative-lookup caching, you're likely to search half of those directories in vain before finding the one that contains each include file, every time the including page is requested. (This pattern does occur in other environments as well, including Python Web applications, but common PHP frameworks cause those applications to be hit the hardest.) Just as the effects are severe, the benefits of adding a negative-lookup cache can be significant. For example, a C version of such a translator decreased average include-search times nearly seven-fold. What could a Python version do?

Here's part of a translator based on glupy:

def lookup_fop (frame, this, loc, xdata):
      pargfid = uuid2str(loc.contents.pargfid)
      print "lookup FOP: %s:%s" % (pargfid, loc.contents.name)
      # Check the cache.
      if cache.has_key(pargfid) and (loc.contents.name in cache[pargfid]):
              return 0
      key = dl.get_id(frame)
      requests[key] = (pargfid, loc.contents.name[:])
      return 0

This is the function that gets called to look up a file, which is the core functionality for this example. Entry to this function represents a transition from C to Python, while its return represents a transition back to C. Calls through the "dl" object—a handle to the C dynamic library that supports glupy—also suspend the Python interpreter while they run. The Python decorator syntax allows you to hide most of the function-type details, and there's also a notable lack of type-conversion code. Most of what's there is domain-specific code, not boiler plate required by the infrastructure.

In the top half of this function, you simply check the cache to see if you already know the requested file won't be there. If the cache check succeeds, the lookup fails immediately, and you "unwind" the translator stack to report that fact. As with the registration functions, each operation type has its own specific wind (call downward) and unwind (return upward) functions as well. This represents a temporary return from the "Python world" to the "C world", and it's worth noting that these transitions between worlds might occur seamlessly many times while processing a single request. In particular, a common GlusterFS translator idiom is for a completion callback on one request to initiate the next, and if that request completes immediately (as done here), then you can have multiple requests and completions all on the stack at once.

Returning to the code, if you do not find an entry in the cache (and you already know it must not be in the standard positive-lookup cache or else you wouldn't even have been called), you pass the request on to the next translator using wind_lookup. When that next translator is done, it returns control (through the glupy meta-translator) to lookup_cbk. Here you retrieve your request context, conveniently stashed in a dictionary for you by lookup_fop, and use it to update the cache according to whether the file was found.

There are a few other less relevant details of how this particular glupy translator works, but that really is the meat of it. With less than a hundred lines of Python code, including comments and empty lines, you can add a significant piece of functionality to a real filesystem. But, how well does it really work? As it turns out, it works very well; see Table 1. A simple test reveals that the result is slower than the C-based version of the same thing, but still more than four times as fast as the baseline. Clearly, the fact that you're caching these results matters more than what language you're using to do it.

Table 1. Results of Caching Failed-Lookup Requests
ms/lookup minimum average maximum 99th percentile
no caching 0.368 6.898 16.286 9.702
C version 0.379 1.036 18.503 2.180
glupy version 0.381 1.527 21.163 2.916

As promising as these results are, they're more of a beginning than an end. Glupy is still a very young project, and much remains to be done. Support needs to be added for a few dozen more operation types and several data structures. There still are more ways that GlusterFS calls into translators and utility functions that translators themselves call. There are many ways the glupy interface could be made more convenient, and there are undoubtedly performance or concurrency issues still to be resolved. The most important thing is that the basic infrastructure for doing all of these things already exists, and not just for GlusterFS translators. If even a highly multithreaded and asynchronous program like this can take advantage of all that Python has to offer, so can just about any other program. Thanks to Python's extension/embedding interface and ctypes module, a "best of both worlds" approach to developing complex software is more achievable than most people think.


Jython: http://www.jython.org

Cython: http://www.cython.org

GlusterFS: http://www.gluster.org

Glupy Source Repository: https://github.com/jdarcy/glupy

Negative-Lookup Caching Translator in C: https://github.com/jdarcy/negative-lookup

Zend (PHP) Framework on Include Files: http://framework.zend.com/manual/1.12/en/performance.classloading.html


Jeff Darcy has been working on network and distributed storage since that meant DECnet and NFS version 2 in the early 1990s. He is currently at Red Hat where he serves on the GlusterFS architecture team.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

german translation

Professional German Translation's picture

Professional German Translation

German language one of the most competitive to translate, and some of us are just brave enough to take a stand and learn professional german writing

and reading and not just that translating it also, however some us just gets lucky to be native German speakers. Me personally I am none of the

above, I had some problems translating my business proposal into German to achieve the goal, my dream that I was working day and night for, but life

is not that easy, I tried so many different softwares and yes the ugly truth I translated with a low quality and lost the bid. And that was the day I

owed to bring all those professional and certified translators at one place and do not let anybody else to lose, but rather achieves what they

deserve and that place is
Try our high-quality translation and affordable price which you may never have it in the past, but now you do and we Promise nothing else than

service and pride.

well and its really great

qasim's picture

well and its really great post while after read feel good thanks for sharing
Business communication

We are really grateful for

Usnam's picture

We are really grateful for your blog post. You will find a lot of approaches after visiting your post.
academic essay

An excellent post. The post

Raza's picture

An excellent post. The post affects a lot of urgent issues in our minds. We can not be indifferent to these problems. Your article gives the light in which we are able to watch our real life. Keep it up.
LA Fashion

Python language for programming

LeeJackie's picture

Always used the Python language for programming, so these tips are very interesting to me, I'm always looking for new commands! 2 Via Conta

Gluster and phinton

Zabbara's picture

Using ctypes, Python code can call functions and access symbols even in Tirinhas Memes

http://www.hairwigs.de/ I

CK's picture

I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work.seo companyremely fast.

But I'm no longer an edge

plinn's picture

But I'm no longer an edge case, because maps are proving to be essential on smartphones, which today approaches a billion or more people. Digital maps on phones are now among the core portfolio of smartphone apps, alongside voice, text, calendar and otimização de sites

Isant phyton a old school

dennishobson's picture

Isant phyton a old school language ?
i use to code in c# but now a days i only code html and some php... sad sad
madeira plastica

You tool is very useful for

Dayse's picture

You tool is very useful for those who want to get a programmer on my website: Filmes Downloads Tranks You!!!

Your blog is a source of

Tarots gratuit's picture

Your blog is a source of information, I am an avid reader and I wish you good luck.


James jonas's picture

network that blocks SSH or from devices that do not support SSH herbalife