Thursday, 3 September 2009

BeautifulSoup + savannah

At work, we use savanne to power http://savannah.cern.ch where we keep track of all the glite-middleware (amongst other things)

However, one of the problems with savanne/savannah is that it doesn't have a nice machine readable API, but requires the user to click with a browser.

Cue Beautiful Soup and lo, the python SavannahCLI was born.

Well, in its early years it was unloved and handed round from developer to developer, until it arrived as a pedantic teenager to yours truly who decided to get rid of the horrible 'operation, item, name, value" syntax and replace with something more, um, fluid instead. (--patch 1234 --set foo=bar). It also had a huge set of hard coded lookup tables for converting select items to names and vice versa -- WHY?

thanks to some googling, I came across this which, with a little refinement gave me:


def enum_forms():
# we search within the table tag so that we lose the LHS searchbox
selects = soup.find('table').findAll('select')
for sel in selects:
sel_name = sel['name']
print "SELECT NAME=%s" % sel_name
for o in sel.findAll('option'):
print " %s, %s" % (o['value'],o.renderContents())

hey, it's not finished yet, but sure saves a whole pile of hard-coded badness.

No comments:

Plotting Lustre MDS stats

At $dayjob we have several large filesystems - for example our /scratch system has 3.1 PB of space using over 1000 HDDs. Although each vendo...