Tuesday, 15 June 2010

transmission + munin

OK, so things like the Rocks Avalanche Installer (pdf) show how useful bittorrent can be for providing a scalable install service. How do you know the status of bittorrent installs though? transmission-remote -l is all well and good if you're on the console but what about long term graphing? Cue Munin and a quick bit of python:

# Munin plugin for monitorring transmission bittorrent status
# Andrew Elwell
# $Id: $
import sys, os

if len(sys.argv) == 2 and sys.argv[1] == "autoconf":
print "yes"
elif len(sys.argv) == 2 and sys.argv[1] == "config":
print 'graph_args --base 1000'
print 'graph_title Transmission torrent status'
print 'graph_vlabel Count'
print 'graph_category Transmission'

print 'Stopped.label Stopped'
print 'Stopped.draw AREA'
print 'Idle.label Idle'
print 'Idle.draw STACK'
print 'Seeding.label Seeding'
print 'Seeding.draw STACK'
print 'Up.label Up and Down'
print 'Up.draw STACK'
print 'Downloading.label Downloading'
print 'Downloading.draw STACK'
print 'Will.label Will Verify'
print 'Will.draw STACK'
print 'Verifying.label Verifying'
print 'Verifying.draw STACK'

# OK we cheat - initialise ALL values to 0, then overwrite with 'real' ones, then spit out the whole dictionary.
data = {'Stopped': 0, 'Seeding': 0, 'Idle': 0, 'Up': 0, 'Downloading':0, 'Will':0,'Verifying':0 }
for state in os.popen("transmission-remote -l |sed 's/^\s\+//g' | sed 's/\s\{2,\}/~/g' | cut -d '~' -f8 | sort | uniq -c | grep -v '1 $' | grep -v '1 Status' | awk '{print $2, $1}'"):
(s,c) = state.split()
data[s] = c
for k,v in data.iteritems():
print k + '.value' ,v

EDIT: Suppose I should really find a nice template that copes with embedding code, but in the meantime thats what 'view source' is for...

Monday, 14 June 2010

Kerberos & SSH at CERN

Quick blog posting to save me having to explain to people what's needed:

1) RTFM: http://linux.web.cern.ch/linux/docs/kerberos-access.shtml -- thats where most of the good debugging tips are

2) to get it working between say your ubuntu laptop and cern hosts you'll need to append
allow_weak_crypto = true
to /etc/krb5.conf [libdefaults] section. (see bug)

3) make life easy and put a few things in your ~/.ssh/config

host *
Protocol 2
VerifyHostKeyDNS yes
VisualHostKey yes
GSSAPIAuthentication yes
PreferredAuthentications gssapi-with-mic,publickey

host lxplus
hostname lxplus.cern.ch
ForwardAgent yes
GSSAPIDelegateCredentials yes

(despite what the man page says there are NO SPACES between the options in PreferredAuthentications (see SSH bug 1702)

Plotting Lustre MDS stats

At $dayjob we have several large filesystems - for example our /scratch system has 3.1 PB of space using over 1000 HDDs. Although each vendo...