‘Instant’ Upload to Rackspace CloudFiles

Using inotify and the ‘swift’ client tools it is possible to automatically upload files to cloudfiles as they are written to disk.

This code is untested and might cause planes to drop from the sky, use it at your own risk!

Google Authenticator F5 IRule

Two Factor authentication is rather hit and miss in terms of support from web apps.

A quick look around the web turns up an article on DevCentral for a solution to implement google authentication with ldap. As I don’t run a LDAP server at home I needed to hack up the script a bit. This iRule implements the two factor side of things from the above article, but skips the LDAP side of things, as it’s not needed!

Auditd logging all commands

A common requirement for PCI-DSS is for all commands run by a user who has admin privileges to be logged. There are many ways to do this, most of the time people will opt for a change to the bash configuration or to rely on sudo. There are many ways arround this (such as providing the command that you wish to run as a paramater to SSH). As the linux kernel provides a full auditing system. Using a utility such as auditd, we are able to log all commands that are run. The configuration for this is actually quite simple. In /etc/audit/audit.rules we need to ensure that the following exists.

-a exit,always -F arch=b64 -S execve
-a exit,always -F arch=b32 -S execve

This will capture any execve system call (on exit) and will log this to the auditd log. A log entry will look similar to below.

type=SYSCALL msg=audit(1318930500.123:3020171): arch=c000003e syscall=59 success=yes exit=0 a0=7fff65179def a1=7fff65179ec0 a2=7fff6517d060 a3=7ff54ee36c00 items=3 ppid=9200 pid=9202 auid=0 uid=1000 gid=100 euid=1000 suid=1000 fsuid=1000 egid=100 sgid=100 fsgid=100 tty=(none) ses=4 comm="xscreensaver-ge" exe="/usr/bin/perl" key=(null)
type=EXECVE msg=audit(1318930500.123:3020171): argc=5 a0="/usr/bin/perl" a1="-w" a2="/usr/bin/xscreensaver-getimage-file" a3="--name" a4="/home/welby/Pictures"
type=EXECVE msg=audit(1318930500.123:3020171): argc=3 a0="/usr/bin/perl" a1="-w" a2="/usr/bin/xscreensaver-getimage-file"
type=CWD msg=audit(1318930500.123:3020171): cwd="/home/welby/Downloads"
type=PATH msg=audit(1318930500.123:3020171): item=0 name="/usr/bin/xscreensaver-getimage-file" inode=208346 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1318930500.123:3020171): item=1 name=(null) inode=200983 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1318930500.123:3020171): item=2 name=(null) inode=46 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00

This should keep most auditors happy 🙂

Moving part of an lvm vg from one pv to another

Lets say that you’ve got multiple physical volumes (PV) in a Volume Group (VG) and you want to migrate the extents from one PV to another, this can be acomplished with a quick and easy pvmove command.

For example

pvdisplay -m
--- Physical volume ---
PV Name /dev/sdb1
VG Name INTEL_RAID
PV Size 2.73 TiB / not usable 4.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 714539
Free PE 0
Allocated PE 714539
PV UUID XWiRzE-Ol3d-38En-ND6b-qo93-4zeF-xv8zDv

--- Physical Segments ---
Physical extent 0 to 604876:
Logical volume /dev/INTEL_RAID/MEDIA
Logical extents 0 to 604876
Physical extent 604877 to 617676:
Logical volume /dev/INTEL_RAID/backups_mimage_0
Logical extents 25600 to 38399
Physical extent 617677 to 617701:
Logical volume /dev/INTEL_RAID/EPG
Logical extents 0 to 24
Physical extent 617702 to 643301:
Logical volume /dev/INTEL_RAID/backups_mimage_0
Logical extents 0 to 25599
Physical extent 643302 to 714538:
Logical volume /dev/INTEL_RAID/MEDIA
Logical extents 604877 to 676113

--- Physical volume ---
PV Name /dev/sdc1
VG Name INTEL_RAID
PV Size 2.04 TiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 535726
Free PE 430323
Allocated PE 105403
PV UUID laOuKy-5FZa-cJ3h-JffV-qUub-diKC-O0wVqK

--- Physical Segments ---
Physical extent 0 to 25599:
Logical volume /dev/INTEL_RAID/backups_mimage_1
Logical extents 0 to 25599
Physical extent 25600 to 54202:
Logical volume /dev/INTEL_RAID/MEDIA
Logical extents 676114 to 704716
Physical extent 54203 to 67002:
Logical volume /dev/INTEL_RAID/NZB_DOWNLOAD
Logical extents 0 to 12799
Physical extent 67003 to 79802:
Logical volume /dev/INTEL_RAID/backups_mimage_1
Logical extents 25600 to 38399
Physical extent 79803 to 105402:
Logical volume /dev/INTEL_RAID/OLD_VM
Logical extents 0 to 25599
Physical extent 105403 to 535725:
FREE

From here you can see that /dev/INTEL_RAID/MEDIA is a Logical Volume (LV) on both PVs within the VG. If I was wanting to grow my mirrored LV, which requires space on both PVs, I’d have to migrate some of the extents of another LV. If I wanted to move some of the MEDIA lv, I should be able to do the following

pvmove /dev/sdb:643302-714538 /dev/sdc

This will move extents 643302-714538 to the next contiguious block on /dev/sdc

Dahdi In LXC

At home we use various VoIP providers to either get free calls to various places (GTalk/GVoice to america for instance) and various other destinations over SIP providers

I’ve been using Asterisk for years (I remeber the 0.7 release) and have implemented it for companies before, usually with no issues, baring the continuall deadlocks in the 1.2 range. Recently I enabled my VoIP network segment for IPv6 only to find that GTalk stoped working on IPV6 Day. After a bit of digging about, it seems that Asterisk 1.8 does support IPV6! But, gtalk and similar are not supported, SIP is infact the only first class citezen it seems.

I’ve toyed with using freeswitch before, but unfortuantly have had varied sucsess with FreeTDM to Dahdi with BT caller ID and the likes. I did hack in support for it, but I’m not too sure if I trust my code, as my C is quite rusty to say the least.

I did however come up with another solution!

As I’m running a moderatly new Linux Kernel I can use LXC – Linux Containers – which are effectilvy the same idea as a wpar, chroot, openvz, whatever. After setting up asterisk in the LXC I needed to expose my Dahdi card to it. LXC allows you to restrict access on a per device basis. I’ve setup Dahdi on the host machine as normal so the kernel modules can be loaded etc. Once this is done I’ve preformed the following within the LXC

cd /
mkdir dev/dahdi
mknod dev/dahdi/1 c 196 1
mknod dev/dahdi/2 c 196 2
mknod dev/dahdi/3 c 196 3
mknod dev/dahdi/4 c 196 4
mknod dev/dahdi/channel c 196 254
mknod dev/dahdi/ctl c 196 0
mknod dev/dahdi/pseudo c 196 255
mknod dev/dahdi/timer c 196 253
mknod dev/dahdi/transcode c 196 250

This creates the Device nodes within /dev/ for my 4 dahdi channels (3FXS 1FXO if anyone is interested). After this I’ve added the following to the lxc config file, to allow the LXC to have access to these devices


# If you want to be lazy just add this line
#lxc.cgroup.devices.allow = c 196:* rwm

#Otherwise use the following
lxc.cgroup.devices.allow = c 196:0 rwm
lxc.cgroup.devices.allow = c 196:1 rwm
lxc.cgroup.devices.allow = c 196:2 rwm
lxc.cgroup.devices.allow = c 196:3 rwm
lxc.cgroup.devices.allow = c 196:4 rwm
lxc.cgroup.devices.allow = c 196:250 rwm
lxc.cgroup.devices.allow = c 196:253 rwm
lxc.cgroup.devices.allow = c 196:254 rwm
lxc.cgroup.devices.allow = c 196:255 rwm

This will obviously only work for the first 4 dahdi channels, but if you need more, just continue adding the 196:x lines, replacing x with the channel number, and also ensuring that you create the device nodes in the same way

A quick (and quite unscientific!) break down of Rackspace CloudFiles UK vs Amazon S3 (Ireland)

(Disclaimer – I’m a Rackspace Employee, the postings on this site are my own, may be bias, and don’t necessarily represent Rackspace’s positions, strategies or opinions. These tests have been preformed independently from my employer by my self)

As Rackspace have recently launched a ‘beta’ Cloudfiles service within the UK I thought I would run a few tests to compare it to Amazon’s S3 service running from Eire (or Southern Ireland).

I took a set of files, totalling 18.7GB, with file sizes ranging from between 1kb and 25MB, text files, and contents being mainly Photos (both JPEG and RAW (cannon and nikon), plain text files, GZiped Tarballs and a few Microsoft Word documents just for good measure.

The following python scripts were used:

Cloud Files
Upload

Download

s3
Upload

Download

The test was preformed from a Linux host which has a 100MBit connection (Uncapped/unthrottled) in London, however the test was also preformed with almost identical results from a machine in Paris (also 100mbit). Tests were also run from other locations (Dallas Fort Worth – Texas, my home ISP (bethere.co.uk)) however these locations were limited to 25mbit and 24mbit , and both reached their maximum speeds. The tests were as follows:

  • Download files from Rackspace Cloudfiles UK (these had been uploaded previously) – This is downloaded directly via the API, NOT via a CDN
  • Upload the same files to S3 Ireland
  • Upload the same files to a new “container” at Rackspace Cloudfiles UK
  • Download the files from S3 Ireland – This is downloaded directly via the API, NOT via a CDN
  • The average speeds for the tests are as follows:
    Cloudfiles
    Download: 90Mbit/s
    Upload: 85MBit/s
    S3 Ireland
    Download: ~40Mbit/s
    Upload : 13Mbit/s

    Observations

  • Cloud files seems to be able to max out a 100mbit connection for both File
  • S3 seems to have a cap of 13mbit for inbound file transfers?
  • S3 seems to either be extremely unpredictable on file transfer speeds for downloading files via the API, or there is some form of cap after a certain amount of data transferred, or there was congestion on the AWS network
  • Below is a graph showing the different connection speeds achieved using CF & S3

    As mentioned before this is a very unscientific test (and I’d say that these results have not been replicated from as many locations or as many times as I’d like to, so I would take them with a pinch of salt) , but it does appear that Rackspace cloudfiles UK is noticeably faster than S3 Ireland

    Slice

    I’ve moved this site (and a few others) from its semi-temporary home of one of my NAS boxes (proper x86 machines remember, not any of these silly 200MHz mips devices!) to a Slice at the wonderful slicehosts. I’d Highly recommend them.

    Lighttpd: mod_security via mod_magnet

    In most large enterprises there is a requirement to comply with various standards. The hot potato in the Ecommerce space at the moment (and has been for a few years!) is PCI-DSS.

    At $WORK we have to comply with PCI-DSS with the full audit and similar occurring due to the number of transactions we perform. Recently we’ve deployed lighttpd for one of our platforms, which has caused an issue for our Information Security Officers and Compliance staff.

    PCI-DSS 6.6 requires EITHER a Code review to be preformed, which whilst this may seem to be an easy task, when you’re talking about complex enterprise applications following a very……… agile development process it’s not always an option. The other option is to use a WAF (Web Application Firewall). Now there are multiple products available that sit upstream and perform this task. There is however an issue if you use SSL for your traffic. Most WAF will not do the SSL decryption / reencryption between the client and server (effectively becoming a Man in the Middle). There are however a few products which do this, F5 networks’ ASM being one that springs to mind. Unfortunately this isn’t always an option due to licensing fees and similar. An alternative is to run a WAF on the server its self. A common module for this is Mod_Security for Apache. Unfortunately, a similar module does not exist for Lighttpd.

    In response to $WORKs requirement for this I’ve used mod_magnet to run a small lua script to emulate the functionality of mod_security (to an extent at least!). Please note that mod_magent is blocking, so will cause any requests to be blocked until the mod_magnet script has completed, so be very careful with the script, and ensure that it’s not causing any lag in a test environment, prior to deploying into live!

    Below is a copy of an early version of the script (most of the mod_security rules that we have are specific to work, so are not being included for various reasons), however I’ll post updates to this soon.

    /etc/lighttpd/mod_sec.lua

    The following needs to be added to lighttpd.conf to attach this LUA script via mod magnet

    *Update – 23 Aug 09* Updated to return code even if one test passes*

    Comments or suggestions are appreciated!