Since I just spent about an hour trying to find this again, mythbuntu diskless clients have an “overlay” where the home directories and some other files specific to each instance are stored. On the server, you can find these in /var/cache/mythbuntu-diskless/overlay/[mac_address]/
This is mostly for my own reference, but someone else may find it useful.
I currently have two raid volumes in my home server. They’re running on four disks, which are partitioned thusly:
root@bender:~# fdisk -l /dev/sd[abcd] Disk /dev/sda: 400.0 GB, 400088457216 bytes 255 heads, 63 sectors/track, 48641 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 48641 390708801 83 Linux Disk /dev/sdb: 400.0 GB, 400088457216 bytes 255 heads, 63 sectors/track, 48641 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 1 48641 390708801 83 Linux Disk /dev/sdc: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdc1 1 48641 390708801 83 Linux /dev/sdc2 48642 60801 97675200 83 Linux Disk /dev/sdd: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xb7e306eb Device Boot Start End Blocks Id System /dev/sdd1 1 48641 390708832 83 Linux /dev/sdd2 48642 60801 97675200 83 Linux
Using the partitions as raid targets, I end up with two raid volumes:
md0 : active raid5 sdd1[3] sda1[0] sdc1[2] sdb1[1] md1 : active raid1 sdd2[1] sdc2[0]
On top of these raid volumes, I’m running an LVM so that they can be easily resized:
root@bender:~# pvdisplay --- Physical volume --- PV Name /dev/md1 VG Name vgraid1 PV Size 93.15 GB / not usable 9.88 MB Allocatable yes (but full) PE Size (KByte) 16384 Total PE 5961 Free PE 0 Allocated PE 5961 PV UUID NX3sku-sNGU-yqyw-3peE-Bd8W-KjBo-xsUol8 --- Physical volume --- PV Name /dev/md0 VG Name vgraid PV Size 1.09 TB / not usable 14.31 MB Allocatable yes (but full) PE Size (KByte) 16384 Total PE 71540 Free PE 0 Allocated PE 71540 PV UUID wtYFqQ-499L-BbuB-zaRb-EV16-Y5sT-ZcG65b
And now, the fun stuff. Here’s how the original raid5 volume was made (which was originally missing a 500G disk, because it died on me when I first got it):
root@bender:~# mdadm --create --verbose /dev/md0 --chunk=128 \ --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1 root@bender:~# pvcreate /dev/md0 root@bender:~# vgcreate -s 16M vgraid /dev/md0 root@bender:~# lvcreate -l 47693 vgraid -n bigdisk root@bender:~# mkfs.xfs /dev/vgraid/bigdisk
[note: this was done a while back, so i”ve lost the output. for output example see below when i create my raid1 array]
I finally received my new 500G drive from RMA, so I had to resize the raid5 to include the new disk. Here’s how it’s done.
First, create the new partitions to match the size on the other drive. (see above) Then, resize the raid array (takes a while…I started it then went to sleep, then it was magically done when I woke up this morning)
root@bender:~# mdadm --add /dev/md0 /dev/sdd1 mdadm: added /dev/sdd1 root@bender:~# mdadm --detail /dev/md0 /dev/md0: Version : 00.90.03 Creation Time : Fri Mar 21 09:52:31 2008 Raid Level : raid5 Array Size : 781417472 (745.22 GiB 800.17 GB) Used Dev Size : 390708736 (372.61 GiB 400.09 GB) Raid Devices : 3 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Sun Apr 20 23:28:58 2008 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 64K UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender) Events : 0.17564 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 8 17 1 active sync /dev/sdb1 2 8 33 2 active sync /dev/sdc1 3 8 49 - spare /dev/sdd1 root@bender:~# mdadm --grow /dev/md0 --raid-devices=4 mdadm: Need to backup 384K of critical section.. mdadm: ... critical section passed. root@bender:~# mdadm --detail /dev/md0 /dev/md0: Version : 00.91.03 Creation Time : Fri Mar 21 09:52:31 2008 Raid Level : raid5 Array Size : 781417472 (745.22 GiB 800.17 GB) Used Dev Size : 390708736 (372.61 GiB 400.09 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Sun Apr 20 23:29:48 2008 State : clean, recovering Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K Reshape Status : 0% complete Delta Devices : 1, (3->4) UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender) Events : 0.17680 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 8 17 1 active sync /dev/sdb1 2 8 33 2 active sync /dev/sdc1 3 8 49 3 active sync /dev/sdd1 root@bender:~# mdadm --detail /dev/md0 /dev/md0: Version : 00.90.03 Creation Time : Fri Mar 21 09:52:31 2008 Raid Level : raid5 Array Size : 1172126208 (1117.83 GiB 1200.26 GB) Used Dev Size : 390708736 (372.61 GiB 400.09 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Mon Apr 21 09:15:38 2008 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender) Events : 0.277372 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 8 17 1 active sync /dev/sdb1 2 8 33 2 active sync /dev/sdc1 3 8 49 3 active sync /dev/sdd1 root@bender:~# pvresize /dev/md0 Physical volume "/dev/md0" changed 1 physical volume(s) resized / 0 physical volume(s) not resized root@bender:~# vgdisplay --- Volume group --- VG Name vgraid System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 1.09 TB PE Size 16.00 MB Total PE 71540 Alloc PE / Size 47693 / 745.20 GB Free PE / Size 23847 / 372.61 GB VG UUID JPEInC-5bNp-f0iC-3611-qIIn-Drhd-A3xInG root@bender:~# lvextend -l +23847 /dev/vgraid/bigdisk Extending logical volume bigdisk to 1.09 TB Logical volume bigdisk successfully resized root@bender:~# xfs_growfs /dev/vgraid/bigdisk meta-data=/dev/mapper/vgraid-bigdisk isize=256 agcount=32, agsize=6104704 blks = sectsz=512 attr=0 data = bsize=4096 blocks=195350528, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 195350528 to 293027840 root@bender:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/hda1 73G 63G 6.2G 91% / varrun 1.9G 264K 1.9G 1% /var/run varlock 1.9G 0 1.9G 0% /var/lock udev 1.9G 120K 1.9G 1% /dev devshm 1.9G 0 1.9G 0% /dev/shm /dev/sde1 280G 271G 8.6G 97% /export/myth /dev/mapper/vgraid-bigdisk 1.1T 453G 665G 41% /export/archive
Since I now have my two 500G disks, I have an extra 100G on each (since I made all the partitions match to raid5 with the 400G disks) so I can raid1 those for a wonderfully redundant backup array.
root@bender:~# mdadm --create --verbose /dev/md1 --chunk=128 \ --level=1 --raid-devices=2 /dev/sdc2 /dev/sdd2 mdadm: chunk size ignored for this level mdadm: size set to 97675136K mdadm: array /dev/md1 started. root@bender:~# pvcreate /dev/md1 Physical volume "/dev/md1" successfully created root@bender:~# vgcreate -s 16M vgraid1 /dev/md1 Volume group "vgraid1" successfully created root@bender:~# vgdisplay --- Volume group --- VG Name vgraid1 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 93.14 GB PE Size 16.00 MB Total PE 5961 Alloc PE / Size 0 / 0 Free PE / Size 5961 / 93.14 GB VG UUID Cxrf05-HaUJ-sPHx-Ai9P-eRYL-Ofoz-mnIsFx --- Volume group --- VG Name vgraid System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 1.09 TB PE Size 16.00 MB Total PE 71540 Alloc PE / Size 71540 / 1.09 TB Free PE / Size 0 / 0 VG UUID JPEInC-5bNp-f0iC-3611-qIIn-Drhd-A3xInG root@bender:~# lvcreate -l 5961 vgraid1 -n backup Logical volume "backup" created root@bender:~# mkfs.xfs /dev/vgraid1/backup meta-data=/dev/vgraid1/backup isize=256 agcount=16, agsize=1526016 blks = sectsz=512 attr=0 data = bsize=4096 blocks=24416256, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=11922, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 root@bender:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/hda1 73G 63G 6.2G 91% / varrun 1.9G 264K 1.9G 1% /var/run varlock 1.9G 0 1.9G 0% /var/lock udev 1.9G 128K 1.9G 1% /dev devshm 1.9G 0 1.9G 0% /dev/shm /dev/sde1 280G 271G 8.6G 97% /export/myth /dev/mapper/vgraid-bigdisk 1.1T 453G 665G 41% /export/archive /dev/mapper/vgraid1-backup 94G 544K 94G 1% /export/backup
Huzzah! Less likeliness of lost data due to my stupidity! Now to write some scripts to do automatic backups. :)
A while back, I posted about a google reader function that I wanted–the ability to seamlessly share things from google reader to my blog. Lazyweb wasn’t much help, so I rolled out a quick-and-dirty greasemonkey script to do it. You can find it here: http://www.dknowles.org/other/google-reader-cross-share.user.js
Right now, it works by clicking on the share icon below a reader entry. I’m not catching keyboard shortcuts, so it requires a physical click. (fine with me, as I’m a keyboard junkie and can’t be bothered with constant naggings about sharing on my blog)
The regular “use-at-own-risk-may-break-at-any-time” disclaimer applies. Ideally, I’d eventually like this to make a greader-styled popup that asks for your blog entry, but that requires more work than I’m willing to put into this. Plus, this works as-is. If there’s interest in this, I can put it in SVN somewhere so people can give back and extend it and stuff.
O RLY? YA RLY.
This is an interesting endorsement. Munroe (of xkcd fame) is typically not very political on his webcomic. I think this just comes to show how important this election is. Maybe this will at least get some of the young, tech-savvy (read: nerdy) vote to reconsider. I get the impression we’ve lost a lot of this demographic to Ron Paul, which is a shame.
Vote!
24 people who mattered in 2007
A nice list of people who mattered (in the technological world) this year. There’s a number of people on that list whose work I follow pretty closely, and a few more that I’ve heard of before. Good stuff.
I’ve got a bug in one of my scripts somewhere that’s saving multiple entries for video files in mythtv’s videometadata table. Here’s a quick hacky SQL query to find them:
SELECT intid, filename, coverfile
FROM videometadata
WHERE filename IN
(SELECT filename FROM
(SELECT filename, COUNT(*) AS ttl FROM videometadata
GROUP BY filename HAVING ttl > 1
) AS tmpfoo
) ORDER BY filename
Maybe one day, I’ll convince myself that using MySQL transactions is a good idea, and start using them whenever I do SQL writes.
In related news, I just NULLed out all my coverdata fields in my mythTV’s mythvideo table. Regenerating all of that crap is really tedious, as the website’s API I use gets flaky after I hit it with a few hundred requests in a matter of minutes. :(
Sigh.
Update:
All fixed. Took ~90 lines of python. Good script to have around anyway: looks for entries with no/missing/invalid coverfile and tries to re-fetch it from thetvdb.com. One day I’ll get all this crap into SVN. I’m getting quite the collection of these handy scripts. :)
I’ve come to the conclusion that I hate all blogging software. It all sucks. None of it fits my specifications, but I’m too lazy/busy to write my own. For now I’m settling with WordPress, but I’m not completely satisfied. I was fairly okay with Movable Type for a while, but I got really frustrated with its admin interface. It’s really slow. And I tried to set up dynamic page loading (lets face it, I’m not getting enough traffic here to take down my shared host), but then realized it wanted to use PHP for some ungodly reason even though the rest of the app is written in Perl. (Yaay consistency!)
I’m hoping I’ll be more satisfied with this WordPress nonsense, but I’m not optimistic. (Well, as optimistic as one can be about a PHP app)
I wish there was a good blogging solution written in Python. Then again, python support on shared hosts isn’t all that great. And likely more work than I’m willing to put forth.
As for the love part of this, OpenID gets my thumbs up. I’ve been a fan since brad first posted about it way back when. I’ve never done much with it until now, though. I get a free identity with my livejournal account, but I’m really not a fan of having my old whiney LJ entries be the face of my identity. So I set up an account with myopenid.com, which seems pretty solid. And I set up dknowles.org to be a delegate for the identity. I’m pretty sure I could do the same thing with LJ serving my identity instead, and dknowles.org would show up in comment posts and what not; but I’m mostly over the LJ thing anyway. I should probably just subscribe my few remaining friends that write there to my google reader.
I installed Leopard on my work Macbook Pro on Friday. Other than some corporate image wonkiness (there’s always something…) it’s been an incredibly pleasant experience thus far.
At first, I wasn’t a huge fan of the new dock UI. It’s growing on me, though. Mostly it’s probably because I made it even smaller than it used to be, so the bright light things that show when an application is open isn’t so offensive anymore. Or maybe I’ve just gotten used to it.
Stacks looked like nice eye candy at first, but not so useful. But I’ve come around. Adding my downloads folder & my applications folder to my dock is a really handy, quick way to access those things. (Of course, I usually default to quicksilver keyboard shortcuts for applications)
Spaces is by far the best virtual desktop implementation I’ve seen yet on mac. It’s still lacking a few things: a keyboard shortcut to move windows to different desktops, proper option+tab app switching behavior, maybe screen edge triggers for moving windows to other desktops. But once I figured out how to have firefox windows on multiple desktops, I was sold. That was always my biggest shortcoming with virtue/desktop manager/any other solution I’ve tried in the past.
The new terminal is wonderful. I spend a lot of my time (at work & at home) inside a terminal session, so having a sane implementation is a godsend. (Although iTerm has become a really good drop-in replacement lately) Built-in tabs, better/more configuration options, proper keybindings are all great additions. The only thing I’ve had a hard time adjusting to is the lack of select-to-copy. Between my linux sessions & the old iTerm functionality, that’s going to be a hard habit to break.
From a work point-of-view, I’m enjoying filevault not taking a shit every few days. I’ve had it want to clean up once or twice now, but it did it automatically and didn’t take very long. It’s about freaking time.
Autofs + kerberos nfs is great. I can access all my work auto shares from terminal without any wacky configuration nightmares. It just works.
Likewise, the new mobile account option is really nifty. Lots of things are auto-syncing from work when I’m there (or VPN’d in) which makes my life easier.
There’s also a lot of small UI tweaks that I really like. An example: the networking pane in system preferences is pretty. Showing a lock next to secured wireless networks in the tray menu. The new finder look. All good things IMO.
Now for my biggest gripe thus far:
The translucent menu bar is really ugly. I really like blue backgrounds (the tiger default background anyone?) and the translucent menu bar turns a pastel blue with this configuration. Bad move, apple. You’ve been known for that glorious grey menu bar for years. Why change now?
All in all, I’m really happy with my Leopard install. Between Apple & Google’s macops team, I’ve got a really awesome cutting-edge OS on my laptop. And it’s pretty. And I like pretty things. :)
