I currently have two raid volumes in my home server. They're running on four disks, which are partitioned thusly:
root@bender:~# fdisk -l /dev/sd[abcd]
Disk /dev/sda: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sda1 1 48641 390708801 83 Linux
Disk /dev/sdb: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 1 48641 390708801 83 Linux
Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdc1 1 48641 390708801 83 Linux
/dev/sdc2 48642 60801 97675200 83 Linux
Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xb7e306eb
Device Boot Start End Blocks Id System
/dev/sdd1 1 48641 390708832 83 Linux
/dev/sdd2 48642 60801 97675200 83 Linux
Using the partitions as raid targets, I end up with two raid volumes:
md0 : active raid5 sdd1[3] sda1[0] sdc1[2] sdb1[1]
md1 : active raid1 sdd2[1] sdc2[0]
On top of these raid volumes, I'm running an LVM so that they can be easily resized:
root@bender:~# pvdisplay
--- Physical volume ---
PV Name /dev/md1
VG Name vgraid1
PV Size 93.15 GB / not usable 9.88 MB
Allocatable yes (but full)
PE Size (KByte) 16384
Total PE 5961
Free PE 0
Allocated PE 5961
PV UUID NX3sku-sNGU-yqyw-3peE-Bd8W-KjBo-xsUol8
--- Physical volume ---
PV Name /dev/md0
VG Name vgraid
PV Size 1.09 TB / not usable 14.31 MB
Allocatable yes (but full)
PE Size (KByte) 16384
Total PE 71540
Free PE 0
Allocated PE 71540
PV UUID wtYFqQ-499L-BbuB-zaRb-EV16-Y5sT-ZcG65b
And now, the fun stuff. Here's how the original raid5 volume was made (which was originally missing a 500G disk, because it died on me when I first got it):
root@bender:~# mdadm --create --verbose /dev/md0 --chunk=128 \
--level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1
root@bender:~# pvcreate /dev/md0
root@bender:~# vgcreate -s 16M vgraid /dev/md0
root@bender:~# lvcreate -l 47693 vgraid -n bigdisk
root@bender:~# mkfs.xfs /dev/vgraid/bigdisk
[note: this was done a while back, so i"ve lost the output. for output example see below when i create my raid1 array]
I finally received my new 500G drive from RMA, so I had to resize the raid5 to include the new disk. Here's how it's done.
First, create the new partitions to match the size on the other drive. (see above) Then, resize the raid array (takes a while...I started it then went to sleep, then it was magically done when I woke up this morning)
root@bender:~# mdadm --add /dev/md0 /dev/sdd1
mdadm: added /dev/sdd1
root@bender:~# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Fri Mar 21 09:52:31 2008
Raid Level : raid5
Array Size : 781417472 (745.22 GiB 800.17 GB)
Used Dev Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 3
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sun Apr 20 23:28:58 2008
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender)
Events : 0.17564
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 - spare /dev/sdd1
root@bender:~# mdadm --grow /dev/md0 --raid-devices=4
mdadm: Need to backup 384K of critical section..
mdadm: ... critical section passed.
root@bender:~# mdadm --detail /dev/md0
/dev/md0:
Version : 00.91.03
Creation Time : Fri Mar 21 09:52:31 2008
Raid Level : raid5
Array Size : 781417472 (745.22 GiB 800.17 GB)
Used Dev Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sun Apr 20 23:29:48 2008
State : clean, recovering
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
Reshape Status : 0% complete
Delta Devices : 1, (3->4)
UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender)
Events : 0.17680
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1
root@bender:~# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Fri Mar 21 09:52:31 2008
Raid Level : raid5
Array Size : 1172126208 (1117.83 GiB 1200.26 GB)
Used Dev Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Mon Apr 21 09:15:38 2008
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
UUID : 2621450e:bb4f9269:a0a0eea7:e933aaa5 (local to host bender)
Events : 0.277372
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1
root@bender:~# pvresize /dev/md0
Physical volume "/dev/md0" changed
1 physical volume(s) resized / 0 physical volume(s) not resized
root@bender:~# vgdisplay
--- Volume group ---
VG Name vgraid
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 1.09 TB
PE Size 16.00 MB
Total PE 71540
Alloc PE / Size 47693 / 745.20 GB
Free PE / Size 23847 / 372.61 GB
VG UUID JPEInC-5bNp-f0iC-3611-qIIn-Drhd-A3xInG
root@bender:~# lvextend -l +23847 /dev/vgraid/bigdisk
Extending logical volume bigdisk to 1.09 TB
Logical volume bigdisk successfully resized
root@bender:~# xfs_growfs /dev/vgraid/bigdisk
meta-data=/dev/mapper/vgraid-bigdisk isize=256 agcount=32, agsize=6104704 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=195350528, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 195350528 to 293027840
root@bender:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 73G 63G 6.2G 91% /
varrun 1.9G 264K 1.9G 1% /var/run
varlock 1.9G 0 1.9G 0% /var/lock
udev 1.9G 120K 1.9G 1% /dev
devshm 1.9G 0 1.9G 0% /dev/shm
/dev/sde1 280G 271G 8.6G 97% /export/myth
/dev/mapper/vgraid-bigdisk
1.1T 453G 665G 41% /export/archive
Since I now have my two 500G disks, I have an extra 100G on each (since I made all the partitions match to raid5 with the 400G disks) so I can raid1 those for a wonderfully redundant backup array.
root@bender:~# mdadm --create --verbose /dev/md1 --chunk=128 \
--level=1 --raid-devices=2 /dev/sdc2 /dev/sdd2
mdadm: chunk size ignored for this level
mdadm: size set to 97675136K
mdadm: array /dev/md1 started.
root@bender:~# pvcreate /dev/md1
Physical volume "/dev/md1" successfully created
root@bender:~# vgcreate -s 16M vgraid1 /dev/md1
Volume group "vgraid1" successfully created
root@bender:~# vgdisplay
--- Volume group ---
VG Name vgraid1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 93.14 GB
PE Size 16.00 MB
Total PE 5961
Alloc PE / Size 0 / 0
Free PE / Size 5961 / 93.14 GB
VG UUID Cxrf05-HaUJ-sPHx-Ai9P-eRYL-Ofoz-mnIsFx
--- Volume group ---
VG Name vgraid
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 1.09 TB
PE Size 16.00 MB
Total PE 71540
Alloc PE / Size 71540 / 1.09 TB
Free PE / Size 0 / 0
VG UUID JPEInC-5bNp-f0iC-3611-qIIn-Drhd-A3xInG
root@bender:~# lvcreate -l 5961 vgraid1 -n backup
Logical volume "backup" created
root@bender:~# mkfs.xfs /dev/vgraid1/backup
meta-data=/dev/vgraid1/backup isize=256 agcount=16, agsize=1526016 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=24416256, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=11922, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
root@bender:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 73G 63G 6.2G 91% /
varrun 1.9G 264K 1.9G 1% /var/run
varlock 1.9G 0 1.9G 0% /var/lock
udev 1.9G 128K 1.9G 1% /dev
devshm 1.9G 0 1.9G 0% /dev/shm
/dev/sde1 280G 271G 8.6G 97% /export/myth
/dev/mapper/vgraid-bigdisk
1.1T 453G 665G 41% /export/archive
/dev/mapper/vgraid1-backup
94G 544K 94G 1% /export/backup
Huzzah! Less likeliness of lost data due to my stupidity! Now to write some scripts to do automatic backups. :)
0 comments:
Post a Comment