================================ LVM mirroring ================================ Mirror setup ------------ 1. Create pv. :: # pvcreate /dev/sd[bc] Writing physical volume data to disk "/dev/sdb" Physical volume "/dev/sdb" successfully created Writing physical volume data to disk "/dev/sdc" Physical volume "/dev/sdc" successfully created 2. Create vg. :: # vgcreate -s 4m vgmirror /dev/sd[bc] Volume group "vgmirror" successfully created 3. Create mirrored lv. :: # lvcreate -L 4g -m 1 -n mirror vgmirror Logical volume "mirror" created 4. Monitor mirror silvering. :: # lvs vgmirror LV VG Attr LSize [[snip]] Move Log Copy% Convert mirror vgmirror mwi-a-m- 4.00g mirror_mlog 7.23 # lvs vgmirror LV VG Attr LSize [[snip]] Move Log Copy% Convert mirror vgmirror mwi-a-m- 4.00g mirror_mlog 27.15 # lvs vgmirror LV VG Attr LSize [[snip]] Move Log Copy% Convert mirror vgmirror mwi-a-m- 4.00g mirror_mlog 100.00 5. Create filesystem and mount :: # mkfs.ext4 /dev/vgmirror/mirror mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) [[snip]] This filesystem will be automatically checked every 33 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. # mkdir -p -m 755 /mnt/mirror # mount /dev/vgmirror/mirror /mnt/mirror # echo "This is a test of the emergency broadcasting system." > /mnt/mirror/README # md5sum /mnt/mirror/README 9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror/README Splitting the mirror and volume group ------------------------------------- 1. W/filesystem still mounted, split the mirror: :: # lvconvert --splitmirrors 1 --name copy vgmirror/mirror Logical volume mirror converted. # lvs vgmirror -o +devices LV VG Attr LSize [[snip]] Convert Devices copy vgmirror -wi-a--- 4.00g /dev/sdc(0) mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0) 2. Split the volume group: :: # lvchange -a n vgmirror/copy # vgsplit vgmirror vgcopy /dev/sdc New volume group "vgcopy" successfully split from "vgmirror" # vgs -o +devices | grep -v vg00 VG #PV #LV #SN Attr VSize VFree Devices vgcopy 1 1 0 wz--n- 10.00g 6.00g /dev/sdc(0) vgmirror 1 1 0 wz--n- 10.00g 6.00g /dev/sdb(0) # lvs -o +devices | grep -v vg00 LV VG Attr LSize [[snip]] Convert Devices copy vgcopy -wi----- 4.00g /dev/sdc(0) mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0) 3. Activate, and mount the split vg/lv :: # lvchange -a y vgcopy/copy # mkdir -p -m 755 /mnt/mirror1 # mount /dev/vgcopy/copy /mnt/mirror1 # md5sum /mnt/mirror/README /mnt/mirror1/README 9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror/README 9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror1/README Resilver split mirrors and volume groups ---------------------------------------- 1. Unmount/delete split logical volume. Source lv stays mounted. :: # umount /mnt/mirror1 # lvremove -f vgcopy/copy Logical volume "copy" successfully removed # lvs vgmirror vgcopy -o +devices LV VG Attr LSize [[split]] Convert Devices mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0) 2. Merge split volume groups: :: # vgmerge vgmirror vgcopy Volume group "vgcopy" successfully merged into "vgmirror" 3. Remirror logical volume: :: # lvconvert -m 1 vgmirror/mirror vgmirror/mirror: Converted: 0.0% vgmirror/mirror: Converted: 4.3% [[snip]] vgmirror/mirror: Converted: 99.3% vgmirror/mirror: Converted: 100.0% # vgs VG #PV #LV #SN Attr VSize VFree vg00 1 2 0 wz--n- 19.51g 0 vgmirror 2 1 0 wz--n- 19.99g 11.99g # lvs LV VG Attr LSize [[snip]] Move Log Copy% Convert root vg00 -wi-ao-- 17.54g swap vg00 -wi-ao-- 1.97g mirror vgmirror mwi-aom- 4.00g mirror_mlog 100.00 Error conditions: simulated disk failure ----------------------------------------- 1. Shutdown guest, delete one of the disks, and restart guest :: # virsh destroy guest Domain guest destroyed # virsh domblklist guest Target Source ------------------------------------------------ hda /var/lib/libvirt/images/guest.img hdb /var/lib/libvirt/images/guest-1.img hdc /var/lib/libvirt/images/guest-2.img # virsh detach-disk guest hdb --persistent Disk detached successfully # virsh domblklist guest Target Source ------------------------------------------------ hda /var/lib/libvirt/images/guest.img hdc /var/lib/libvirt/images/guest-2.img # virsh start guest Domain guest started 2. Check for mounted filesystem, errors: :: # h guest # mount | grep mirror # mount /dev/vgmirror/mirror /mnt/mirror mount: you must specify the filesystem type # lvdisplay /dev/vgmirror/mirror **Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.** --- Logical volume --- LV Path /dev/vgmirror/mirror LV Name mirror VG Name vgmirror LV UUID TXwTkn-umnN-4Xh3-OuAE-SDJn-0Mh4-ELqIeT LV Write Access read/write LV Creation host, time guest, 2013-03-24 08:57:35 -0500 **LV Status NOT available** LV Size 4.00 GiB Current LE 1024 Mirrored volumes 2 **Segments 1** Allocation inherit Read ahead sectors auto # vgdisplay vgmirror Couldn't find device with uuid DzqgRx-kwyZ-kp6O-xSSt-MO3m-sDiq-v9baRp. --- Volume group --- VG Name vgmirror System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 29 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 **Cur PV 2** **Act PV 1** VG Size 19.99 GiB PE Size 4.00 MiB Total PE 5118 Alloc PE / Size 2049 / 8.00 GiB Free PE / Size 3069 / 11.99 GiB VG UUID 0sof6K-R7ub-8mtG-3jYJ-xXzP-RYCD-6nDRPS # lvchange -a y vgmirror/mirror Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt. Refusing activation of partial LV mirror. Use --partial to override. # lvchange -a y --partial vgmirror/mirror Partial mode. Incomplete logical volumes will be processed. Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt. # mount /dev/vgmirror/mirror /mnt/mirror # mount | grep mirror /dev/mapper/vgmirror-mirror on /mnt/mirror type ext4 (rw) # pvs **Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.** PV VG Fmt Attr PSize PFree /dev/sda2 vg00 lvm2 a-- 19.51g 0 /dev/sdb vgmirror lvm2 a-- 10.00g 6.00g **unknown device vgmirror lvm2 a-m 10.00g 10.00g** 3. Fix the volume group: :: # vgreduce --removemissing --force vgmirror Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt. Wrote out consistent volume group vgmirror # vgdisplay vgmirror --- Volume group --- VG Name vgmirror System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 26 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 10.00 GiB PE Size 4.00 MiB Total PE 2559 Alloc PE / Size 1024 / 4.00 GiB Free PE / Size 1535 / 6.00 GiB VG UUID 0sof6K-R7ub-8mtG-3jYJ-xXzP-RYCD-6nDRPS # lvdisplay /dev/vgmirror/mirror --- Logical volume --- LV Path /dev/vgmirror/mirror LV Name mirror VG Name vgmirror LV UUID TXwTkn-umnN-4Xh3-OuAE-SDJn-0Mh4-ELqIeT LV Write Access read/write LV Creation host, time guest, 2013-03-24 08:57:35 -0500 LV Status available # open 1 LV Size 4.00 GiB Current LE 1024 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:6 4. Replace disk, remirror logical volume: :: # virsh destroy guest Domain guest destroyed # virsh domblklist guest Target Source ------------------------------------------------ hda /var/lib/libvirt/images/guest.img hdc /var/lib/libvirt/images/guest-2.img # virsh vol-create-as default guest-3.img 10g Vol guest-3.img created # virsh attach-disk guest /var/lib/libvirt/images/guest-3.img hdd --persistent Disk attached successfully # virsh domblklist guest Target Source ------------------------------------------------ hda /var/lib/libvirt/images/guest.img hdc /var/lib/libvirt/images/guest-2.img hdd /var/lib/libvirt/images/guest-3.img # virsh start guest Domain guest started # ssh guest Last login: Sun Mar 24 10:13:12 2013 from 192.168.122.1 # grep sd /proc/partitions | sort -k 4 8 0 20971520 sda 8 1 512000 sda1 8 2 20458496 sda2 8 16 10485760 sdb 8 32 10485760 sdc # pvs PV VG Fmt Attr PSize PFree /dev/sda2 vg00 lvm2 a-- 19.51g 0 /dev/sdb vgmirror lvm2 a-- 10.00g 6.00g # pvcreate /dev/sdc Writing physical volume data to disk "/dev/sdc" Physical volume "/dev/sdc" successfully created # vgextend vgmirror /dev/sdc Volume group "vgmirror" successfully extended # lvconvert -m 1 vgmirror/mirror vgmirror/mirror: Converted: 0.0% vgmirror/mirror: Converted: 2.2% vgmirror/mirror: Converted: 4.3% [[snip]] vgmirror/mirror: Converted: 96.2% vgmirror/mirror: Converted: 98.3% vgmirror/mirror: Converted: 100.0% # lvdisplay vgmirror/mirror | grep -i mirrored Mirrored volumes 2 # vgdisplay vgmirror | grep -i pv Max PV 0 Cur PV 2 Act PV 2