LVM mirroring¶
Mirror setup¶
Create pv.
# pvcreate /dev/sd[bc]
Writing physical volume data to disk "/dev/sdb"
Physical volume "/dev/sdb" successfully created
Writing physical volume data to disk "/dev/sdc"
Physical volume "/dev/sdc" successfully created
Create vg.
# vgcreate -s 4m vgmirror /dev/sd[bc]
Volume group "vgmirror" successfully created
Create mirrored lv.
# lvcreate -L 4g -m 1 -n mirror vgmirror
Logical volume "mirror" created
Monitor mirror silvering.
# lvs vgmirror
LV VG Attr LSize [[snip]] Move Log Copy% Convert
mirror vgmirror mwi-a-m- 4.00g mirror_mlog 7.23
# lvs vgmirror
LV VG Attr LSize [[snip]] Move Log Copy% Convert
mirror vgmirror mwi-a-m- 4.00g mirror_mlog 27.15
# lvs vgmirror
LV VG Attr LSize [[snip]] Move Log Copy% Convert
mirror vgmirror mwi-a-m- 4.00g mirror_mlog 100.00
Create filesystem and mount
# mkfs.ext4 /dev/vgmirror/mirror
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
[[snip]]
This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
# mkdir -p -m 755 /mnt/mirror
# mount /dev/vgmirror/mirror /mnt/mirror
# echo "This is a test of the emergency broadcasting system." > /mnt/mirror/README
# md5sum /mnt/mirror/README
9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror/README
Splitting the mirror and volume group¶
W/filesystem still mounted, split the mirror:
# lvconvert --splitmirrors 1 --name copy vgmirror/mirror
Logical volume mirror converted.
# lvs vgmirror -o +devices
LV VG Attr LSize [[snip]] Convert Devices
copy vgmirror -wi-a--- 4.00g /dev/sdc(0)
mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0)
Split the volume group:
# lvchange -a n vgmirror/copy
# vgsplit vgmirror vgcopy /dev/sdc
New volume group "vgcopy" successfully split from "vgmirror"
# vgs -o +devices | grep -v vg00
VG #PV #LV #SN Attr VSize VFree Devices
vgcopy 1 1 0 wz--n- 10.00g 6.00g /dev/sdc(0)
vgmirror 1 1 0 wz--n- 10.00g 6.00g /dev/sdb(0)
# lvs -o +devices | grep -v vg00
LV VG Attr LSize [[snip]] Convert Devices
copy vgcopy -wi----- 4.00g /dev/sdc(0)
mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0)
Activate, and mount the split vg/lv
# lvchange -a y vgcopy/copy
# mkdir -p -m 755 /mnt/mirror1
# mount /dev/vgcopy/copy /mnt/mirror1
# md5sum /mnt/mirror/README /mnt/mirror1/README
9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror/README
9a75a8122c57b3ae0e4831d30b744593 /mnt/mirror1/README
Resilver split mirrors and volume groups¶
Unmount/delete split logical volume. Source lv stays mounted.
# umount /mnt/mirror1
# lvremove -f vgcopy/copy
Logical volume "copy" successfully removed
# lvs vgmirror vgcopy -o +devices
LV VG Attr LSize [[split]] Convert Devices
mirror vgmirror -wi-ao-- 4.00g /dev/sdb(0)
Merge split volume groups:
# vgmerge vgmirror vgcopy
Volume group "vgcopy" successfully merged into "vgmirror"
Remirror logical volume:
# lvconvert -m 1 vgmirror/mirror
vgmirror/mirror: Converted: 0.0%
vgmirror/mirror: Converted: 4.3%
[[snip]]
vgmirror/mirror: Converted: 99.3%
vgmirror/mirror: Converted: 100.0%
# vgs
VG #PV #LV #SN Attr VSize VFree
vg00 1 2 0 wz--n- 19.51g 0
vgmirror 2 1 0 wz--n- 19.99g 11.99g
# lvs
LV VG Attr LSize [[snip]] Move Log Copy% Convert
root vg00 -wi-ao-- 17.54g
swap vg00 -wi-ao-- 1.97g
mirror vgmirror mwi-aom- 4.00g mirror_mlog 100.00
Error conditions: simulated disk failure¶
Shutdown guest, delete one of the disks, and restart guest
# virsh destroy guest
Domain guest destroyed
# virsh domblklist guest
Target Source
------------------------------------------------
hda /var/lib/libvirt/images/guest.img
hdb /var/lib/libvirt/images/guest-1.img
hdc /var/lib/libvirt/images/guest-2.img
# virsh detach-disk guest hdb --persistent
Disk detached successfully
# virsh domblklist guest
Target Source
------------------------------------------------
hda /var/lib/libvirt/images/guest.img
hdc /var/lib/libvirt/images/guest-2.img
# virsh start guest
Domain guest started
Check for mounted filesystem, errors:
# h
guest
# mount | grep mirror
# mount /dev/vgmirror/mirror /mnt/mirror
mount: you must specify the filesystem type
# lvdisplay /dev/vgmirror/mirror
**Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.**
--- Logical volume ---
LV Path /dev/vgmirror/mirror
LV Name mirror
VG Name vgmirror
LV UUID TXwTkn-umnN-4Xh3-OuAE-SDJn-0Mh4-ELqIeT
LV Write Access read/write
LV Creation host, time guest, 2013-03-24 08:57:35 -0500
**LV Status NOT available**
LV Size 4.00 GiB
Current LE 1024
Mirrored volumes 2
**Segments 1**
Allocation inherit
Read ahead sectors auto
# vgdisplay vgmirror
Couldn't find device with uuid DzqgRx-kwyZ-kp6O-xSSt-MO3m-sDiq-v9baRp.
--- Volume group ---
VG Name vgmirror
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 29
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
**Cur PV 2**
**Act PV 1**
VG Size 19.99 GiB
PE Size 4.00 MiB
Total PE 5118
Alloc PE / Size 2049 / 8.00 GiB
Free PE / Size 3069 / 11.99 GiB
VG UUID 0sof6K-R7ub-8mtG-3jYJ-xXzP-RYCD-6nDRPS
# lvchange -a y vgmirror/mirror
Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.
Refusing activation of partial LV mirror. Use --partial to override.
# lvchange -a y --partial vgmirror/mirror
Partial mode. Incomplete logical volumes will be processed.
Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.
# mount /dev/vgmirror/mirror /mnt/mirror
# mount | grep mirror
/dev/mapper/vgmirror-mirror on /mnt/mirror type ext4 (rw)
# pvs
**Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.**
PV VG Fmt Attr PSize PFree
/dev/sda2 vg00 lvm2 a-- 19.51g 0
/dev/sdb vgmirror lvm2 a-- 10.00g 6.00g
**unknown device vgmirror lvm2 a-m 10.00g 10.00g**
Fix the volume group:
# vgreduce --removemissing --force vgmirror
Couldn't find device with uuid WupKj6-4auC-H3Qg-xdK2-Fb8g-ttKJ-XMQEEt.
Wrote out consistent volume group vgmirror
# vgdisplay vgmirror
--- Volume group ---
VG Name vgmirror
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 26
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 10.00 GiB
PE Size 4.00 MiB
Total PE 2559
Alloc PE / Size 1024 / 4.00 GiB
Free PE / Size 1535 / 6.00 GiB
VG UUID 0sof6K-R7ub-8mtG-3jYJ-xXzP-RYCD-6nDRPS
# lvdisplay /dev/vgmirror/mirror
--- Logical volume ---
LV Path /dev/vgmirror/mirror
LV Name mirror
VG Name vgmirror
LV UUID TXwTkn-umnN-4Xh3-OuAE-SDJn-0Mh4-ELqIeT
LV Write Access read/write
LV Creation host, time guest, 2013-03-24 08:57:35 -0500
LV Status available
# open 1
LV Size 4.00 GiB
Current LE 1024
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:6
Replace disk, remirror logical volume:
# virsh destroy guest
Domain guest destroyed
# virsh domblklist guest
Target Source
------------------------------------------------
hda /var/lib/libvirt/images/guest.img
hdc /var/lib/libvirt/images/guest-2.img
# virsh vol-create-as default guest-3.img 10g
Vol guest-3.img created
# virsh attach-disk guest /var/lib/libvirt/images/guest-3.img hdd --persistent
Disk attached successfully
# virsh domblklist guest
Target Source
------------------------------------------------
hda /var/lib/libvirt/images/guest.img
hdc /var/lib/libvirt/images/guest-2.img
hdd /var/lib/libvirt/images/guest-3.img
# virsh start guest
Domain guest started
# ssh guest
Last login: Sun Mar 24 10:13:12 2013 from 192.168.122.1
# grep sd /proc/partitions | sort -k 4
8 0 20971520 sda
8 1 512000 sda1
8 2 20458496 sda2
8 16 10485760 sdb
8 32 10485760 sdc
# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 vg00 lvm2 a-- 19.51g 0
/dev/sdb vgmirror lvm2 a-- 10.00g 6.00g
# pvcreate /dev/sdc
Writing physical volume data to disk "/dev/sdc"
Physical volume "/dev/sdc" successfully created
# vgextend vgmirror /dev/sdc
Volume group "vgmirror" successfully extended
# lvconvert -m 1 vgmirror/mirror
vgmirror/mirror: Converted: 0.0%
vgmirror/mirror: Converted: 2.2%
vgmirror/mirror: Converted: 4.3%
[[snip]]
vgmirror/mirror: Converted: 96.2%
vgmirror/mirror: Converted: 98.3%
vgmirror/mirror: Converted: 100.0%
# lvdisplay vgmirror/mirror | grep -i mirrored
Mirrored volumes 2
# vgdisplay vgmirror | grep -i pv
Max PV 0
Cur PV 2
Act PV 2