Josh Bryan

January 2, 2008

LVM2 Mirrors vs. MD Raid 1

Filed under: Projects,Software — jbryan @ 5:19 pm

I recently added a new hard drive to my desktop and reorganized the partition structure. In doing so I wanted my /home partition to be mirrored for safety, but I also wanted to use lvm to make managing my multiple partitions less painful. In my research, I found there are basically two ways to do this. One is to create a linux RAID1 device from the two drives and then use that device as a physical extent in a lvm volume group (here is a basic tutorial). The other method is to us LVM2’s mirroring capabilities. I personally would rather use the latter solution if there is no performance impact since moving the partitions around later would be much easier, however what I read gave the impression that LVM2 mirroring would be slower.

To test this, I setup a quick informal benchmark. I created 2 partitions on each of two drives and set one up as plain raid1 (no lvm) and the other as lvm with a mirror. I then ran bonnie++ on each of the partitions. The results were surprising. MD RAID1 gave about 25MB/sec write and 50MB/sec read. However LVM2 mirrored gave about 50MB/sec write and 55MB/sec read. Since the test was not perfectly sterile (there were other processes running at the same time) I would be willing to give these number +/- 10MB/sec. However still, LVM write was considerably faster than MD RAID1. I so far have no explanation for this, but will test further. If anyone has an explanation, I’d be happy to hear.

12 Comments »

  1. Thanks for the comparison. I too am having the same debate. If LVM is just as fast or faster, there seems to be no question which is the better solution. Have you noticed any issues since this post?

    Comment by JP — May 12, 2008 @ 10:25 am

  2. Hi,
    I also made some comparisons, but under Ubuntu Hardy.
    My first results showed almost the same throughput for LVM-mirror and MD-RAID1 with a slight better performance on LVM-mirror (but only 1-2 MB/sec).
    But then I tested with _two_ files simultaneously on the same filesystem and there I found a much better performance on read with MD-RAID1 (about 1.5 times of LVM-mirror or no mirror).
    The explanation is: MD always reads simultaneously from both disks, whereas LVM-mirror only reads from one mirror and uses the second mirror only in case of failure!
    That’s a poor design flaw in my opinion. :-(

    This design flaw and the fact, that on my system mirror failures were not recognized correctly, makes LVM-mirror useless for me. I suspect one needs dmeventd to recognize mirror failures correctly, but this is not yet available in Ubuntu and I don’t want to go into long compilation sessions.

    It’s a pity, because I like the idea of LVM-mirroring more than the MD-mirroring.

    Comment by Markus — July 2, 2008 @ 1:17 am

  3. LVM is not safe in a power failure, it does not respect write barriers and pass those down to the lower drives.

    hence, it is often faster than MD by default, but to be safe you would have to turn off your drive’s write caches, which ends up making it slower than if you used write barriers.

    http://lwn.net/Articles/283161/

    “Unless, of course, their filesystem is based on an LVM volume (as certain distributions do by default); it turns out that the device mapper code does not pass through or honor barriers.”

    Comment by Scott C. — October 2, 2008 @ 3:30 pm

  4. How does RAID behave in case of presence of an intermittently connected storage, and in case of hot unplug of the same on a booted system ?

    Comment by Prateek Donni — January 21, 2009 @ 12:06 am

  5. […] Der Grund ist einfach: über ein Software-RAID erhalte ich höhere Leseraten (siehe auch: http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/, insbesondere auch die Kommentare). Beim Dateisystem setze ich auf ext3 bzw. ext2 für […]

    Pingback by Der neue Server: Teil 1 Installation und Basiskonfiguration | johker’s blog — March 17, 2009 @ 1:32 pm

  6. Definitely follow the advice from post #3

    Barriers are lost in devicemapper layer, so unless you use a filesystem right on top of the md (or DR:BD) devices you are in this situation.

    [ ] Faster
    [ ] Safe
    chose one.

    I’ll add some moreo n that in my blog and link back here

    Comment by darkfader — April 4, 2011 @ 5:09 pm

  7. […] You can find some nice article about clvm mirroring here http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1 […]

    Pingback by Linux LVM mirroring comes at a price | deranfangvomen.de — April 4, 2011 @ 5:31 pm

  8. Update: write barriers are fully supported in linux kernel 2.6.33 according to : http://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29

    Comment by gwpl — July 25, 2011 @ 5:35 am

  9. […] http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/ […]

    Pingback by LVM mirroring VS RAID1 - Admins Goodies — August 21, 2011 @ 2:24 am

  10. […] ????? ????????? ?????????? ????? ????? ? ????????? (????????: http://serverfault.com/questions/126851/linux-lvm-mirror-vs-md-mirror ??? http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/). […]

    Pingback by ???????? ? mdadm ? lvm | Just another IT blog… — October 18, 2011 @ 2:12 am

  11. […] dynamic partition management opened. While older kernels make LVM inferior to RAID1, as pointed out here, modern kernels make RAID1 and LVM mirroring on par, at least I could not find any relevant […]

    Pingback by Mini-ITX home server – part two, software | amichalec.net :: homepage — August 7, 2012 @ 6:33 am

  12. […] http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/ […]

    Pingback by LVM on software RAID - Just just easy answers — September 6, 2013 @ 2:23 am

RSS feed for comments on this post. TrackBack URL

Leave a comment