Josh Bryan

January 2, 2008

LVM2 Mirrors vs. MD Raid 1

Filed under: Projects,Software — jbryan @ 5:19 pm

I recently added a new hard drive to my desktop and reorganized the partition structure. In doing so I wanted my /home partition to be mirrored for safety, but I also wanted to use lvm to make managing my multiple partitions less painful. In my research, I found there are basically two ways to do this. One is to create a linux RAID1 device from the two drives and then use that device as a physical extent in a lvm volume group (here is a basic tutorial). The other method is to us LVM2′s mirroring capabilities. I personally would rather use the latter solution if there is no performance impact since moving the partitions around later would be much easier, however what I read gave the impression that LVM2 mirroring would be slower.

To test this, I setup a quick informal benchmark. I created 2 partitions on each of two drives and set one up as plain raid1 (no lvm) and the other as lvm with a mirror. I then ran bonnie++ on each of the partitions. The results were surprising. MD RAID1 gave about 25MB/sec write and 50MB/sec read. However LVM2 mirrored gave about 50MB/sec write and 55MB/sec read. Since the test was not perfectly sterile (there were other processes running at the same time) I would be willing to give these number +/- 10MB/sec. However still, LVM write was considerably faster than MD RAID1. I so far have no explanation for this, but will test further. If anyone has an explanation, I’d be happy to hear.

16 Comments »

  1. Thanks for the comparison. I too am having the same debate. If LVM is just as fast or faster, there seems to be no question which is the better solution. Have you noticed any issues since this post?

    Comment by JP — May 12, 2008 @ 10:25 am

  2. Hi,
    I also made some comparisons, but under Ubuntu Hardy.
    My first results showed almost the same throughput for LVM-mirror and MD-RAID1 with a slight better performance on LVM-mirror (but only 1-2 MB/sec).
    But then I tested with _two_ files simultaneously on the same filesystem and there I found a much better performance on read with MD-RAID1 (about 1.5 times of LVM-mirror or no mirror).
    The explanation is: MD always reads simultaneously from both disks, whereas LVM-mirror only reads from one mirror and uses the second mirror only in case of failure!
    That’s a poor design flaw in my opinion. :-(

    This design flaw and the fact, that on my system mirror failures were not recognized correctly, makes LVM-mirror useless for me. I suspect one needs dmeventd to recognize mirror failures correctly, but this is not yet available in Ubuntu and I don’t want to go into long compilation sessions.

    It’s a pity, because I like the idea of LVM-mirroring more than the MD-mirroring.

    Comment by Markus — July 2, 2008 @ 1:17 am

  3. LVM is not safe in a power failure, it does not respect write barriers and pass those down to the lower drives.

    hence, it is often faster than MD by default, but to be safe you would have to turn off your drive’s write caches, which ends up making it slower than if you used write barriers.

    http://lwn.net/Articles/283161/

    “Unless, of course, their filesystem is based on an LVM volume (as certain distributions do by default); it turns out that the device mapper code does not pass through or honor barriers.”

    Comment by Scott C. — October 2, 2008 @ 3:30 pm

  4. How does RAID behave in case of presence of an intermittently connected storage, and in case of hot unplug of the same on a booted system ?

    Comment by Prateek Donni — January 21, 2009 @ 12:06 am

  5. [...] Der Grund ist einfach: über ein Software-RAID erhalte ich höhere Leseraten (siehe auch: http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/, insbesondere auch die Kommentare). Beim Dateisystem setze ich auf ext3 bzw. ext2 für [...]

    Pingback by Der neue Server: Teil 1 Installation und Basiskonfiguration | johker’s blog — March 17, 2009 @ 1:32 pm

  6. Definitely follow the advice from post #3

    Barriers are lost in devicemapper layer, so unless you use a filesystem right on top of the md (or DR:BD) devices you are in this situation.

    [ ] Faster
    [ ] Safe
    chose one.

    I’ll add some moreo n that in my blog and link back here

    Comment by darkfader — April 4, 2011 @ 5:09 pm

  7. [...] You can find some nice article about clvm mirroring here http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1 [...]

    Pingback by Linux LVM mirroring comes at a price | deranfangvomen.de — April 4, 2011 @ 5:31 pm

  8. Update: write barriers are fully supported in linux kernel 2.6.33 according to : http://en.wikipedia.org/wiki/Logical_Volume_Manager_%28Linux%29

    Comment by gwpl — July 25, 2011 @ 5:35 am

  9. [...] http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/ [...]

    Pingback by LVM mirroring VS RAID1 - Admins Goodies — August 21, 2011 @ 2:24 am

  10. [...] ????? ????????? ?????????? ????? ????? ? ????????? (????????: http://serverfault.com/questions/126851/linux-lvm-mirror-vs-md-mirror ??? http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/). [...]

    Pingback by ???????? ? mdadm ? lvm | Just another IT blog… — October 18, 2011 @ 2:12 am

  11. [...] dynamic partition management opened. While older kernels make LVM inferior to RAID1, as pointed out here, modern kernels make RAID1 and LVM mirroring on par, at least I could not find any relevant [...]

    Pingback by Mini-ITX home server – part two, software | amichalec.net :: homepage — August 7, 2012 @ 6:33 am

  12. [...] http://www.joshbryan.com/blog/2008/01/02/lvm2-mirrors-vs-md-raid-1/ [...]

    Pingback by LVM on software RAID - Just just easy answers — September 6, 2013 @ 2:23 am

  13. Save yourself the hassle of moving, plus Fri day and enjoy it.
    The above talked about three methods to move Favorites in Internet Explorer 9.

    There are tanks and a lot of men to the left, and you need to let the friendly tanks take
    care of them.

    Feel free to visit my web-site – moving companies denver

    Comment by moving companies denver — August 13, 2014 @ 7:50 am

  14. The process of implementing a social media campaign can be daunting.
    The best examples of the changing marketing environment right now probably can be found in the
    movie industry, with such films as “Snakes on a Plane”
    and “Cloverfield” being marketed exclusively through the Internet and relying to a great degree on social networking websites spreading the word.

    After so much spent or so many visits, offer a free
    product or percent off.

    my web page: palm beach state marketing

    Comment by palm beach state marketing — August 16, 2014 @ 5:53 pm

  15. Everyone wants cheap bait these days but the big problem is that cheap often means rubbish.
    After users select their avatar they’re transported into their apartment.
    A camera and photo album are another way for a family to record their boating trips.

    Visit my weblog dock box for sale used

    Comment by dock box for sale used — August 31, 2014 @ 12:39 am

  16. You need to get your boxes early but be aware that even flat they take up
    a lot of space. The above talked about three methods to move Favorites in Internet Explorer 9.
    The expense apart from covering the vehicle also involves in spending for the moving boxes.

    Visit my homepage … bekins moving and storage seattle

    Comment by bekins moving and storage seattle — September 6, 2014 @ 12:34 am

RSS feed for comments on this post. TrackBack URL

Leave a comment