2 Comments

5TB LVM Volume with an LSI 9265-8i RAID Controller

This article outlines how to get a 5TB LVM volume created with an LSI 9265-8i RAID controller.

Background

RAID Array

I’ve been running software RAID for a while. Specifically, I’ve got an ASUS P6T Deluxe V2 motherboard with 6 SATA ports. Up until now, I’ve had 1 SATA connected to a single 1 TB drive with the Fedora OS on it, one to a SATA DVD/Blu-ray drive, and the other 4 to a 4x1TB software RAID 5. This has worked great. When I started to max that out, I had a decision to make. It seems I could either:

  • Continue with the small array and just continue to increase the disk size.  This is easiest, but given that 4 disks in RAID 5 give you a 25% loss of storage space (i.e., 3 used, 1 for parity), you have to buy bigger disks and the biggest ones usually cost the most.
  • Make the 1-time investment to get an 8-port RAID card and grow the array with disks that are large, but not necessarily the largest out there.

I decided latter made more sense for me and went with the LSI 9265-8i based on various reviews.  My plan was to build a 6x1TB SATA array (5TB storage) with 2 available ports on which I could add 2 additional drives when/if needed.

The Frustration

I had two major issues right away.  The first was with WEBBIOS.  It’s a BIOS utility that’s supposed to allow you to configure your card as your system boots up by pressing CTRL+H when prompted.  Unfortunately for me, every time I would press it, the prompt would disappear and my system would boot normally.  This is still an issue, but the rest of the post works around it.

The second issue is that I couldn’t get the MegaRAID software installed in Fedora.  Again, this is supposed to be a bunch of tools that allow me to control my card, but instead it just gave me a lot of headaches with no results.  Every time I would try to install the software, it would complain about missing dependencies, etc.  This is also still an issue, but I have no work around.  (If you have ideas, I’m all ears.)

Getting It To Work Without WEBBIOS

At the same time you’re prompted to press CTRL+H, you’re also given the option to press CTRL+Y to go into the Preboot CLI (command line interface).  I thought, “hooray, I’m good at the command line.”  I soon found out it’s the most cryptic, undocumented CLI ever.  However, it does work.  Here’s what I did:

Press CTRL-Y When Prompted

This will take you into the CLI.  Note, that you’ll probably want to throw “-page20” at the end of all of the commands below.  This will make them only output 20 lines at a time, which is the only way to make sense of the ton of output it gives.

Check Out What’s Going On

Before building your array, you’ll want to check out the how things look before you touch them.  Unless you’ve got more than 1 card, your adapter number is 0.  You’ll see -a0 a lot in the lines below.

-AdpAllInfo -a0

That’s the mother lode.  It shows you EVERYTHING.  Take note of your enclosure devise number.  Mine is 252.  I’ll use 252 for the rest of the article, but you should use your number.

-PDList -a0

This will give you detailed info on each drive you have.  I recommend you use the serial numbers shown along with the slot numbers to physically label your cables if you haven’t already.

Build the Array

Once you’ve collected all the info you need, build the array.

-CfgLdAdd -r5 [252:0, 252:1, 252:2, 252:3, 252:4, 252:5] -a0
  • -r5 means RAID 5
  • :0, :1, :2, etc., are the ports my drives are on
  • 252 is my enclosure device number
  • 0 is my adapter

Tada!  You’re ready to get going in your OS.  (HUGE thanks to Moritz Mertinkat for putting together this cheat sheet, which helped quite a bit.)

Creating a 5TB LVM

The bottom line here is: don’t create a partition for a volume this size.  You’ll find out that it maxes out a 2TB and you’ll be left scratching your head.  Instead, we’re just going to create the standard GUID Partition Table layout, but not actually put any sort of actual partition on it.

# parted
(parted) mklabel gpt
(parted) mkpart primary 0 4998GB
[Ignore the warning you get here.  That's expected and ok.]
(parted) quit

Create the Physical Volume:

# pvcreate /dev/sdb1

Create the Volume Group (I called mine vg_store):

# vgcreate -s 256M vg_store /dev/sdb1

Create the Logical Volume (I called mine lv_store):

# lvcreate -L 4640G -n lv_store vg_store

Format the volume.  (I used ext4):

# mke2fs -t ext4 /dev/vg_store/lv_store

Make a folder in which to mount the volume (wherever you like):

# mkdir /mnt/store

Mount the volume (you may also want to edit /etc/fstab)

# mount -t ext4 /dev/mapper/vg_store-lv_store /mnt/store/

Enjoy your new GIANT storage volume!

2 Comments

  1. had this exact problem with an LSI card today while building a whitebox vm server. thanks for this , and a very similar reaction to the CLI. first helpful page i found. thanks.

  2. same here – thanks a lot for this documentation, man! nearly wanted to throw this controller out of the window and install a software raid when i saw your documentation ;)

Comments are closed.