With thousands of photos stored on my server, I needed more speed in Adobe Lightroom, Adobe Photoshop, and Sony Vegas.
Previously, I ran software RAID-5 using 6 Hitachi 4TB hard disks for 18TB of available capacity. Disk performance was quite decent for reads - 380-500 MB.s but relatively poor for writes (as most software raid servers are) at 70 MB/sec using ATTO benchmarks. This made Lightroom's editing of photos relatively slow and painful.
I tried to speed up the system by using a Samsung 840 500GB SSD as my boot drive (running Windows x64 Ultimate), which was initially hooked up to a Marvell SATA port on my Asus P8Z68-V Pro motherboard. The Marvel controller was performing poorly - as I was seeing 330MB/sec reads and 220 MB/sec writes. When I moved the Samsung 840 SSD to Intel's SATA3/6 Gbps ports, bandwidth increased to 550MB/sec for reads and 340 MB/sec for writes (almost 50-100% improvement!
I decided to try a hardware raid card to increase the speed of the Intel software RAID-5. After looking around, I found that LSI MegaRAID 9260-8i was highly rated but a bit expensive (around $470 on Newegg.com). This was too expensive, so I decided to see what was available on eBay. A few people recommended looking for a used RAID card from an older server.
I found a seller listing a Dell OEM LSI 9260-8i for $250 (new). This sounded like a good deal, so I purchased the card. I received the card rather quickly, but it took a long time to figure out how to get it to work. First, it would not boot from my motherboard's second PCI Express 2.0 x16 slot (the first occupied by the Zotac GTX 570 Graphics card). Next, I tried the 3rd PCI Express 2.0 x16 slot, and the LSI BIOS popped up. I was excited until I realized this slot only ran at 1x speed (shared with 2 other PCIe 1x slots, a USB controller, and other devices).
I decided to update the firmware on the LSI. However, when running MegaCLI to burn the firmware, it complained about firmware corruption. What the heck! I found it on a bulletin board. I had to install an intermediate firmware release first, and then I would install the latest release. It worked! But what a pain! LSI does not do an excellent job of testing its products.
The LSI's WebBios was impossible to use as my mouse control was unstable. However, I could use MegaCLI commands to do the same thing. So, using quite cryptic commands, I initialized my array (taking over 10 hours).
I decided that if I could not get the LSI Raid card to work in an x16 slot, I would return it. But how? I decided to try it again - I would move the Zotac GTX 570 to the 2nd PCIe 2.0 x16 slot and use the main x16 2.0 slot for the LSI 9260-8i controller. It worked! This had to be one of the more painful RAID installations for me. I can't believe I had to go through the steps to get this to work!
After initializing the disk array, I tried formatting the volume, but Windows x64 reported Format Failure each time. I tried a quick format standard format - all failed. I was getting quite frustrated!
I called LSI Technical Support for help but hung up after waiting on hold for 30 minutes.
Finally, I realized that 4TB drives were not supported by "Basic" disks in Windows. I needed to set the disk array up as a "Dynamic" volume, not a "Basic." Once I converted the array to Dynamic, the quick format worked beautifully.
Computer Configuration
Exact Install Procedure of the LSI MegaRaid 9260-8i
1. Setup ASUS BIOS in preparation for LSI MegaRaid 9260-8i
2. Install LSI MegaRaid 9260-8i in PCIe_1 2.0 x16 slot (main slot)
3. Install GTX570 Graphics card in PCIe_2 2.0 x16 slot
4. Boot the system and log in as Administrator
5. Install the LSI MegaRaid's latest software driver
The quick format is fast, and you should be able to use the disk immediately.
Previously, I ran software RAID-5 using 6 Hitachi 4TB hard disks for 18TB of available capacity. Disk performance was quite decent for reads - 380-500 MB.s but relatively poor for writes (as most software raid servers are) at 70 MB/sec using ATTO benchmarks. This made Lightroom's editing of photos relatively slow and painful.
I tried to speed up the system by using a Samsung 840 500GB SSD as my boot drive (running Windows x64 Ultimate), which was initially hooked up to a Marvell SATA port on my Asus P8Z68-V Pro motherboard. The Marvel controller was performing poorly - as I was seeing 330MB/sec reads and 220 MB/sec writes. When I moved the Samsung 840 SSD to Intel's SATA3/6 Gbps ports, bandwidth increased to 550MB/sec for reads and 340 MB/sec for writes (almost 50-100% improvement!
I decided to try a hardware raid card to increase the speed of the Intel software RAID-5. After looking around, I found that LSI MegaRAID 9260-8i was highly rated but a bit expensive (around $470 on Newegg.com). This was too expensive, so I decided to see what was available on eBay. A few people recommended looking for a used RAID card from an older server.
I found a seller listing a Dell OEM LSI 9260-8i for $250 (new). This sounded like a good deal, so I purchased the card. I received the card rather quickly, but it took a long time to figure out how to get it to work. First, it would not boot from my motherboard's second PCI Express 2.0 x16 slot (the first occupied by the Zotac GTX 570 Graphics card). Next, I tried the 3rd PCI Express 2.0 x16 slot, and the LSI BIOS popped up. I was excited until I realized this slot only ran at 1x speed (shared with 2 other PCIe 1x slots, a USB controller, and other devices).
I decided to update the firmware on the LSI. However, when running MegaCLI to burn the firmware, it complained about firmware corruption. What the heck! I found it on a bulletin board. I had to install an intermediate firmware release first, and then I would install the latest release. It worked! But what a pain! LSI does not do an excellent job of testing its products.
The LSI's WebBios was impossible to use as my mouse control was unstable. However, I could use MegaCLI commands to do the same thing. So, using quite cryptic commands, I initialized my array (taking over 10 hours).
I decided that if I could not get the LSI Raid card to work in an x16 slot, I would return it. But how? I decided to try it again - I would move the Zotac GTX 570 to the 2nd PCIe 2.0 x16 slot and use the main x16 2.0 slot for the LSI 9260-8i controller. It worked! This had to be one of the more painful RAID installations for me. I can't believe I had to go through the steps to get this to work!
After initializing the disk array, I tried formatting the volume, but Windows x64 reported Format Failure each time. I tried a quick format standard format - all failed. I was getting quite frustrated!
I called LSI Technical Support for help but hung up after waiting on hold for 30 minutes.
Finally, I realized that 4TB drives were not supported by "Basic" disks in Windows. I needed to set the disk array up as a "Dynamic" volume, not a "Basic." Once I converted the array to Dynamic, the quick format worked beautifully.
Computer Configuration
- Intel i7-2600K CPU at 3.4GHz
- ASUS P8Z68-V Pro Motherboard with BIOS 3402
- Crucial Ballistix 16GB (8GBx2) DDR3-1600
- Samsung 840 SSD (500Gbyte) connected to SATA1 of Intel (6Gbps) <-boot drive
- Seagate 1TB ST31999524AS hard disk connected to SATA2 of Intel (6Gbps)
- Dell LSI MegaRaid 9260-8i card in PCIe 2.0 slot 1(running at 8x 2.0 mode)
- 6x Hitachi 4TB Hard Drives connected to the LSI MegaRaid
- Zotac Amp! GeForce GTX 570 Video Card in PCIe 2.0 slot 2 (running at 8x 2.0 mode)
- ASUS Xonar Essence STX in PCIe x1 slot
- Blu-Ray Drive connected to SATA3 (1.5Gbps)
- HP ZR30w 30" Monitor at 2560x1600
- SIIG USB Keyboard
- Logitech MX518 mouse
- 800W Power Supply
- Antec Titan Server Case
- Windows 7 x64 Ultimate
Exact Install Procedure of the LSI MegaRaid 9260-8i
1. Setup ASUS BIOS in preparation for LSI MegaRaid 9260-8i
2. Install LSI MegaRaid 9260-8i in PCIe_1 2.0 x16 slot (main slot)
3. Install GTX570 Graphics card in PCIe_2 2.0 x16 slot
4. Boot the system and log in as Administrator
- With the LSI card present, you may no longer be able to change settings in your BIOS, as LSI BIOS corrupts the behavior of USB mice.
- You do not need to get into WebBIOS (unless you need to boot from the array). WebBIOS requires a PS/2 mouse, but you can do all the same functions in the MegaCLI command line in Windows.
5. Install the LSI MegaRaid's latest software driver
- Download the latest LSI MegaRaid software driver.
- Update the driver (Control Panel->Device Manager->Storage Controls->Properties->Driver Update (point the driver directory)
- Download and install LSI MegaCLI software
- Open a Windows command line interpreter in Administrator Mode
- cd into LSI MegaCLI install folder
- Do the following commands (see the following guide for more help on MegaCLI):
C:\Users\Admin> megacli64 -adpfwflash -f mr2108.rom -a0
- If the firmware succeeded, great, you're done. If you get a message about disk corruption, you have ancient firmware and may need to first flash an intermediate firmware file
- Reboot the PC and verify the new version
- Note that a reboot is slow; don't give up hope
Let's assume we have a server with one MegaRAID SAS card. The first one is already set up, but we have just plugged a disk bay on the second card.
List physical disks on the second card (only print enclosure and slot numbers):
List physical disks on the second card (only print enclosure and slot numbers):
C:\Users\Admin\CLI> megacli64 -PDlist -a0 Enclosure Device ID: 0 Slot Number: 0 Enclosure Device ID: 0 Slot Number: 1 Enclosure Device ID: 0 Slot Number: 2 Enclosure Device ID: 0 Slot Number: 3 Enclosure Device ID: 0 Slot Number: 4 Enclosure Device ID: 0 Slot Number: 5 Enclosure Device ID: 0 Slot Number: 6
Now we have all enclosures and slot numbers. Let's create the new array:
C:\Users\Admin\CLI> megacli64 -CfgLdAdd -r5 [0:0,0:1,0:2,0:3,0:4,0:5,0:6] -a0 Adapter 0: Created VD 0 Adapter 0: Configured the Adapter!! Exit Code: 0x00
Read Cache, Write Cache, ReadAhead, and Battery
A quick section about performance tunning...
Let's enable Read Cache and always cache data:
C:\Users\Admin\CLI> megacli64 -LDSetProp -Cached -LAll -aAll Set Cache Policy to Cached on Adapter 0, VD 0 (target id: 0) success
Enable disks' cache:
C:\Users\Admin\CLI> megacli64 -LDSetProp EnDskCache -LAll -aAll Set Disk Cache Policy to Enabled on Adapter 0, VD 0 (target id: 0) success
8. Once the array has initialized, you can format the disk and start using it:
- Open Control Panel->Administrative Tools->Computer Management->Disk Management
- Click on Disk Management and select the disk array
- Right click on the disk and select Convert to Dynamic Disk
- Click on the a disk itself and select Format
- Choose NTFS, 8192, Quick Format and press OK
The quick format is fast, and you should be able to use the disk immediately.
0 comments:
Post a Comment