BadassBob's badass NAS overhaul

BadassBob

Digitally rendered
First, a brief background. While in my laundry room last night, I heard that familiar clickity-clack of a failing hard drive coming from my server. Made a dash for the workstation, SSHed in, ran smartctl on my drives, and found a pair of my 500GB WD Blacks will soon be giving up the ghost. They have served me well for 6 years in a RAID 10 mdadm configuration hosting my mess of virtual machines 24/7...now I have an excuse to upgrade. Last night, I backed up all of my virtual machines, so no worries until the new stuff arrives. I slept a little better last night.

I decided to replace the 500GB Blacks with 2TB Reds, and will run the VMs from a 2x 1TB WD Red 2.5" RAID 1 configuration. I ended up purchasing a 2x 2.5" hot-swap bay for my case as well, along with 32GB of RAM (upping it from 16GB) and another 3Ware RAID card. This got me thinking, maybe I will go with another OS this time instead of CentOS, thinking a distro specific for virtualization such as Proxmox or Ovirt, as they provide intuitive web GUIs for VM management, storage pool management, and system monitoring. Previously, I did everything the old fashioned way, over SSH and ran tools from the command line...time for me to get a little more modern in that regard.

Aside from the RAM upgrade and additional 3ware card, I'm still keeping my base hardware which consists of a pair of 2.5GHz Xeon L5420 CPUs on a Supermicro X7DWE mainboard. Even though they're long in the tooth (from 2008), they work very well for what I do, but that stuff will also be upgraded late this year or early next year with a newer single or dual Xeon mainboard, as they're much more power efficient than my dinosaurs.

The final setup will look like this:
4x 2TB WD Red
4x 2TB Hitachi Ultrastar
2x 1TB WD Red 2.5" for VMs
1x IBM BR10i SAS/SATA host bus adapter
1x 3ware 9650SE-4PLPML SATA RAID controller
Supermicro X7DWE Mainboard
2x Xeon L5420 CPU
4x 8GB Hynix DDR2 667 FB-DIMM

The virtual machine pool:
MythTV master back end - network PVR system
MythTV slave back end - failover for the master back end
Vortexbox - music server
CentOS - Amanda network backup for bare metal backups and snapshots of network clients
CentOS - ownCloud to access all of my media from any internet connected device
CentOS - Piwigo for photo hosting
Debian - Transmission Daemon bit torrent server
CentOS - Squid Proxy and DansGuardian for keeping pesky ads and malware away from my network
OpenMediaVault - file storage

As it currently stands, my server has been running 24/7 for the last 6 years across 2 base hardware upgrades (went from a Core2 Quad/4GB RAM to dual Xeon/16GB RAM) with no reinstallation of the OS and just updating the virtual machines to newer OSes as they approach their EOL. Aside from initial configuration, it has been a mostly hands-off experience, but with Proxmox or Ovirt, it will be even easier. Stay tuned!
 
Quite an impressive setup, Bob. I keep 2 backups of my server, but so far no problems.
 
Bob that IS impressive. Is it possible to take some pics of your system when up and runnning?

Sure, I plan on posting screenshots and all that other good stuff, too. I should have my hard drives next Tuesday according to UPS tracking. This is the current iteration. It'll look very similar after it's finished with a few more drives added. The chassis has 8 hot-swap bays, 2 more will be added for the OS drives.

gBIxbIx.jpg


xaDjjDfl.jpg
 
Last edited:
Thanks....looks powerful! I suppose thats in the next room over from the gear? Whats the red cable? Looks just like a WW Starlight 7. Nice looking board!
 
Thanks....looks powerful! I suppose thats in the next room over from the gear? Whats the red cable? Looks just like a WW Starlight 7. Nice looking board!

The red cable is just your standard SATA cable. When that pic was taken, I had a DVD-ROM on there being passed through to my Vortexbox virtual machine for disc ripping purposes, but have since removed it and put it on my desktop. Now my DVD-ROM is shared over the network so I can rip right on my Vortexbox VM without having to go to the laundry room and feed discs to the server.
 
Quite an impressive setup, Bob.

Hearing Bob spool off that tech talk is even more impressive than the system! I have an MSEE and I can't talk like that. :no:

Looks like a cool setup. Let the music stream forth!
 
Now I am perusing craigslist for a server with a Xeon processor and I found a Dell T7400 Workstation. Thanks Bob. :dammit:
 
Now I am perusing craigslist for a server with a Xeon processor and I found a Dell T7400 Workstation. Thanks Bob. :dammit:

The T7400 uses the same family of CPUs as my build, the LGA771 5400 Xeons. They aren't very power friendly by today's standards, but I don't complain as my electric bill is rather reasonable. When I first built this particular one 4 years ago, I sourced the board, CPUs, and RAM from eBay and spent about $180. The chassis was purchased from Amazon for $80 on sale, and the chassis-specific hot-swap bays were $60 each from NeworldIT. Not bad for $320 total. My hard drives were already a couple of years old at the time so I didn't factor those into the cost. Later this year, I'll upgrade to 5500 or 5600 series LGA1366 Xeons and up the RAM to 48GB or so. I love the modularity of a hypervisor/virtual machine setup. Much less headache if something goes south, with everything easily recovered after restoring from a backed up virtual machine image.
 
The T7400 uses the same family of CPUs as my build, the LGA771 5400 Xeons. They aren't very power friendly by today's standards, but I don't complain as my electric bill is rather reasonable. When I first built this particular one 4 years ago, I sourced the board, CPUs, and RAM from eBay and spent about $180. The chassis was purchased from Amazon for $80 on sale, and the chassis-specific hot-swap bays were $60 each from NeworldIT. Not bad for $320 total. My hard drives were already a couple of years old at the time so I didn't factor those into the cost. Later this year, I'll upgrade to 5500 or 5600 series LGA1366 Xeons and up the RAM to 48GB or so. I love the modularity of a hypervisor/virtual machine setup. Much less headache if something goes south, with everything easily recovered after restoring from a backed up virtual machine image.
I am not that worried about the cost of the electricity because I want to eliminate the server farm (6 computers) I have acquired over the years. Thank you for detailing the prices you paid for all the equipment so I think what I am about to pay is a fair price. For $150 I will be getting the T7400 with the following:

2x XEON E5420 2.5 quad processors
Standard Graphics Card (instead of a Nvidia Quadro FX4800)
12gb ECC RAM DDR2
2x 300gb SAS 15k (Hitachi usually) for RAID usage
980w PSU
DVDRW Drive

There is no OS and I want to get into virtualization so what do you suggest? I am the most comfortable with Ubuntu but have never really used a virtual machine except for a BOINC project on a Windows 7 machine which set it self up. I have also never used a RAID configuration. It seems like I have a lot to learn.
 
I am not that worried about the cost of the electricity because I want to eliminate the server farm (6 computers) I have acquired over the years. Thank you for detailing the prices you paid for all the equipment so I think what I am about to pay is a fair price. For $150 I will be getting the T7400 with the following:

2x XEON E5420 2.5 quad processors
Standard Graphics Card (instead of a Nvidia Quadro FX4800)
12gb ECC RAM DDR2
2x 300gb SAS 15k (Hitachi usually) for RAID usage
980w PSU
DVDRW Drive

There is no OS and I want to get into virtualization so what do you suggest? I am the most comfortable with Ubuntu but have never really used a virtual machine except for a BOINC project on a Windows 7 machine which set it self up. I have also never used a RAID configuration. It seems like I have a lot to learn.

Sounds like a nice machine. You can use Ubuntu no problem. Upon install, it'll give you the option to install to a RAID array. My favorite Linux hypervisor is KVM, or kernel-based virtual machine. With KVM, everything is at the kernel level and you can even pass through controller cards and other hardware to the virtual machines, so they see them as their own. If you had a pair of hard drive controllers, you could use one for the host OS and pass the other along to a virtualized NAS-specific OS such as OpenMediaVault, FreeNAS, etc. Other popular choices are Virtualbox, Xen, and ESXi. KVM can be used from a desktop GUI via virt-manager, web GUI with WebVirtMgr or Kimchi, or command line with SSH and QEMU. Check out the tutorials over on How To Forge, lots of fantastic ones on that site.
 
I just watched a video on how to use Proxmox and I might go with that. Though Ubuntu server is still in the running because I am familiar with it. Now if the guy would just email me back so I can pick the computer up.

I noticed that you use CentOS for most of your VMs. Is that just personal preference or is there another reason?
 
I noticed that you use CentOS for most of your VMs. Is that just personal preference or is there another reason?

I'm using Cent because each release gets full updates for 6 years, and security updates for 10 years. Cent leans more towards stability instead of bleeding edge. They take an "if it ain't broke don't fix it" approach.
 
Gents - in layman's terms (I'm talking the "Crayola level" :) ) what are the advantages of setting up virtual machines in an environment like this? I was thinking about doing something similar, where Ubuntu Server would have both MPD and the cloud server running on the same box. If I'm reading this right, you've got a primary OS (CentOS) and then running additional OS's inside CentOS as VM's? So what would be the advantage of - in my case - running MPD or the cloud server in a VM?
 
Gents - in layman's terms (I'm talking the "Crayola level" :) ) what are the advantages of setting up virtual machines in an environment like this? I was thinking about doing something similar, where Ubuntu Server would have both MPD and the cloud server running on the same box. If I'm reading this right, you've got a primary OS (CentOS) and then running additional OS's inside CentOS as VM's? So what would be the advantage of - in my case - running MPD or the cloud server in a VM?

Yes, virtual machines are separate guest OSes running under a host OS or hypervisor. The biggest advantage is application isolation. Instead of installing a bunch of software on a single OS, you can install one or a few applications within virtual environments. Now you have room to experiment with new software without breaking anything. If something does break, it's a snap restoring a virtual machine from a backup and be up and running in minutes. You can also clone your virtual machines and run them concurrently testing new software while the "workhorse" VM still chugs away handling the important stuff.
 
Hard drives were just delivered, now to finish off the work day and get down to some surgery later on :D
 
I am on my way to pick up the T7400. I think I am going to run Ubuntu Server as the host operating system and CentOS as most of the VM's.

For VM's that use less 4GB RAM am I better off running a 32-bit version of CentOS?

LMS needs to access two USB hard drives for the music library. I have heard that I should not have the host mount them and have them mounted on the VM. Is this true?
 
I am on my way to pick up the T7400. I think I am going to run Ubuntu Server as the host operating system and CentOS as most of the VM's.

For VM's that use less 4GB RAM am I better off running a 32-bit version of CentOS?

LMS needs to access two USB hard drives for the music library. I have heard that I should not have the host mount them and have them mounted on the VM. Is this true?

I'd use 64-bit OSes for the VMs, they do run much better. It's best to run the same architecture that the host uses. USB mounting should be no problem, as USB devices can be operated in pass-through mode, handing them off directly to the VM.
 
Had a little hangup, UPS mis-sorted the package with my 2x2.5" hot-swap bay, so I won't get that till tomorrow sometime :(. I was hoping to get this guy going, looks like tomorrow is the day. Now I just get to sit and stare at my pile of hard drive for another day.
 
Last edited:
Back
Top Bottom