I’m in the planning stages on a new build. Ultimately, this will be able to house something like 18 HDDs and 5 SSDs (probably more). Raw HDD capacity of up to 324 TB.
Currently running a 75TB JBOD attached as DAS to an iMac. I’m looking to get some redundancy and eventually offload some of the compute (Plex, arr services, bittorrent, etc.) to a dedicated server. Likely will run Unraid.
I’ve gone through the NAS Killer threads, but used hardware is crazy expensive currently. So much so that it seems to make more sense to get mostly new hardware primarily. I’d like the server to be relatively quiet and low power (in x86 terms, anyway).
I was originally looking at an ASUS WS C246 Pro ATX board ($260) and Xeon E-2124G ($227) with 32GB RAM, but an 11th gen i5 seems like a better deal, no?
Here’s what I have so far:
Case: Fractal Design Meshify 2 XL - $205
PSU: Corsair HX750 80 Plus Platinum Modular - $155
Mobo: Gigabyte Z590 AORUS Ultra - $240
Alt Mobo: MSI MPG Z590 Gaming Carbon Wifi - $270
CPU: Intel Core i5-11400 6C 2.6GHz 65W - $220
Alt CPU: i3-10100 4C 3.6Ghz 65W for $130
RAM: G.SKILL Ripjaws V Series 32GB DDR4 3200 - $115
CPU Cooler: Stock Boot Drive: SanDisk 32GB Ultra Flair CZ73 USB 3.0 - $7 M.2 Drive (Write Cache): Samsung 970 EVO Plus Gen 3.0 x4 NVME - $140
HBA: LSI 9207-8i - PCIe 3.0 x8 - $65 on eBay
Alt HBA: LSA SAS 9201-16i - $200
SAS Expander: Intel 36-Port RES2CV360 - $180
Alt SAS Exp: Intel 24-Port RES2SV240 - $100 OR HP 468406-B21 - $95 (3gb SATA only)
10Gbe NIC: Intel X540-T2 - $85
Tyan S5512 (NK4.1) is still $45 and includes onboard SAS2308. 1155 chips are super cheap. DDR3 hasn’t gotten any cheaper, but you don’t need much for NAS. L4500 plus your existing DAS would be plenty of bays. X540-T2 should be around $60, or if you’re willing to go SFP+, CX311 (mlx4_en) for $40, or S7120 (sfc) for $20.
Agreed 100%. eBay prices for SAS expanders is crazy. I’m trying to figure out a better way to get at least 16 SAS lanes without 1) wasting pcie lanes and 2) spending a ton of cash.
Possibly a single LSI 9201-16i (for $280… ugh) or two LSI 9207-8i for ($130). Using two HBAs would eat up all of my pcie lanes. Then the x540 wants another x8.
It would be a write cache. I do a lot of sustained sequential writes which will benefit from a write cache. Since the CPU has a dedicated x4 lanes for NVME, might as will use one.
Planning to use the X540-T2 mainly because adding a SFP+ port to my iMac will be ridiculously expensive. 10Gbe is more economical (in this case).
There’s a lot to unpack here. I’ll do some research and circle back.
It’s generally more economical to get a SFP+ 10Gb switch and use these transceivers when you need 10Gb Base-T. There are other, cheaper versions of this, but I prefer to use 10Gtek when possible.
The problem with SFP+ is Mac support is pricey. Adding a SFP+ port to an iMac (with a Thunderbolt PCIe enclosure) costs around $550. I can get a 10Gbe BaseT (RJ-45) to Thunderbolt dongle for around $150. If I were using a PC, SFP+ would be cheaper.
I’ve been researching processors with more PCIe lanes and the Threadripper 1900x looks like a good option at $150. Similar passmark to the i5, but it has 64 PCIe lanes (vs 20). Of course I’d need a dGPU for Plex transcoding…
Sorry, that’s what happens when I try to multitask…
I appreciate the switch suggestion. I’ve only taken a cursory look at 10Gbe RJ-45 switches, but Unifi has a 4-port 10GbE RJ45 switch for $299 or an 8-port 10G SFP+ switch for $269.
I haven’t 100% decided on how to network everything, but I’m probably going to just direct connect the Mac to the NAS over 10Gbe. Then connect the NAS to my 1Gbe LAN using the motherboard LAN port.
yes, direct connect 10GbE is just fine. You can put the 10GbaseT (RJ45) NIC on the Mac, run cat6 to the NAS, and put a cheap SFP+ NIC plus RJ45-SFP+ transceiver in the NAS.
I put the large HDDs in an Unraid array with two parity drives. Array capacity is 82TB currently (118TB raw capacity). If I use an 18TBx16 config, the total possible array size is 252TB (assuming two parity drives).
I still have room for another (8) 3.5" HDDs, plus (5) 2.5" drives for future expandability. I plan to add a third M.2 drive, likely a 2TB P31 to be a new cache drive. Or if I can find a cheap-ish used U.2 NVME SSD, I’ll put that on the U.2 port. Then I will use the existing 1TB M.2 drives as a mirrored VM/Docker array.
Since this motherboard and processor support PCIe bifurcation, I’m also planning to add an ASUS Hyper M.2 X16 AIC to add support for 4 additional M.2 NVME drives. I’ll use those drives either as additional VM drives or as additional cache.