Critique my 100TB+ Build

I’m in the planning stages on a new build. Ultimately, this will be able to house something like 18 HDDs and 5 SSDs (probably more). Raw HDD capacity of up to 324 TB.

Currently running a 75TB JBOD attached as DAS to an iMac. I’m looking to get some redundancy and eventually offload some of the compute (Plex, arr services, bittorrent, etc.) to a dedicated server. Likely will run Unraid.

I’ve gone through the NAS Killer threads, but used hardware is crazy expensive currently. So much so that it seems to make more sense to get mostly new hardware primarily. I’d like the server to be relatively quiet and low power (in x86 terms, anyway).

I was originally looking at an ASUS WS C246 Pro ATX board ($260) and Xeon E-2124G ($227) with 32GB RAM, but an 11th gen i5 seems like a better deal, no?

Here’s what I have so far:

Case: Fractal Design Meshify 2 XL - $205
PSU: Corsair HX750 80 Plus Platinum Modular - $155
Mobo: Gigabyte Z590 AORUS Ultra - $240
Alt Mobo: MSI MPG Z590 Gaming Carbon Wifi - $270
CPU: Intel Core i5-11400 6C 2.6GHz 65W - $220
Alt CPU: i3-10100 4C 3.6Ghz 65W for $130
RAM: G.SKILL Ripjaws V Series 32GB DDR4 3200 - $115
CPU Cooler: Stock Boot Drive: SanDisk 32GB Ultra Flair CZ73 USB 3.0 - $7 M.2 Drive (Write Cache): Samsung 970 EVO Plus Gen 3.0 x4 NVME - $140
HBA: LSI 9207-8i - PCIe 3.0 x8 - $65 on eBay
Alt HBA: LSA SAS 9201-16i - $200
SAS Expander: Intel 36-Port RES2CV360 - $180
Alt SAS Exp: Intel 24-Port RES2SV240 - $100 OR HP 468406-B21 - $95 (3gb SATA only)
10Gbe NIC: Intel X540-T2 - $85

I’d be grateful for any feedback or suggestions.

Don’t spend 200 on an expander would be my first take.

What’s the Samsung 970 for? Seems a bit overkill for a storage server.

I think he messed up the formatting

CPU Cooler: Stock
Boot Drive: SanDisk 32GB Ultra Flair CZ73 USB 3.0 - $7 M.2 Drive
(Write Cache): Samsung 970 EVO Plus Gen 3.0 x4 NVME - $140

is how I read it

Tyan S5512 (NK4.1) is still $45 and includes onboard SAS2308. 1155 chips are super cheap. DDR3 hasn’t gotten any cheaper, but you don’t need much for NAS. L4500 plus your existing DAS would be plenty of bays. X540-T2 should be around $60, or if you’re willing to go SFP+, CX311 (mlx4_en) for $40, or S7120 (sfc) for $20.

1 Like

Yup! now that makes sense!

Agreed 100%. eBay prices for SAS expanders is crazy. I’m trying to figure out a better way to get at least 16 SAS lanes without 1) wasting pcie lanes and 2) spending a ton of cash.

Possibly a single LSI 9201-16i (for $280… ugh) or two LSI 9207-8i for ($130). Using two HBAs would eat up all of my pcie lanes. Then the x540 wants another x8.

It would be a write cache. I do a lot of sustained sequential writes which will benefit from a write cache. Since the CPU has a dedicated x4 lanes for NVME, might as will use one.

Planning to use the X540-T2 mainly because adding a SFP+ port to my iMac will be ridiculously expensive. 10Gbe is more economical (in this case).

There’s a lot to unpack here. I’ll do some research and circle back.

Might want to check out the Adaptec prices [Official] Adaptec ASR series SAS HBA overview

Adaptec ASR-7805 SAS2 HBA - $24.74 OBO + shipping

1 Like

It’s generally more economical to get a SFP+ 10Gb switch and use these transceivers when you need 10Gb Base-T. There are other, cheaper versions of this, but I prefer to use 10Gtek when possible.

The problem with SFP+ is Mac support is pricey. Adding a SFP+ port to an iMac (with a Thunderbolt PCIe enclosure) costs around $550. I can get a 10Gbe BaseT (RJ-45) to Thunderbolt dongle for around $150. If I were using a PC, SFP+ would be cheaper.

I’ve been researching processors with more PCIe lanes and the Threadripper 1900x looks like a good option at $150. Similar passmark to the i5, but it has 64 PCIe lanes (vs 20). Of course I’d need a dGPU for Plex transcoding…

I believe you’ve totally ignored what I said.

Who said you needed sfp on your mac?

Sorry, that’s what happens when I try to multitask…

I appreciate the switch suggestion. I’ve only taken a cursory look at 10Gbe RJ-45 switches, but Unifi has a 4-port 10GbE RJ45 switch for $299 or an 8-port 10G SFP+ switch for $269.

I haven’t 100% decided on how to network everything, but I’m probably going to just direct connect the Mac to the NAS over 10Gbe. Then connect the NAS to my 1Gbe LAN using the motherboard LAN port.

yes, direct connect 10GbE is just fine. You can put the 10GbaseT (RJ45) NIC on the Mac, run cat6 to the NAS, and put a cheap SFP+ NIC plus RJ45-SFP+ transceiver in the NAS.

Then when you want to expand, Aruba S2500.

Finalized Build — Completed

Case: Fractal Design Meshify 2 XL - $205
Case Fans: Noctua 140mm ULN (qty. 5) - $100
PSU: Corsair HX750 80 Plus Platinum Modular - $155
Mobo: ASRock X399 Taichi - $339
CPU: AMD Threadripper 1900X - $149
CPU Cooler: Noctua NH-U14S TR4-SP3 - $89
RAM: Corsair Vengeance LPX 32GB - $115
GPU: Nvidia Quadro P400 - $80
Boot Drive: SanDisk 32GB Ultra Flair CZ73 USB 3.0 - $7
M.2_1 Drive (Write Cache): Samsung 970 EVO Plus 1TB Gen 3.0 x4 NVME - $140
M.2_2 Drive (VMs & Dockers): SK Hynix P31 1TB Gen 3.0 x4 NVME - $110
HDD: WD HC550 18TB (qty. 4)
HDD: Toshiba MG09ACA18TE 18TB (qty. 1)
HDD: WD 14TB easystore shucks (qty. 2)
HDD: Old Seagate 3TB (qty. 1) – downloads cache
HBA: Adaptec ASR-71605 - $95
10Gbe NIC: Intel X540-T2 - $85

I put the large HDDs in an Unraid array with two parity drives. Array capacity is 82TB currently (118TB raw capacity). If I use an 18TBx16 config, the total possible array size is 252TB (assuming two parity drives).

I still have room for another (8) 3.5" HDDs, plus (5) 2.5" drives for future expandability. I plan to add a third M.2 drive, likely a 2TB P31 to be a new cache drive. Or if I can find a cheap-ish used U.2 NVME SSD, I’ll put that on the U.2 port. Then I will use the existing 1TB M.2 drives as a mirrored VM/Docker array.

Since this motherboard and processor support PCIe bifurcation, I’m also planning to add an ASUS Hyper M.2 X16 AIC to add support for 4 additional M.2 NVME drives. I’ll use those drives either as additional VM drives or as additional cache.