CSE-846 Unraid/Plex Build Help

Builds [LGA1151,LGA1200] NAS Killer 6.0, Plex QSV Builds
1

I’ve been running Unraid for several years now and I absolutely love it! I mainly use it as a plex host, but over the past year or so I’ve run out of more physical slots to add more storage. I’m currently sitting at 94% usage of 62TB so I started my journey into bigger and better hardware.

My current specs are as follows:

  • Case: Cooler Master Cosmos II
  • Motherboard: Asus Prime X370 Pro
  • CPU: Ryzen 7 1800X
  • RAM: 32GB Corsair Vengeance
  • GPU: Nvidia Quadro P2000
  • HBA: LSI 9211-8i
  • Storage:
    *4x Ironwolf 4TB
    *2x Ironwolf 8TB
    *3x Ironwolf 10TB
    *2x Exos 10TB (Parity)
    *3x Ironwolf SSD’s in Raid 0 (Cache)

I started looking into multi drive rackmount server chassis and settled on the infamous CSE-846. A quick Ebay trip later and I have one sitting empty in my basement rack.

This is the part where I need some guidance.
I already have the SQ PSU’s but my main concern right now is HBA cards and backplane. The HBA’s I have are AOC-SASLP-MV8. (These came out of an 847) and I have a BPN-SAS-846A backplane. From what I understand these do not support drives larger than 2TB and I have quite a few of those.

I am leaning towards getting a BPN-SAS2-846EL1 since it’s SAS2 but looking for more specific information about the different backplanes has left me even more confused. I have no idea what HBA to use either. Since I already have a 9211-81 should I just get 2 more of those for all of the 24 bays? I think the only problem there is my motherboard doesn’t have enough 8x slots for 3 HBA’s and my GPU so if I go that route I’m looking at a new motherboard as well.

Ideally I’d like to continue to use my Ryzen system, but I have an X8-DTH-IF motherboard laying around so I could snag another E5 Processor and with that I could use the 9211’s with the SAS2 backplane and repurpose my Ryzen machine and I don’t think I will have any bottlenecks but I’m having a really hard time thinking this through and tying it all together.

Any help or experience is greatly appreciated!

2

Break down of each back plane below :slight_smile:

BPN-SAS-846TQ
It has no expander, you must connect each of theses 24 SAS/SATA ports to HBAs.
So for this you will require more than one HBA (if you use a 8 ports HBA, you will need 3 HBA for connect all drives).
You must use splitter cables like:

  • SFF-8087 to sas/sata (4 ports per cable) in order to connect the HBA to the backplane for use with a SAS2 HBA
  • SFF-8643 to sas/sata (4 ports per cable) in order to connect the HBA to the backplane for use with a SAS3 HBA

The advantage of this backplane is to support SAS3 HBA, that means it can be an entry level to SAS3 area.
The SAS3 compatibility is not certified from SuperMicro.

BPN-SAS-846A
It’s the same than BPN-SAS-846TQ, it has no expander.
But instead to have 24 single SATA/SAS ports to connects, you have 3 SFF-8087 ports.
With this backplane, you will reduce the cabling mess.
In term of capabilities, it’s the same. If you use a SFF-8646 to SFF-8087, you could use SAS3 HBA (but it’s not certified from SuperMicro)

BPN-SAS-846EL1
It has one expander supporting only SAS1 version.
That means you will may have issues with disk larger than 2 Tb.
With spinning disks, the SAS1 bandwidth maybe not a large issue, but with SSD it’s a very bottleneck.
It has only one expender, so you can’t use the “dual port” SAS feature that allow a redundancy path from HBAs to the disks.
You use SFF-8087 to SFF-8087 cable (just one) for connect the HBA to the backplane

BPN-SAS-846EL2
It’s the same than the BPN-SAS-846EL1 except it has 2 expanders that allow the “dual port” SAS feature.
But you have same limitations in term of bandwidth and 2 Tb limit.
For using the dual-port, you need 2 cables and 2 hbas instead of one

BPN-SAS2-846EL1
The most popular backplane for “home labs”.
It has one expander, so you haven’t “dual port”, but this backplane supports SAS2 protocol.
You haven’t any issue with disk larger than 2Tb but bandwidth will be a limit with SSD (24x SSD will saturate 4x SAS2 lanes)
You use SFF-8087 to SFF-8087 cable (just one) for connect the HBA to the backplane
It seems, you can connect 2 cables in order to have better bandwidth (8 lanes instead of 4 lanes), but i have no experience with this.

BPN-SAS2-846EL2
It’s the same than the BPN-SAS2-846EL1 except it has 2 expanders that allow the “dual port” SAS feature.

BPN-SAS3-846EL1 / BPN-SAS3-846EL2
Variants with the support of the SAS3 protocols.
It requires SFF-8643 cables (instead of SFF-8087)

3

Thanks! I did some more digging after my post last night and I believe I ran across a similar description on the STH Forums.

I think the SAS2-EL1 is probably what I want because according to the documentation it only has 3 SAS ports on it ( I believe one is for failover) so I could get away with 2 LSI cards and my GPU to fill all 3 ports on my current motherboard.

Any other suggestions?

4

Your cheapest option is going to be to keep your 846A backplane, get a 9201-8i, 9210-8i, or 9211-8i (these three cards are functionally equivalent, the only important difference for you being the location of the SAS connectors) and an HP expander. Add a couple SFF-8087 to SFF-8087 cables to connect the bottom 2 ports of the expander to the controller, then plug the 6 ports from the 846A backplane into the remaining ports on the expander, buying additional 8087 to 8087 cables if your chassis didn’t come with them. The 846A backplane has no 2TB limitation, and the HP expander works with large drives as well. The only real downside to this option is all the extra cables.

If you want to switch to the 846EL1 backplane, you can skip the HP expander and most of the cables, but you won’t gain any additional benefits over the 846A/HP expander combo detailed above.

5

@Mthrboard
Thanks for the info. I never thought about adding an extra expander. In that scenario would I only need a single 92118i and the HP Expander?

I know you said this is the cheapest option, is there a better alternative that’s more expensive? Any issues with my SSD cache here?

6

Yes, in the scenario I laid out you would just need a single SAS controller and the expander. Going this route is fundamentally the same thing as switching to the 846EL1 backplane, the 846EL1 backplane just moves the expander chipset from a separate card to the backplane itself.

As for other options, maybe “cheapest” wasn’t the right word to use, because the price partially depends on what was included with the specific chassis you have. Like, did you get the 6x SFF-8087 cables that usually come standard with these chassis, and if you did are the long enough to reach the rear ports on the HP expander? Maybe post a few pics of the inside with the air shroud removed. Your other options could include switching to the 846EL1 or 846EL2 backplane, or just getting another 9211-8i controller and using the onboard SATA ports for the last 8 drives with a pair of SFF-8087 to 4x SATA Reverse breakout cables. Now that I think about it, this last option might actually be cheaper overall. And see my notes regarding your SSDs below before you make your decision.

Lastly, regarding the SSD, you don’t want to connect it through the SAS controller, regardless of which option you choose for a backplane. SAS controllers don’t allow SATA SSDs to TRIM properly, meaning your drives can wear out prematurely. SATA SSDs should be connected directly to the SATA ports on your motherboard. Normally with the 846 chassis I recommend mounting your SSDs in the back half of the chassis. There is a bracket you can mount to the interior sidewall of the chassis by the power supplies that allows you to mount 2x 2.5" drives, or you can velcro or double-sided tape the drives to the sidewall as well. However, if you keep the 846A backplane you have another option. Since your backplane is just passing the drives through, you could get an SFF-8087 to 4x SATA Reverse breakout cable to plug four of your front bays to four SATA ports on your motherboard. You’d just need to check the manual for your backplane to figure out which port maps to which bays on the front.

7

@Mthrboard You are AWESOME. The 846 I have came barebones outside of PSU’s and the backplane. I got an amazing deal on an 847 last year that came loaded with motherboard, backplane etc, what I didn’t realize is that the motherboard tray is only half height so I wouldn’t be able to mount my GPU.

Ideally I would like the SSD’s to be up front because I already have 2.5" trays for them. I also already have an 8087 to 4x SATA from my current setup so that should work fine. Thanks for the note about TRIM. I already had that issue once previously when I had my SSD’s plugged into my 9211 so breaking out 4 slots is a good call.

F̶o̶r̶ ̶t̶h̶e̶ ̶e̶x̶p̶a̶n̶d̶e̶r̶ ̶c̶a̶r̶d̶,̶ ̶w̶o̶u̶l̶d̶ ̶I̶ ̶j̶u̶s̶t̶ ̶n̶e̶e̶d̶ ̶a̶ ̶s̶i̶n̶g̶l̶e̶ ̶9̶2̶1̶1̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶e̶x̶p̶a̶n̶d̶e̶r̶ ̶o̶r̶ ̶2̶,̶ ̶9̶2̶1̶1̶’̶s̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶e̶x̶p̶a̶n̶d̶e̶r̶?̶ (Disregard, there are only 2 HBA ports on the expander duh) I will need to get some 8087 cables, but that shouldn’t be a major issue/expense. There are some with my 847 but getting them out has proved to be a major pain as I have already tried that. Looking through the expanders there seem to be 2 types. 3gbps and 6gbps. Neither are really expensive, I am assuming it would be better to get the 6gbps for this application?

8

If you already have a cable, make sure it’s a REVERSE breakout cable, and not a forward breakout. A forward cable can go from an SFF-8087 port on a SAS controller to 4 individual drives, but it won’t work the other way. You need a reverse cable to go from a backplane to 4 SATA ports on a motherboard.

You would just need a single 9211 card and the expander, plus 2 cables to connect the 9211 to the expander.

9

You reply faster than I can edit! :joy:

Thanks for the info on the reverse cable. My current one goes from my 9211 directly to my drives so I will need to get a reverse, but that’s not a big deal.

Looking through the expanders there seem to be 2 types. 3gbps and 6gbps. Neither are really expensive, I am assuming it would be better to get the 6gbps for this application?

Lastly are there special 8087 cables I need to get to keep the HDD activity/fail lights or will that not function properly with the expander?

10

The expanders I linked above were originally sold as 3Gbps cards, but HP released a firmware that increased the speed to 6Gbps for SAS drives only. SATA drives are stuck at 3Gbps regardless of what firmware version you use. But honestly, 3Gbps (375MB/sec) is still almost twice as fast as your hard drives will read or write (210MB/sec). And each drive gets their own 3Gbps link to the expander. The expander can connect to the SAS controller at up to 24Gbps (3Gbps per link, 4 links per cable, 2 cables connecting expander to controller), which means you could have 15 drives accessing simultaneously before you hit any IO bottlenecks. So with that all said, I would just look for the cheapest expander on the list that includes an expansion slot bracket.

Look for 8087 cables that are labelled as 36-pin to 36-pin. There are some 36-pin to 29-pin cables that don’t include the sideband wires necessary for drive activity. As for whether the expander will light up the activity LEDs, I’m honestly not sure - I’ve never connected my expander to a bay that has activity lights.

11

Thanks for the info. I just picked up some SAS 8087 that are supposedly sideband from 10Gtek on Amazon and a reverse 8087 to 4x SATA.

This is the Expander card I picked up. At $14 it was kind of a no brainer. Plus going this route, I have another BPN-SAS-846A in the 847 so if this backplane ever tanks I can swap it out with that one. Everything should be here early next week so I will check back in and let you know how it’s working out.

Thanks for all your help!!!

12

I just upgraded to a very similar setup to what your current config is. I went a different route on upgrading. I went from an AMD 8320 to Ryzen 7 1700, I kept the LSI and replaced all my 4TB drives with 14TB drives, going from 23TB to over 40TB while going from 8 drive down to 4 (not counting Cache SSD mirror). I am now saving over $100 / year in electric and the new drives are about twice as fast. I then sold my old MB and drives to help pay for the upgrade.
Joe

13

I Recently upgraded my server as well & i went with the supermicro cse-846 as well as a X9DR3-F with a pair of e5-2696 v2’s to keep it all cool i use a pair of gammax GT coolers it stays in th low 30’s idle & 45ish under 100% load i also added 128gb of ecc ram 10gb nic with a lsi 9211-8i & a Quadro moded the 2 back fans with 80mm bquiet i still have the stock 1200w dual psu’s & i have 8-10 3.5 hdd’s some sata some sas as well as 3 ssd’s a 1tb cashe drive & 2 more for plex meta data & vm’s And as far as speed wise i can get 1gbps when copying movies to it & as far as i know my back plain is stock but my sas card is flashed to it mode !

14

@Mthrboard
Thanks for all of your help! Server is working great. Anyone have recommendations on fans for the middle fan wall and the 2 at the rear? My server is in my basement where ambient is pretty cool to begin with so the fanwall is empty right now and drive temps are good, but I have the back 2 in and I can hear them on my 1st floor.

15

@Mthrboard On the HP expansion card you recommended if I understand correctly it’s just using PCI for power correct? If so is it okay if I run it on a backless PCI slot?

16

Yeah, a backless slot is fine. Or, if you don’t have any slots available, you could even use a powered PCI-E riser. That’s how I have one of my cards set up, it’s connected to a riser that uses a 6-pin PCI-E power connector.

1 Like
17

So something like this?
Should I be concerned about the Molex to SATA connection?

https://smile.amazon.com/Dr-meter-6-Pack-Powered-Adapter-Extension/dp/B078WLF5F3/ref=sr_1_4?dchild=1&keywords=powered%2Bpcie%2Briser&qid=1602090864&sr=8-4&th=1

18

Yep, that’ll do, but don’t use the included SATA cable. It’s a standard 6-pin PCI-E power connector, so you can just use one of the PCI-E cables from your power supply, if you have a spare. If not, I would look for a different riser that has a Molex or SATA power connector instead of the PCI-E connector.

19

Would I even need the 1x adapter portion with the USB cable if the riser is being powered off of the PCI-E power cable?

20

No, you only need the card with the x16 slot on it, everything else can be recycled. But I would only use this card as a last resort option, if you don’t have any open PCI-E slots on your board that will fit the card. If your board has an available slot, that’s the cleanest and easiest option.