Observation: It only recognizes one DIMM slot (or 4GB RAM - not sure) when I have the 16xPCie populated with a HP OEM version of a Mellanox ConnectX-3 dual port 10/40/56GB network card installed. This is repeatable over many tests. I also have the M.2 NVME populated as a boot device. I will have to look at the motherboard block diagram and the processor data sheet. Maybe I don’t have enough resources/PCie lanes between the CPU and the motherboard for M.2 NVME+Dual RAM DIMMS+Mellanox 8x PCIE card? Also M.2 wifi card?
Time to educate myself and conduct more testing.
Update:
After much hardware testing, it looks like two of my Mellanox ConnectX-3 cards cause some sort of conflict with the motherboard and cause DIMM1 to not be recognized to the BIOS. I was thinking it was a motherboard flex issue, but I loosened up the motherboard screws to free float the board and took off the card bracket and still had this exact repeatable problem with two of the same cards on two identical machines. Other network cards did not have this issue. I even flashed the NIC’s to the latest firmware. Hopefully a future HP BIOS update will fix it, or I will have to choose a different NIC or live with one DIMM slot. Am playing with 10G/40G fiber, so thought I might as well deploy fiber to these machines and my UNRAID NAS box.
Solution: Its a SMBUS conflict and it is fairly common on “OEM” hardware used outside of its scope. On the PCIe card there are two pins B5/B6 that send the SMBUS signal/data to the motherboard which is supposed to be a standard form of hardware state communication. If the signal is not standard (Like in my OEM HP NIC) it can conflict with the motherboard BIOS/operation. Fix is to insulate those two pins on the PCIe card edge with nail polish or tape. I cut and applied extra brother label maker tape (2mm wide) to insulate those pins. Working great, everything back to normal. See here: Yannick's Tech Blog: Modding a Dell Perc 6 / Dell H310 / Dell H710 (other LSI 1078 or 9223-8i based) SAS Raidcontroller
I̶ ̶s̶p̶o̶k̶e̶ ̶t̶o̶o̶ ̶s̶o̶o̶n̶.̶ ̶A̶f̶t̶e̶r̶ ̶a̶ ̶f̶e̶w̶ ̶r̶e̶b̶o̶o̶t̶s̶,̶ ̶I̶ ̶l̶o̶s̶t̶ ̶D̶I̶M̶M̶1̶ ̶s̶l̶o̶t̶ ̶a̶g̶a̶i̶n̶.̶ ̶G̶o̶t̶ ̶t̶o̶ ̶b̶e̶ ̶h̶a̶r̶d̶w̶a̶r̶e̶.̶ ̶I̶ ̶w̶i̶l̶l̶ ̶R̶M̶A̶ ̶i̶t̶.̶
̶I̶ ̶h̶a̶d̶ ̶t̶h̶e̶ ̶s̶a̶m̶e̶ ̶p̶r̶o̶b̶l̶e̶m̶,̶ ̶m̶e̶m̶o̶r̶y̶ ̶s̶l̶o̶t̶ ̶#̶1̶ ̶(̶D̶I̶M̶M̶ ̶1̶)̶ ̶d̶i̶d̶ ̶n̶o̶t̶ ̶r̶e̶c̶o̶g̶n̶i̶z̶e̶ ̶r̶a̶m̶ ̶n̶o̶ ̶m̶a̶t̶t̶e̶r̶ ̶w̶h̶a̶t̶ ̶I̶ ̶d̶i̶d̶.̶ ̶I̶ ̶h̶a̶v̶e̶ ̶t̶h̶r̶e̶e̶ ̶i̶d̶e̶n̶t̶i̶c̶a̶l̶ ̶m̶a̶c̶h̶i̶n̶e̶s̶ ̶a̶n̶d̶ ̶s̶w̶a̶p̶p̶e̶d̶ ̶m̶e̶m̶o̶r̶y̶ ̶b̶e̶t̶w̶e̶e̶n̶ ̶t̶h̶e̶m̶,̶ ̶s̶t̶i̶l̶l̶ ̶n̶o̶ ̶l̶u̶c̶k̶.̶ ̶I̶ ̶e̶v̶e̶n̶ ̶s̶c̶h̶e̶d̶u̶l̶e̶d̶ ̶a̶n̶ ̶R̶M̶A̶ ̶w̶i̶t̶h̶ ̶H̶P̶ ̶(̶T̶h̶e̶y̶ ̶p̶a̶y̶ ̶s̶h̶i̶p̶p̶i̶n̶g̶ ̶b̶o̶t̶h̶ ̶w̶a̶y̶s̶ ̶i̶f̶ ̶y̶o̶u̶ ̶a̶r̶e̶ ̶s̶t̶i̶l̶l̶ ̶i̶n̶ ̶w̶a̶r̶r̶a̶n̶t̶y̶)̶.̶ ̶O̶n̶e̶ ̶l̶a̶s̶t̶ ̶d̶i̶t̶c̶h̶ ̶e̶f̶f̶o̶r̶t̶,̶ ̶I̶ ̶p̶u̶l̶l̶e̶d̶ ̶t̶h̶e̶ ̶B̶I̶O̶S̶/̶C̶M̶O̶S̶ ̶b̶a̶t̶t̶e̶r̶y̶ ̶f̶o̶r̶ ̶1̶0̶ ̶m̶i̶n̶u̶t̶e̶s̶ ̶a̶n̶d̶ ̶p̶u̶t̶ ̶i̶t̶ ̶b̶a̶c̶k̶ ̶i̶n̶ ̶a̶n̶d̶ ̶r̶e̶b̶o̶o̶t̶e̶d̶.̶ ̶B̶I̶O̶S̶ ̶c̶o̶m̶p̶l̶a̶i̶n̶e̶d̶ ̶a̶b̶o̶u̶t̶ ̶c̶h̶e̶c̶k̶s̶u̶m̶ ̶e̶r̶r̶o̶r̶ ̶a̶n̶d̶ ̶h̶a̶d̶ ̶t̶o̶ ̶r̶e̶b̶o̶o̶t̶ ̶o̶n̶e̶ ̶m̶o̶r̶e̶ ̶t̶i̶m̶e̶.̶ ̶N̶o̶w̶ ̶b̶o̶t̶h̶ ̶D̶I̶M̶M̶ ̶s̶l̶o̶t̶s̶ ̶a̶r̶e̶ ̶r̶e̶c̶o̶g̶n̶i̶z̶e̶d̶.̶ ̶T̶h̶i̶s̶ ̶w̶a̶s̶ ̶p̶r̶o̶b̶a̶b̶l̶y̶ ̶d̶u̶e̶ ̶t̶o̶ ̶s̶o̶m̶e̶ ̶B̶I̶O̶S̶ ̶c̶o̶r̶r̶u̶p̶t̶i̶o̶n̶ ̶a̶f̶t̶e̶r̶ ̶t̶h̶e̶ ̶l̶a̶s̶t̶ ̶B̶I̶O̶S̶ ̶u̶p̶d̶a̶t̶e̶ ̶I̶ ̶d̶i̶d̶.̶ ̶T̶r̶y̶ ̶i̶t̶ ̶a̶n̶d̶ ̶s̶e̶e̶ ̶i̶f̶ ̶i̶t̶ ̶w̶o̶r̶k̶s̶ ̶t̶h̶e̶ ̶s̶a̶m̶e̶ ̶f̶o̶r̶ ̶y̶o̶u̶.̶
̶