Connect a Supermicro JBOD (Just a Bunch Of Disks) to an SSG (SuperStorage/Server Group)

Connecting a Supermicro JBOD (Just a Bunch Of Disks) to an SSG (SuperStorage/Server Group) that supports NVMe involves understanding the NVMe over PCIe architecture and how it expands externally.

Here I detail the key steps and considerations:

1. Understand NVMe architecture:

• Native NVMe: Unlike SAS/SATA, where an HBA (Host Bus Adapter) or RAID controller manages the drives, NVMe SSDs connect directly to PCIe lanes. This provides much higher performance and lower latency.
• JBOF (Just a Bunch of Flash): For NVMe expansion, Supermicro and other manufacturers use what are known as JBOFs. These are not simply SAS/SATA "expanders," but often contain more complex logic to handle the multiple PCIe connections of NVMe SSDs.
• Connectivity: The connection between an SSG (which will act as the "host" or "server") and a JBOF is done via PCIe cables or, more commonly, via OCuLink, SlimSAS cards and cables, or specific high-density NVMe cables that allow multiple PCIe lanes on a single cable.

2. Identify the necessary components:

• SSG (Server/Host): Your SSG (SuperStorage or SuperServer) must have available PCIe slots and support NVMe. Ideally, you'll need a PCIe/NVMe controller card (Host Bus Adapter - HBA) designed to connect to external NVMe JBODs. These cards typically have external OCuLink or SlimSAS ports.
• Supermicro JBOD/JBOF: The JBOD chassis must be a specific JBOF that supports NVMe drives and has the corresponding external ports (OCuLink, SlimSAS, etc.). Supermicro has models like the SSG-136R-NR32JBF, which are JBOFs designed for NVMe.
• Specific NVMe cables: You'll need high-speed cables that are compatible with your HBA ports on the SSG and the JBOF ports. These are not standard SAS or SATA cables. Look for OCuLink to OCuLink, SlimSAS to SlimSAS, or cables that match your exact hardware model.

3. Steps for connection:

1. Install the NVMe HBA card into the SSG:
• Turn off the SSG and unplug it from the power supply.
• Open the SSG casing.
• Insert the NVMe HBA card into a compatible PCIe slot (make sure the slot provides enough PCIe lanes, for example, PCIe x16 or x8, as required by the card and JBOF).
• Secure the card.
• Close the SSG casing.
2. Prepare the JBOF:
• Make sure the NVMe drives are properly installed in the JBOF.
• Verify that the JBOF is connected to power and that all NVMe controller or expander modules are in place if it has them.
3. Connect the NVMe cables:
• Connect one end of the high-speed NVMe cables to the external ports of the HBA card on the SSG.
• Connect the other end of the cables to the corresponding input ports on the JBOF. It's crucial to ensure the cables are properly connected and fit securely. Some JBOFs may have multiple ports to maximize bandwidth.
4. Turn on the equipment:
• First, power on the JBOF and wait for it to fully boot and for its NVMe drives to be detected internally.
• Then, turn on the SSG.
5. Operating System/BIOS Configuration:
• Access the SSG's BIOS/UEFI. In many cases, NVMe SSDs should be detected automatically, similar to how internal PCIe devices are detected.
• You may need to install specific drivers for your NVMe HBA in the operating system if they are not automatically detected or if you want to optimize performance.
• Once the JBOF NVMe SSDs are visible to the operating system, you can format them, create software RAID volumes (if desired), or use them as individual disks.

Important considerations:

• Compatibility: The most critical thing is to verify compatibility! Consult the user manuals for both the SSG and the Supermicro JBOF. Supermicro has an extensive support site where you can find compatibility lists for its systems, including recommended HBA cards and supported JBOF models.
• PCIe Version: Ensure the SSG, HBA card, and JBOF all support the same PCIe version (PCIe Gen3, Gen4, or Gen5) for maximum performance. While they are backward compatible, performance will be limited to the lowest version.
• Bandwidth: A full NVMe JBOF can have a large number of SSDs. Make sure your SSG's HBA card and PCIe lanes can provide enough bandwidth to handle all those drives at peak performance.
• Cooling and Power: NVMe systems, especially JBOFs with multiple drives, generate considerable heat and require a robust power supply. Make sure your rack has adequate cooling and that the power supplies are sufficient.
• Monitoring: Use Supermicro monitoring tools (such as IPMI) to check the disk status and temperature of the JBOF and SSG.

Since there are many Supermicro SSG and JBOF models available, I strongly recommend that you consult your model-specific manuals or contact Supermicro technical support for the most accurate and up-to-date connection instructions.