Fat-tree networks are traditionally built with InfiniBand switches. But what if you wanted to use benefits of fat-trees in Ethernet-based networks? It is possible, although with some reservations. One of my readers tried to build a fat-tree network with some legacy hardware, and it didn’t work. More specifically, the switches failed to recognise and use multiple paths in the network (see more on which technologies must be supported).
This page is intended to list hardware that is known to work out of the box — at least, according to the manufacturers. Feel free to add your comments to this page if you have updated information.
1. Mellanox
Mellanox Technologies produces InfiniBand and Ethernet switches, and both types are suitable for building fat-tree networks. For Ethernet products see, for example, this white paper from Mellanox: “Mellanox Virtual Modular Switch”.
Switch models recommended by the manufacturer:
1. SX1036 (36-port QSFP 40/56GbE 1U Ethernet switch). If you wish to use 10Gb/s speed, then using special breakout (also called fanout) cables you can “split” some 40Gb/s ports into four 10Gb/s ports but leave other ports running at 40Gb/s. On SX1036, some of the switch ports can be split into four, and some into two 10Gb/s SFP+ ports (and some ports get disabled during this procedure), eventually resulting into 56 10Gb/s ports. The remaining 8 ports can continue to work in a 40Gb/s mode or can be downgraded to 10Gb/s mode. See section 3.11 in the switch user manual for more details.
2. SX1012 (12-port QSFP 40/56GbE 1U half-width Ethernet switch). Each of the 12 ports can be split into four (or, if you wish, two) 10Gb/s ports, thereby resulting into up to 48 10Gb/s ports.
3. SX1024 (48-port SFP+ 10GbE, 12 port QSFP 40/56GbE, 1U Ethernet switch). One or two 40Gb/s ports can be split into four 10Gb/s ports, should you need it, but at the same time one or two other 40Gb/s ports will become disabled. Splitting ports on this switch is not much needed since it already provides both 10Gb/s and 40Gb/s ports.
You can buy Mellanox switches from Colfax Direct, a web shop of Colfax International.
2. Dell
Switches made by Dell are also suitable for building fat-tree networks. See this guide, “Designing Scalable 10Gb Ethernet Networks”, by Roger Goff: Part 1, Part 2.
Switch model recommended by the manufacturer:
Dell Networking Z9000 (32-port QSFP 40GbE 2U Ethernet switch)
(This switch is apparently made by Force10, as follows from the former model name in the guide, “Dell Force10 Z9000TM“. Force10 was acquired by Dell in 2011)
