In case you wanted something more than an informal introduction into the world of fat-tree and torus networks — here you are. A formal problem statement (and solution!) in an academic form is given in these two articles hosted at arXiv:
K.S. Solnushkin, “Automated Design of Two-Layer Fat-Tree Networks”
(link: arXiv:1301.6179) (cite with BibTeX)
K.S. Solnushkin, “Automated Design of Torus Networks”
(link: arXiv:1301.6180) (cite with BibTeX)
arXiv is the leading open access repository of scientific papers.
Hi,
How to check the topology of my cluster. I know its a fat tree. But is there any option to check the configuration details of my infiniband topology like
which topology ?
what blocking ratio it is using ?
Thanks in advance !
Hi Sandeep,
1. There are tools made by InfiniBand vendors that can help you discover the topology. But you probably need to be a super-user to run them on your cluster.
2. (Difficult and unreliable path) You can write a program that sends many packets in all-to-all manner between all possible pairs of hosts and writes down latency of sending a packet and then receiving a reply. Pairs of hosts that consistently have a low latency when communicating are probably connected to the same edge switch. Those that always exhibit a higher latency are probably connected to different edge switches.
3. If you have physical access to the cluster, you can just track all InfiniBand cables manually and detect the port they come from and the port they go to. Draw a diagram with switches and then draw each cable as you inspect it. After you are done, you will see your fat-tree on paper.
4. Just ask your system administrator / vendor engineer who installed it for you. There is nothing secret in these details, so there are no reasons for them to withhold this information.
Wish you luck!