Making the case for 15 boxes per rack

We are now putting exactly 15 1U machines into each rack. This number fits nicely for our setup.

15 machines equal to:
  • 15x3 switch ports. Each machine gets out-of-band management, in-band management and a VLAN trunk port.
  • 16x2 power ports, as each machine gets two PSUs for redundancy requirements. The switch obviously also has redundant PSUs, so this totals 32 power ports. With CEE7/4 plugs this already takes lots of space just for the plugs/power bars.
  • ~ 4000kW power requirement; we calculate with ~260W per server (actually more during boot etc).
  • 15x5 cables. Two power cables, 3 Ethernet patch cables per machine. Plus a few more for switch power and switch/rack interconnects.

It's already a challenge to actually handle all this stuff in a single rack. It gets really messy if you surpass 15 machines per rack.

We group these 15 1U machines in groups of three. After each group we leave 1U empty. After the second group we leave an extra 1U empty and mount the 4U switch on the read side. Then again we leave 2U empty and mount the remaining three groups below. Power bars go to the rear sides, heavier machines go to the bottom.
My colleague detailed this layout on the amd.co.at/AdminWiki.


Maybe we can do 20 machines per rack when we know more about power on the new HP ProLiant server generation.