We just ordered some new servers from Dell, and got some SC1425 (to make load balancers - those SC1425 servers are really cheap for dual CPUs) and some PowerEdge 1850.

Those new boxes are propelled by the latest Intel Xeon CPUs, which feature EM64T (they are 64 Bit CPUs) - the load balancers only have one (with hyper-threading), and the PE1850 got 2 CPUs.

Among the cool features on those boxes, there is one which is very nice if you want to monitor them properly, it's IPMI.
IPMI is a fairly recent standard which provides a simple abstraction layer to monitor and interact with the computer hardware - and it also includes a watchdog.
On Dell servers, this means no more complicate installation of partially closed source code to get OpenManage running.

For the SC1425, I wanted to have the most stable system possible (who would like to have load balancers crash - even if those are redundant) and so chose Debian Sarge (3.1 Stable) for i386 - and decided not to play with 64 bit support.
Moreover I'm not sure that would have changed much speed wise.

Installation of Debian went smoothly on this system and everything was supported (the SC1425 were using SATA - I'm not sure the SCSI models wouldn't have issues).
The next steps involved making a custom kernel (with the latest and greatest 2.6.14 with the ck performance patches for servers) including only the hardware support I wanted (IPMI), SATA, e1000, etc… plus the features I needed in the load balancers (plus some extra stuffs which could be useful someday).
Here is my kernel .config for Dell SC1425.
Here is an IPMI patch I back-ported from the next kernel which greatly improves performance (unless you want to wait a minute to get the sensor status) and fixes issues on some Dell hardware.

In order to get IPMI working you can follow the steps described here - to sum up they are :

  • Compile the kernel with IPMI support (I added i2c support for the piix chipset as well) and install it
  • Install OpenIPMI (it should be in the apt repository)
  • Load the IPMI modules you want (in my case : ipmi_msghandler, ipmi_si, ipmi_poweroff, ipmi_devintf, ipmi_watchdog)
  • cat /proc/devices | grep ipmi - get the number here
  • mknod -m 0600 /dev/ipmi0 c 254 0 (254 being the number you had)
  • Get ipmitool, decompress it, go in the source directory, do a dpkg-buildpackage and then install it
  • Check it's working fine after a reboot : ipmitool -I open chassis status
  • Have fun :) (check the doc I pointed before)

Getting all this working on the SC1425 was fairly easy (except the patch I made after which was very helpful).
The hard part came when playing with the PE1850 and Debian AMD64 - those boxes are pretty new and the regular Debian CD wouldn't play fair with the hardware Raid card I had (Perc4e/SI).
I ended picking up Debian Testing (code-name "etch") boot CDs which you can find here.
While in the installer, pick Stable (unless you really want etch) - and for the raid card detection which won't work, you'll have to hand pick megaraid_mbox.

Once you have installed all this, DON'T REBOOT - go to a shell, chroot in your new installation and get the tools required to build a kernel - as the stable 2.6 kernel won't work with the raid card which is too new.
Get the latest kernel, patch it with the previously mentioned patches and build it with support for the latest megaraid drivers and the Intel chipset for IDE and i2c (here is my kernel .config for the PE1850).
I chose to build the raid support in the kernel and not as a module to not have to play with initrd.
Check your /etc/modules before rebooting - and DON'T add the IPMI modules in there yet (or system will hand on loading ipmi_si), and reboot.

Once you managed all this - go in /etc/modprobe.d/ and make a text file called ipmi_si with your favorite editor - vim !
x86_64 don't save the APIC table at boot, and IPMI have issues finding its stuffs, you'll have to put the following for the PE1850 :
options ipmi_si type="kcs" ports="0xca8" regspacings="4"
To get those numbers (in case you are on another system) - get dmidecode and look for DMI type 38.

You can now try loading the DMI modules, it should work as expected :)

UPDATE (20/12/2005) : For a PE 2850 - just pick the kernel .config for the PE1850 and you will be fine (if you have a dual-core dual-cpu box, don't forget to increase the maximum number of CPUs).