There are so many different hardware sensors on machines now. If you have a machine with a iLOM (like Sun’s line of x86 hardware) you can probe all of this information over the network using ipmitool and direct it at the iLOM.
Usage for ipmitool over the network:
ipmitool -I lan -H (ip address of your lom) -U (username on lom) (command)
Typically Sun iLOMs use either “root” or “admin” as the username. Adjust to your hardware / environment. Note, ipmitool does send traffic across the wire in clear text. It isn’t encrypting a password, so keep that in mind.
Reporting on if there are any active fault LEDs on the box. (sunoem is of course, just for Sun gear)
ipmitool -I lan -H lomipddy -U root sunoem sbled get
Password: LOCATE | OFF OK | ON SERVICE | ON FAN_FAULT | OFF TEMP_FAULT | OFF PS_FAULT | ON MB/P0/D0/SVC | OFF MB/P0/D1/SVC | OFF MB/P0/D2/SVC | OFF MB/P0/D3/SVC | OFF ....
Reporting what components make up the machine (FRU = field replaceable units)
ipmitool -I lan -H ilomipaddy -U root fru
Password: FRU Device Description : Builtin FRU Device (ID 0) Product Manufacturer : ASPEED Product Name : BMC CONTROLLER FRU Device Description : /SYS (ID 27) Chassis Type : Rack Mount Chassis Chassis Part Number : 541-1982-06 Board Product : ASSY,MOTHERBOARD,DORADO/TUCANA Board Part Number : 511-1394-02 Board Extra : 50 Board Extra : DT_MB Product Manufacturer : SUN MICROSYSTEMS Product Name : SUN FIRE X4140 Product Part Number : 4534567-4 FRU Device Description : MB/P0/D4 (ID 12) Product Manufacturer : Micron Technology Product Name : 4096MB DDR-II 666 Product Part Number : 36HTF51272PY-667E1 Product Version : 0100 Product Serial : D93D8C07 FRU Device Description : MB/P1 (ID 7) Product Manufacturer : AMD Product Name : Six-Core AMD Opteron(tm) Processor 2435 Product Part Number : 1008 Product Version : 00
iLOM event logs of what happened to the machine in the past
ipmitool -I lan -H ilomipaddy -U root sel elist
Password: 1 | 12/24/2009 | 02:49:24 | Power Supply PS1/PWROK | State Deasserted 2 | 06/05/2010 | 07:49:00 | System ACPI Power State ACPI | S0/G0: working | Asserted 3 | 06/05/2010 | 07:49:03 | Power Supply PS1/PWROK | State Deasserted 4 | 06/05/2010 | 07:49:04 | Power Supply PS1/VINOK | State Deasserted 5 | 06/05/2010 | 07:55:14 | Power Supply PS1/PWROK | State Asserted 6 | 06/05/2010 | 07:55:16 | Power Supply PS1/VINOK | State Asserted 7 | 06/05/2010 | 08:09:04 | Power Supply PS0/PWROK | State Deasserted 8 | 06/05/2010 | 08:09:06 | Power Supply PS0/VINOK | State Deasserted 9 | 06/05/2010 | 08:09:10 | Power Supply PS0/PWROK | State Asserted a | 06/05/2010 | 12:23:46 | System ACPI Power State ACPI | S0/G0: working | Asserted b | 06/05/2010 | 12:23:51 | Power Supply PS0/VINOK | State Deasserted ....
And of course, current voltage / temperature / RPMs of fans, etc.
ipmitool -I lan -H ipaddyoflom -U root sdr elist
Password: ACPI | EAh | lnc | 7.0 | 0 unspecified INTSW | EBh | ok | 23.0 | MB/P0/PRSNT | 01h | ok | 3.0 | Device Present MB/P1/PRSNT | 02h | ok | 3.1 | Device Present MB/P0/T_CORE | 09h | ok | 3.0 | 7 degrees C MB/P1/T_CORE | 0Ah | ok | 3.1 | 18 degrees C MB/P0/V_VDDCORE | 05h | lnr | 3.0 | 0 Volts MB/P1/V_VDDCORE | 06h | ok | 3.1 | 1.03 Volts MB/P0/V_+0V9 | 0Dh | ok | 3.0 | 0.90 Volts MB/P1/V_+0V9 | 0Eh | ok | 3.1 | 0.90 Volts MB/P0/V_+1V8 | 11h | ok | 3.0 | 1.80 Volts MB/P1/V_+1V8 | 12h | ok | 3.1 | 1.79 Volts MB/P0/V_VDDNB | 15h | lnr | 3.0 | 0 Volts MB/P1/V_VDDNB | 16h | ok | 3.1 | 1.31 Volts MB/P0/PROCHOT | 19h | lnc | 3.0 | 0 unspecified MB/P1/PROCHOT | 1Ah | lnc | 3.1 | 0 unspecified MB/T_AMB | 32h | ok | 7.0 | 29 degrees C MB/V_+12V | 1Eh | ok | 7.0 | 12.10 Volts MB/V_+1V2HT | 26h | ok | 7.0 | 1.22 Volts MB/V_+1V5 | 20h | ok | 10.0 | 1.50 Volts
Note, since we are hitting a iLOM here to query this information using ipmitool, we’re not directly interacting with a machine. In all the examples above, the actual server was powered off. Some cool stuff!
Also, all of these ipmitool commands can also be ran locally from the O/S. Instead of using -I Lan to communicate over the network, you’ll want to use -I BMC (if you leave it out, it defaults to this) so the O/S knows to communicate with its own LOM.
Sweet! Only thing left is to gather this data and graph it into Zenoss or throw alerts in Nagios based off of poor values. =)
Note, that ipmitool gathering data is only as useful as the hardware it probes that support it. Various SPARC / x64 machines may return other values. Dell / HP gear may also report on things differently, so check out your vendor’s LOM’s ability to use ipmitool. You can download the source for ipmitool here.