While the days of conflicting IRQ lines are almost behind us, it is still useful to see the interrupt request lines associated with the devices in a given system. To view IRQ to device mappings on Solaris systems, the mdb ::interrupts DCMD (requires a recent build of Nevada) can be used:
$ mdb -k
Loading modules: [ unix krtld genunix specfs ufs pcipsy ip sctp fctl nca md sd zfs
random logindmux ptm cpc fcip sppp nfs ]
> ::interrupts
Device Shared Type MSG # State INO Mondo Pil CPU
glm#0 no Fixed --- enbl 0x14 0x7d4 4 0
hme#0 no Fixed --- enbl 0x21 0x7e1 6 0
su#1 no Fixed --- enbl 0x2a 0x7ea 12 0
su#0 no Fixed --- enbl 0x29 0x7e9 12 0
se#0 no Fixed --- enbl 0x2b 0x7eb 12 0
m64#0 no Fixed --- enbl 0xf 0x7cf 9 0
power#0 no Fixed --- enbl 0x25 0x7e5 14 0
uata#0 no Fixed --- enbl 0x20 0x7e0 4 0
On Linux systems you can cat /proc/interrupts to get IRQ information:
$ cat /proc/interrupts
CPU0
0: 162248 XT-PIC timer
1: 335 XT-PIC i8042
2: 0 XT-PIC cascade
5: 31561 XT-PIC eth0
6: 1 XT-PIC uhci_hcd:usb1
7: 1 XT-PIC parport0
8: 1 XT-PIC rtc
9: 165 XT-PIC acpi, ide2, uhci_hcd:usb2
12: 278 XT-PIC i8042
14: 8285 XT-PIC ide0
15: 5596 XT-PIC ide1
NMI: 0
LOC: 0
ERR: 0
MIS: 0
I’ve been a Solaris Volume Manager (formerly disksuite) user for the past 6-years, and have always been annoyed that I had to use dXXX and hspXXX to label mirrors, stripes, sub-devices and hot spares. As of Nevada build 37, you are no longer required to use dXXX and hspXXX, and are free to choose a name that fits the device:
$ metainit foomirror1 1 1 c1t5d0s0
foomirror1: Concat/Stripe is setup
$ metainit foomirror2 1 1 c1t6d0s0
foomirror2: Concat/Stripe is setup
$ metainit foovol -m foomirror1
foovol: Mirror is setup
$ metattach foovol foomirror2
foovol: submirror foomirror2 is attached
$ metastat -c
foovol m 16GB foomirror1 foomirror2 (resync-0%) foomirror1 s 16GB c1t5d0s0 foomirror2 s 16GB c1t6d0s0
W00t! W00t!
While catching up on one of the Solaris discussion lists I subscribe to, I came across Peter Tribble’s miniature Solaris installation procedure. Peter’s site has some super useful information, and his procedure reminded me of the work Clay did years ago to strip down the Solaris systems we put into production.
After installing Solaris 10 Update 1 last week, I wanted to see how well MPXIO worked with a Clariion CX500. After reading through a bunch of BLOG posts and various manual pages, I decided to configure a few LUNs and present them to the host. Once I created and masked the LUNs, I added the necessary zoning configurations, and ran format to see if the LUNs were visible:
$ format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@83,4000/FJSV,ulsa@2,1/sd@0,0
1. c0t1d0
/pci@83,4000/FJSV,ulsa@2,1/sd@1,0
2. c2t50060168082006E2d0
/pci@83,2000/pci@2/lpfc@4/fp@0,0/ssd@w50060168082006e2,0
3. c2t50060160082006E2d0
/pci@83,2000/pci@2/lpfc@4/fp@0,0/ssd@w50060160082006e2,0
4. c2t50060168082006E2d1
/pci@83,2000/pci@2/lpfc@4/fp@0,0/ssd@w50060168082006e2,1
5. c2t50060160082006E2d1
/pci@83,2000/pci@2/lpfc@4/fp@0,0/ssd@w50060160082006e2,1
6. c3t50060169082006E2d0
/pci@83,2000/pci@2/lpfc@5/fp@0,0/ssd@w50060169082006e2,0
7. c3t50060161082006E2d0
/pci@83,2000/pci@2/lpfc@5/fp@0,0/ssd@w50060161082006e2,0
8. c3t50060161082006E2d1
/pci@83,2000/pci@2/lpfc@5/fp@0,0/ssd@w50060161082006e2,1
9. c3t50060169082006E2d1
/pci@83,2000/pci@2/lpfc@5/fp@0,0/ssd@w50060169082006e2,1
Specify disk (enter its number): ^D
The devices were visible, but several targets were listed as “drive type unknown.” Since I had yet to configure the multi-pathing solution (MPXIO), and since the storage array I was testing with was an active/passive array, the eight devices off c2 and c3 were actually two devices seen down four paths. To get the multi-pathing configured properly, I needed to add the array type to the device-type-scsi-options-list in /kernel/drv/scsi_vhci.conf:
$ cat /kernel/drv/scsi_vhci.conf
#
# Copyright 2004 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
#pragma ident "@(#)scsi_vhci.conf 1.9 04/08/26 SMI"
#
name="scsi_vhci" class="root";
#
# Load balancing global configuration: setting load-balance="none" will cause
# all I/O to a given device (which supports multipath I/O) to occur via one
# path. Setting load-balance="round-robin" will cause each path to the device
# to be used in turn.
#
load-balance="round-robin";
#
# Automatic failback configuration
# possible values are auto-failback="enable" or auto-failback="disable"
auto-failback="disable";
#
# For enabling MPxIO support for 3rd party symmetric device need an
# entry similar to following in this file. Just replace the "SUN SENA"
# part with the Vendor ID/Product ID for the device, exactly as reported by
# Inquiry cmd.
#
device-type-scsi-options-list =
"DGC RAID", "symmetric-option";
symmetric-option = 0x1000000;
Once the scsi_vhci.conf configuration file was adjusted, I ran the stmsboot command to enable MPXIO multi-pathing on all fibre channel interfaces:
$ stmsboot -e
WARNING: This operation will require a reboot.
Do you want to continue ? [y/n] (default: y) y
The changes will come into effect after rebooting the system.
Reboot the system now ? [y/n] (default: y) y
This required a reboot, so I rebooted the machine, and low and behold the devices appeared correctly:
$ format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@83,4000/FJSV,ulsa@2,1/sd@0,0
1. c0t1d0
/pci@83,4000/FJSV,ulsa@2,1/sd@1,0
2. c4t6006016061B71000AD0810C9979CD911d0
/scsi_vhci/ssd@g6006016061b71000ad0810c9979cd911
3. c4t6006016061B7100055B12704989CD911d0
/scsi_vhci/ssd@g6006016061b7100055b12704989cd911
Specify disk (enter its number): ^D
I have a bunch of additional testing to do, but things are working pretty well so far.
If you use MPXIO on your Solaris servers to multiplex I/Os over one or more HBAs, it is often valuable to view the I/O distribution on a per HBA basis. This is easily accomplished with the iostat “-X” option:
$ iostat -zxnXM 5
extended device statistics
r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b device
1170.6 0.0 146.3 0.0 0.0 1.4 0.0 1.2 1 98 c4t6006016061B71000AD0810C9979CD911d0
585.4 0.0 73.2 0.0 0.0 0.0 0.0 0.0 0 0 c4t6006016061B71000AD0810C9979CD911d0.fp0
585.2 0.0 73.2 0.0 0.0 0.0 0.0 0.0 0 0 c4t6006016061B71000AD0810C9979CD911d0.fp1
extended device statistics
r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b device
1175.2 0.0 146.9 0.0 0.0 1.3 0.0 1.1 1 98 c4t6006016061B71000AD0810C9979CD911d0
587.6 0.0 73.4 0.0 0.0 0.0 0.0 0.0 0 0 c4t6006016061B71000AD0810C9979CD911d0.fp0
587.6 0.0 73.4 0.0 0.0 0.0 0.0 0.0 0 0 c4t6006016061B71000AD0810C9979CD911d0.fp1
In this example we can see that the Solaris host (a Fujitsu 250 w/ two Emulex 9802s) was able to push almost 150 MB/s to our storage array (a Clariion CX500), and was balancing I/O evenly across both adaptors. Nice!