In a SAN environment when dealing with external storage concepts such as EMC BCV’s, you’ll often have a request to create volumes on two different machines that are identical so replication on the back-end can occur.
When you look at a LUN presented to Solaris, it’ll appear with a cryptic name like the following:
103. c20t60060480000190100665533030393836d0 <EMC-SYMMETRIX-5771 cyl
36826 alt 2 hd 60 sec 128>
<a>/scsi_vhci/ssd@g60060480000190100665533030353339</a>
The c20 relates to the HBA (Fiber, SCSI, iSCSI) that provides a path to the device. The “middle” sequence 60060480000190100665533030393836 between the “t (target)” and “d (device” is the WWN of the LUN.
Now, say your SAN engineer approaches you with some information like the following….
PROD DEV: 936
PROD LUN: 47
STAGE DEV: 986
STAGE LUN: 68
Ok… so what does that mean to us? Using luxadm, we can probe a target to find out some specifc information about its back-end LUN name.
(root)> luxadm display
/dev/rdsk/c20t60060480000190100665533030393836d0s2
DEVICE PROPERTIES for disk:
/dev/rdsk/c20t60060480000190100665533030393836d0s2
Vendor: EMC
Product ID: SYMMETRIX
Revision: 5771
Serial Num: xxxxxxxxxxx
Unformatted capacity: 138105.000 MBytes
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0xffff
Device Type: Disk device
Path(s):
/dev/rdsk/c20t60060480000190100665533030393836d0s2
<a>/devices/scsi_vhci/ssd@g60060480000190100665533030393836:c,raw</a>
Controller
<a>/devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0</a>
** Device Address 50060482d52d2e76,68**
Host controller port WWN 210100e08ba0147a
Class primary
State ONLINE
Controller
<a>/devices/pci@17c,600000/SUNW,qlc@1,1/fp@0,0</a>
** Device Address 50060482d52d2e59,68
** Host controller port WWN 210100e08ba0da73
Class primary
State ONLINE
The Device Address is the field we are interested in. This shows us that the WWN of the Port we are plugged into is 50060482d52d2e59and the LUN number is 68.
Now that we have the LUN number, we know what this LUN maps to. We can then find the cooresponding LUN on the other machine (LUN Number 47) which maps to the BCV pair.
There are some other useful leadville commands that you may be interested in…
To display all HBAs available for use:
(root)> luxadm -e port
<a>/devices/pci@15c,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
CONNECTED
<a>/devices/pci@15d,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@15d,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@17c,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@17c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
CONNECTED
<a>/devices/pci@17d,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@17d,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@19c,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@19c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@19d,600000/SUNW,qlc@1/fp@0,0:devctl</a>
NOT CONNECTED
<a>/devices/pci@19d,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
NOT CONNECTED
Now that you have the device name, you can map that back to what device it cooralates to under /dev.
In this case since i’m using a Fiber channel HBA…
(root)> ls -l /dev/fc | grep
<a>/devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
lrwxrwxrwx 1 root root 55 Sep 26 2007 **fp1 ->**
<../../devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0:devctl>
Sweet. So one of my HBAs is fp1.
Want to see more detailed information about what that Fiber HBA is connected to?
umt1a-bio-stg1:~
(root)> luxadm -e dump_map
<a>/devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 611813 0 50060e8004769000 50060e8004769000 0x0 (Disk
device)
1 624613 0 50060482d52d2e76 50060482d52d2e76 0x0 (Disk
device)
2 617c13 0 210100e08ba0147a 200100e08ba0147a 0x1f (Unknown
Type,Host Bus Adapter)
Note that the Port WWN 50060482d52d2e76 is the same WWN we saw above when looking for the LUN number.
Want a dump of all LUNs attached to a controller?
(root)> cfgadm -o show_FCP_dev -al
Ap_Id Type Receptacle Occupant
Condition
c4::50060482d52d2e76,0 disk connected configured
unknown
c4::50060482d52d2e76,69 disk connected configured
unknown
c4::50060482d52d2e76,70 disk connected configured
unknown
c4::50060482d52d2e76,71 disk connected configured
unknown
c4::50060482d52d2e76,72 disk connected configured
unknown
c4::50060482d52d2e76,73 disk connected configured
unknown
c4::50060482d52d2e76,74 disk connected configured
unknown
c4::50060482d52d2e76,75 disk connected configured
unknown
c4::50060482d52d2e76,76 disk connected configured
unknown
c4::50060482d52d2e76,77 disk connected configured
unknown
c4::50060482d52d2e76,78 disk connected configured
unknown
c4::50060482d52d2e76,79 disk connected configured
unknown
c4::50060482d52d2e76,80 disk connected configured
unknown
<snip>
....
Another super useful utility is the new fcinfo command which was introduced into Solaris 10. The -l (linkstat) shows some detailed statistics on the HBA.
(root)> fcinfo hba-port -l
HBA Port WWN: 210100e08ba0147a
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Model: 375-3108-xx
Firmware Version: 3.3.26
FCode/BIOS Version: fcode: 1.13;
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 200100e08ba0147a
Link Error Statistics:
Link Failure Count: 0
Loss of Sync Count: 0
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
With the WWN of the HBA, we can query the remote port information…
(root)> fcinfo remote-port -p 210100e08ba0147a
Remote Port WWN: 50060482d52d2e76
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 50060482d52d2e76
Remote Port WWN: 50060e8004769000
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 50060e8004769000
Check it out! Its the same 50060482d52d2e76 we saw twice before with the WWN of the port we’re plugged into the fabric with.
Throw in a -s, and it’ll return all SCSI targets with their LUN Number.
(root)> fcinfo remote-port -p 210100e08ba0147a -s
Remote Port WWN: 50060482d52d2e76
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 50060482d52d2e76
LUN: 0
Vendor: EMC
Product: SYMMETRIX
OS Device Name: /dev/rdsk/c4t50060482D52D2E76d0s2
LUN: 69
Vendor: EMC
Product: SYMMETRIX
OS Device Name:
/dev/rdsk/c20t60060480000190100665533030344539d0s2
LUN: 70
Vendor: EMC
Product: SYMMETRIX
OS Device Name:
/dev/rdsk/c20t60060480000190100665533030344639d0s2
LUN: 71
Vendor: EMC
Product: SYMMETRIX
OS Device Name:
/dev/rdsk/c20t60060480000190100665533030353039d0s2
LUN: 72
Vendor: EMC
Product: SYMMETRIX
OS Device Name:
/dev/rdsk/c20t60060480000190100665533030353139d0s2
LUN: 73
Vendor: EMC
Product: SYMMETRIX
OS Device Name:
/dev/rdsk/c20t60060480000190100665533030353239d0s2
Want to force a specific Fiber HBA to reinitialized and re-login into the Fabric?
(root)> luxadm -e forcelip <device>
i.e.
(root)> luxadm -e forcelip
<a>/devices/pci@15c,600000/SUNW,qlc@1,1/fp@0,0:devctl</a>