Viewing PCI bus throughput on Solaris systems
The PCI and PCI express interconnect technologies have become the defacto standard for connecting peripherals to most motherboards. 64-bit 66 MHZ PCI interconnects run with speeds up to 528MB/s, and share the available bandwidth between devices on the PCI bus. PCI express x32 runs with speeds up to 8 GB/s, and provides dedicated “lanes” to connect each peripheral directly to the motherboards chipsets. This allows PCI express devices to utilize all of the available bandwidth, and maximizes throughput since PCI express devices do not need to compete with other devices on the bus. On systems that utilize gigabit Ethernet and 4 GB/s HBAs, it is possible to saturate a 33 or 66 MHZ PCI bus during peak loads. If you happen to be using a PCI interconnect and the Solaris Operating System, you can easily measure the current PCI bus utilization with the busstat(1m) command:
$ busstat -w pcis0,pic0=dvma_cycles,pic1=dvma_wd_xfr -w pcis1=pic0=dvma_cycles,pic1=dvma_wd_xfr 5
time dev event0 pic0 event1 pic1 5 pcis0 dvma_cycles 482928 dvma_wd_xfr 115184 5 pcis1 dvma_cycles 20217638 dvma_wd_xfr 32815051 10 pcis0 dvma_cycles 482778 dvma_wd_xfr 115046 10 pcis1 dvma_cycles 20989675 dvma_wd_xfr 34068291 15 pcis0 dvma_cycles 482643 dvma_wd_xfr 115069 15 pcis1 dvma_cycles 20834960 dvma_wd_xfr 33817211
bustat’s “-w (instrument PIC to collect statistics) option will program the PCI interconnects PICs (programmable interrupt controller) to collect the statistic passed as an option. These statistics including the number of PIO mode reads and writes, bus throughput, stream transfers, interrupts, and the number of DMA and DVMA operations performed. These counters are documented in the Systems Performance and Tuning book, and can also be viewed by sifting through writing Solaris PCI device drivers for Sun SPARC platforms.
Matt Warner on March 10th, 2009
Thanks for posting this information. I think your note about bus speed is supposed to be bits, with a little “b”, not bytes, with a capital “B.” So PCIe speeds are up to 8Gb/s, not 8GB.