Displaying NFS Client Operations
I have spent a fair amount of time in the past few weeks reading through RFC 1813 (NFSv3), and wanted to find a way to see if a given client operation resulted in a physical or logical I/O operation. Since the DTrace FBT provider allows you to observe entry and exit from 99.999% of the functions in the Solaris kernel, I decided DTrace would be ideal for answering this question. After reading through the vast majority of the NFSv3 client source code on opensolaris.org, I crafted a DTrace program named nfstrace to display NFSv3 client operations by process:
$ nfstrace
Executable Operation Type Time Size Path mkdir nfs3_lookup physical 359953 N/A /opt/nfs/htdocs/test mkdir nfs3_getattr logical 17481 N/A /opt/nfs/htdocs/test mkdir nfs3_getattr logical 7577 N/A /opt/nfs/htdocs/test mkdir nfs3_mkdir physical 843500 N/A /opt/nfs/htdocs/test/test rmdir nfs3_access logical 19772 N/A /opt/nfs/htdocs/test rmdir nfs3_lookup logical 69222 N/A /opt/nfs/htdocs/test/test rmdir nfs3_access logical 7744 N/A /opt/nfs/htdocs/test rmdir nfs3_rmdir physical 1390474 N/A /opt/nfs/htdocs/test/test touch nfs3_access logical 19566 N/A /opt/nfs/htdocs/test touch nfs3_lookup logical 68824 N/A /opt/nfs/htdocs/test/1 touch nfs3_getattr logical 17842 N/A /opt/nfs/htdocs/test/1 touch nfs3_access logical 7746 N/A /opt/nfs/htdocs/test touch nfs3_lookup logical 26527 N/A /opt/nfs/htdocs/test/1 touch nfs3_setattr logical 597203 N/A /opt/nfs/htdocs/test/1 ln nfs3_lookup physical 299999 N/A /opt/nfs/htdocs/test/2 ln nfs3_access physical 20033 N/A /opt/nfs/htdocs/test ln nfs3_lookup physical 222977 N/A /opt/nfs/htdocs/test/2 ln nfs3_access physical 9553 N/A /opt/nfs/htdocs/test ln nfs3_lookup physical 222109 N/A /opt/nfs/htdocs/test/2 ln nfs3_symlink physical 899939 N/A /opt/nfs/htdocs/test/2 -> /opt/nfs/htdocs/test/1 cat nfs3_access logical 19528 N/A /opt/nfs/htdocs/test cat nfs3_lookup logical 67471 N/A /opt/nfs/htdocs/test/2 cat nfs3_access logical 7722 N/A /opt/nfs/htdocs/test cat nfs3_lookup logical 26941 N/A /opt/nfs/htdocs/test/1 cat nfs3_access physical 278486 N/A /opt/nfs/htdocs/test/1 cat nfs3_getattr logical 17554 N/A /opt/nfs/htdocs/test/1 cat nfs3_read logical 54848 8192 /opt/nfs/htdocs/test/1 cat nfs3_read logical 10082 8192 /opt/nfs/htdocs/test/1 bash nfs3_access logical 19707 N/A /opt/nfs/htdocs/test bash nfs3_lookup logical 66602 N/A /opt/nfs/htdocs/test/. bash nfs3_access logical 7740 N/A /opt/nfs/htdocs/test cp nfs3_write physical 38821 3935 /opt/nfs/htdocs/test/99
The script leverages the fact that each file system implements a set of operations call VOPS (VNODE Operations), which are called by the Solaris kernel in response to a system call that operates on a file or directory in a given file system. In the case of NFSv3, when you issue a read(2) system call, the kernel will determine that the operation applies to a file on an NFSv3 share, and will invoke nfs3_read, which is the NFSv3-specific read routine. For further details on VNODE Operations, VFS and the NFS implementation, I will refer you to Solaris and UNIX internals.
