While debugging an interesting NFSv3 problem last week, I came across the following information in one of my snoop captures:
snoop -d hme0 host netapp1
yappy -> netapp1 NFS C WRITE3 FH=94EC at 0 for 824 (ASYNC) netapp1 -> yappy NFS R WRITE3 OK 824 (FSYNC) yappy -> nfsfly1 NFS C WRITE3 FH=30D9 at 0 for 285 (ASYNC) netapp1 -> yappy NFS R WRITE3 OK 285 (FSYNC) yappy -> nfsfly1 NFS C WRITE3 FH=36C6 at 0 for 231 (ASYNC) netapp1 -> yappy NFS R WRITE3 OK 231 (FSYNC)
Prior to reviewing the snoop capture, I didn’t know that NFSv3 supported asynchronous I/O. While reading through various pieces of information, I came across the following nugget of information:
“NFS Version 3 introduces the concept of “safe asynchronous writes.” A Version 3 client can specify that the server is allowed to reply before it has saved the requested data to disk, permitting the server to gather small NFS write operations into a single efficient disk write operation. A Version 3 client can also specify that the data must be written to disk before the server replies, just like a Version 2 write. The client specifies the type of write by setting the stable_how field in the arguments of each write operation to UNSTABLE to request a safe asynchronous write, and FILE_SYNC for an NFS Version 2 style write.”
This explained why we were seeeing what we were seeing (more on that later), and is one of those things to keep in the back of your mind. One interesting tidbit to note. The Solaris snoop command displays the string “ASYNC” for an asynchronous write procedure, but RFC 1813 uses the term “UNSTABLE” to reference asynchronous I/O procedures. It’s a small nit, but the term mismatch can lead to some confusion if your using the RFCs verbatim to interpret the output of snoop “-vv”.