Using sharemgr to manage NFS file systems on OpenSolaris hosts

I’ve supported Solaris NFS servers for as long as I can remember. The typical drill I follow to get a server up and running is:

1. Work with a developer / app support to determine how much space is needed.

2. Provision space on the server.

3. Enabled the NFS server processes.

4. Edit the /etc/dfs/sharetab file to share out the file system I provisioned.

5. Update clients with the new mount information.

With recent opensolaris builds, step 4 has changed slightly. The new way to add NFS shares is through the sharemgr utility, which allows you to create share groups (i.e., these are groupings of similar network shares) and shares, add and remove properties to these shares, and enable and disable shares. To create a new share group, the sharemgr utility can be run with the “create” option, the protocol (NFS or SMB) to associate with this share, and the name of the share:

$ sharemgr create -P nfs build

Once a share group is created, you can use the “list” subcommand to view the group’s properties:

$ sharemgr list -v
default enabled nfs
zfs enabled
build enabled nfs

To add one or more shares to a group, you can run the sharemgr utility with the “add-share” subcommand, the directory to share out, and an optional description to associate with the share:

$ sharemgr add-share -s /bits/provisioning -d “Build infrastructure” build

To view shares, you can run sharemgr with the “show” option:

$ sharemgr show -v

          /bits/provisioning    "Build infrastructure"

Now assuming you don’t want the world to be able to access the share, you will most likely want to configure some type of security to limit who can access it. To limit the list of clients that can access shares in the build group to the network, you can run sharemgr with the “set” option, the protocol, the properties to set, and the share group to attach these properties to:

$ sharemgr set -P nfs -p anon=0,ro=@ build

To view the properties, the sharemgr utility can once again be run with the “show” subcommand:

$ sharemgr show -vp

default nfs=()
build nfs=(anon="0,ro=@")
          /bits/provisioning    "Build infrastructure"

While a bit more complex than using echo to send a string to /ec/dfs/dfstab, the real utility comes into play when you are configuring SMB shares. Once I get around to testing out the opensolaris CIFS server, I will add a follow up post detailing how to use sharemgr to share out CIFS shares.

Stopping nfsmapid from querying DNS TXT records

With the introduction of NFSv4, user and group identifiers were changed to use the username@domain format. On Solaris hosts, the domain is determined using the following methods:

1. The NFSMAPID_DOMAIN variable is checked in /etc/default/nfs

2. DNS is queried for the _nfsv4idmapdomain TXT record

3. The configured DNS domain is used

4. The file /etc/defaultdomain is consulted

If a site doesn’t update the NFSMAPID_DOMAIN variable when deploying NFSv4, DNS will be queried for the domain to use. If the DNS server doesn’t contain a _nfsv4idmapdomain TXT record, you will see failed queries similar to the following:

host1 -> host2 ETHER Type=0800 (IP), size = 77 bytes
host1 -> host2 IP D= S= LEN=63, ID=19779, TOS=0x0, TTL=255
host1 -> host2 UDP D=53 S=52032 LEN=43
host1 -> host2 DNS C _nfsv4idmapdomain. Internet TXT ?
host2 -> host1 ETHER Type=0800 (IP), size = 77 bytes
host2 -> host1 IP D= S= LEN=63, ID=26996, TOS=0x0, TTL=254
host2 -> host1 UDP D=52032 S=53 LEN=43
host2 -> host1 DNS R Error: 3(Name Error)

This can of course pose a problem for large sites, since the DNS server will be inundated with queries for records that don’t exist. If you want to stop these DNS queries from happening, you can add the domain to the NFSMAPID_DOMAIN variable in /etc/default/nfs. Shibby!

NFSv3 guarded writes

While debugging another NFSv3 problem this week, I came across a create procedure with the “GUARDED” flag set:

$ snoop -d hme0 host netapp1

yappy -> netapp1      NFS C CREATE3 FH=C5D2 (GUARDED) file.dat
     netapp1 -> yappy NFS R CREATE3 OK FH=E700

This was the first time I have reviewed an NFSv3 packet capture with the GUARDED flag set, so I decided to read RFC 1813 to see how the create procedure should be implemented. Here is what the RFC says about the GUARDED flag:

“Creation modes:
One of UNCHECKED, GUARDED, and EXCLUSIVE. UNCHECKED means that the file should be created without checking for the existence of a duplicate file in the same directory. In this case, how.obj_attributes is a sattr3 describing the initial attributes for the file. GUARDED specifies that the server should check for the presence of a duplicate file before performing the create and should fail the request with NFS3ERR_EXIST if a duplicate file exists. If the file does not exist, the request is performed as described for UNCHECKED. EXCLUSIVE specifies that the server is to follow exclusive creation semantics, using the verifier to ensure exclusive creation of the target. No attributes may be provided in this case, since the server may use the target file metadata to store the createverf3 verifier.”

This is rather interesting, and it makes sense given that the mode passed as the second argument to creat() should be transparently passed through to the NFSv3 create procedure. RFC 1813 is extremely well writtten, and a great read if your looking to learn more about NFS!

Asynchronous writes in NFSv3

While debugging an interesting NFSv3 problem last week, I came across the following information in one of my snoop captures:

$ snoop -d hme0 host netapp1
yappy -> netapp1 NFS C WRITE3 FH=94EC at 0 for 824 (ASYNC)
netapp1 -> yappy NFS R WRITE3 OK 824 (FSYNC)
yappy -> nfsfly1 NFS C WRITE3 FH=30D9 at 0 for 285 (ASYNC)
netapp1 -> yappy NFS R WRITE3 OK 285 (FSYNC)
yappy -> nfsfly1 NFS C WRITE3 FH=36C6 at 0 for 231 (ASYNC)
netapp1 -> yappy NFS R WRITE3 OK 231 (FSYNC)

Prior to reviewing the snoop capture, I didn’t know that NFSv3 supported asynchronous I/O. While reading through various pieces of information, I came across the following nugget of information:

“NFS Version 3 introduces the concept of “safe asynchronous writes.” A Version 3 client can specify that the server is allowed to reply before it has saved the requested data to disk, permitting the server to gather small NFS write operations into a single efficient disk write operation. A Version 3 client can also specify that the data must be written to disk before the server replies, just like a Version 2 write. The client specifies the type of write by setting the stable_how field in the arguments of each write operation to UNSTABLE to request a safe asynchronous write, and FILE_SYNC for an NFS Version 2 style write.”

This explained why we were seeeing what we were seeing (more on that later), and is one of those things to keep in the back of your mind. One interesting tidbit to note. The Solaris snoop command displays the string “ASYNC” for an asynchronous write procedure, but RFC 1813 uses the term “UNSTABLE” to reference asynchronous I/O procedures. It’s a small nit, but the term mismatch can lead to some confusion if your using the RFCs verbatim to interpret the output of snoop “-vv”.