Adventures in ZFS: Solaris 11.1 Share Properties

In Solaris 11.1 Oracle decided to change the way that they stored filesystem share information. I personally loved the change, but that was without understanding what the change really was. For more information on how to enable NFS exports see my previous article here.

Setup Our Environment

Everything we are doing will be on some test file systems, with no data. So lets put everything in place.

bash
zfs create tank/fs1
zfs set share=name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15 tank/fs1
name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15

Now lets create our child filesystem, and export it the same way.

bash
zfs create tank/fs1/sub1
zfs set share=name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15 tank/fs1/sub1
name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15

Now this will simulate a situation that I was in. I had originally exported everything at a higher level (fs1) and then as the dataset grew it became necessary to export it at the child level (sub1) in order to have more granular control. So now we have two exports, and only need sub1.

Disable Unused NFS Export

Since fs1 is no longer needed lets disable its export.

bash
zfs set share.nfs=off tank/fs1

Lets take a look at fs1 to see if our configuration is still active.

bash

zfs get share tank/fs1

OK it is empty. But since I don’t need this any more I want to make sure that there are no remaining artifacts. So lets re-enable it and confirm that the configuration is gone.

bash
zfs set share.nfs=on tank/fs1
zfs get share tank/fs1
NAME      PROPERTY  VALUE  SOURCE
tank/fs1  share     name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15  local

OK, so our configuration is still there, but it is disabled by share.nfs=off. It must be kept elsewhere in the file system.

FYI, this is included in the ZFS metadata, so if you were to export you zpool and take it to another system, it would come over to the other system.

Explore ZFS Properties

Lets disable fs1 and start poking around.

bash
zfs set share.nfs=off tank/fs1

First thing to do is a get all. This will list all of the properties of the filesystem. I trimmed it back to only display the one that I found interesting.

bash

zfs get all tank/fs1
tank/fs1  share.*               ...                    local

OK so lets look a little bit closer at this share.* property.

bash
zfs get share.* tank/fs1
bad property list: invalid property specifier 'share.*', use 'share.all'
For more info, run: zfs help get

So share.* is not a valid property, it means that there are sub-properties, apparently share.all will display the sub-properties.

bash
zfs get share.all tank/fs1
NAME      PROPERTY         VALUE      SOURCE
tank/fs1  share.auto       off        local
tank/fs1  share.autoname              default
tank/fs1  share.desc                  default
tank/fs1  share.nfs        off        local
tank/fs1  share.nfs.*      ...        default
tank/fs1  share.point      /tank/fs1  default
tank/fs1  share.protocols             -
tank/fs1  share.smb        off        default
tank/fs1  share.smb.*      ...        default

Oh boy, now we are getting somewhere, here is our share.nfs, and we can also see that share.nfs has sub-properties as well. Also notice that smb is also stored here in the same fashion.

bash
zfs get share.nfs.all tank/fs1
NAME      PROPERTY             VALUE  SOURCE
tank/fs1  share.nfs.aclok      off    default
tank/fs1  share.nfs.anon              default
tank/fs1  share.nfs.charset.*  ...    default
tank/fs1  share.nfs.cksum             default
tank/fs1  share.nfs.index             default
tank/fs1  share.nfs.log               default
tank/fs1  share.nfs.noaclfab   off    default
tank/fs1  share.nfs.nosub      off    default
tank/fs1  share.nfs.nosuid     off    default
tank/fs1  share.nfs.sec               default
tank/fs1  share.nfs.sec.*      ...    default

Under share.nfs.all we have a lot of NFS parameters, the most important at this point is share.nfs.sec.all, so lets look there.

bash
zfs get share.nfs.sec.all tank/fs1
NAME      PROPERTY                 VALUE  SOURCE
tank/fs1  share.nfs.sec.default.*  ...    default
tank/fs1  share.nfs.sec.dh.*       ...    default
tank/fs1  share.nfs.sec.krb5.*     ...    default
tank/fs1  share.nfs.sec.krb5i.*    ...    default
tank/fs1  share.nfs.sec.krb5p.*    ...    default
tank/fs1  share.nfs.sec.none.*     ...    default
tank/fs1  share.nfs.sec.sys.*      ...    default

We are getting warmer. Now we have the share.nfs.sec.sys.all to take a look at.

bash
zfs get share.nfs.sec.sys.all tank/fs1
NAME      PROPERTY                        VALUE  SOURCE
tank/fs1  share.nfs.sec.sys.none                 default
tank/fs1  share.nfs.sec.sys.ro                   default
tank/fs1  share.nfs.sec.sys.root                 default
tank/fs1  share.nfs.sec.sys.root_mapping         default
tank/fs1  share.nfs.sec.sys.rw                   default

OK well we have run out of levels to go deeper, but I still don’t see anything. Perhaps if we call the property by name?

bash
zfs get share.nfs.sec.sys.rw tank/fs1
NAME      PROPERTY              VALUE  SOURCE
tank/fs1  share.nfs.sec.sys.rw         default

So that is empty as well. Perhaps that has something to do with the export being disabled? Lets confirm our export is disabled.

bash
zfs get share.nfs tank/fs1
NAME      PROPERTY   VALUE  SOURCE
tank/fs1  share.nfs  off    local

Doesn’t get any more disabled than that. Well lets check the sub1 file system.

bash
zfs get share.nfs tank/fs1/sub1
NAME           PROPERTY   VALUE  SOURCE
tank/fs1/sub1  share.nfs  on     local

So we can see that it is enabled, lets check the share property to see what our whole string is.

bash
zfs get share tank/fs1/sub1
NAME           PROPERTY  VALUE  SOURCE
tank/fs1/sub1  share     name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,rw=@10.0.0.15  local

Based on that string we should have share.nfs.sec.sys.rw = 10.0.0.15.

bash
zfs get share.nfs.sec.sys.rw tank/fs1/sub1
NAME           PROPERTY              VALUE  SOURCE
tank/fs1/sub1  share.nfs.sec.sys.rw         default

OK well that was unexpected. Property is empty on the enabled export file system as well. We must be missing something.

Back to the Drawing Board

Out of pure desperation and a fair bit of luck. I was attempting to lookup the help command on zfs get, and accidentally typed list instead. But look at what I found.

bash
zfs help list
usage:
list [-rH][-d max] [-o property[,...]] [-t type[,...]] [-s property] ...
[-S property] ... [filesystem|volume|snapshot|share] ...

There is a share file system type. Certainly that must have something to do with everything I am seeing?

bash
zfs list -t share
NAME                                                                   USED  AVAIL  REFER  MOUNTPOINT
tank/fs1%tank_fs1                                                         -      -      -  /tank/fs1
tank/fs1/sub1%tank_fs1_sub1                                               -      -      -  /tank/fs1/sub1

Clearly we are onto something here. We have a share file system type for both our enabled and our disabled NFS export. Lets see what kind of properties these new file system types have.

ZFS File System Share Type

bash
zfs get all tank/fs1%tank_fs1
NAME               PROPERTY    VALUE                  SOURCE
tank/fs1%tank_fs1  creation    Sat Sep 21 10:09 2013  -
tank/fs1%tank_fs1  mountpoint  /tank/fs1              -
tank/fs1%tank_fs1  share.*     ...                    local
tank/fs1%tank_fs1  zoned       off                    default

Well there is a share.* so that must be what we are looking for. Lets try our previous command against this new file system and take a look at what we get.

bash
zfs get share.nfs.sec.sys.all tank/fs1%tank_fs1
NAME               PROPERTY                        VALUE       SOURCE
tank/fs1%tank_fs1  share.nfs.sec.sys.none                      default
tank/fs1%tank_fs1  share.nfs.sec.sys.ro                        default
tank/fs1%tank_fs1  share.nfs.sec.sys.root                      default
tank/fs1%tank_fs1  share.nfs.sec.sys.root_mapping              default
tank/fs1%tank_fs1  share.nfs.sec.sys.rw            @10.0.0.15  local

Well there is that elusive read write statement.

Updating NFS Security Statements

Now lets see what happens when we change it.

bash
zfs set share.nfs.sec.sys.rw=@10.0.0.15:@10.0.0.16 tank/fs1%tank_fs1

Here we have added a second IP address to be allowed to read and write, on our disabled NFS export.

bash
zfs get share.nfs.sec.sys.all tank/fs1%tank_fs1
NAME               PROPERTY                        VALUE                  SOURCE
tank/fs1%tank_fs1  share.nfs.sec.sys.none                                 default
tank/fs1%tank_fs1  share.nfs.sec.sys.ro                                   default
tank/fs1%tank_fs1  share.nfs.sec.sys.root                                 default
tank/fs1%tank_fs1  share.nfs.sec.sys.root_mapping                         default
tank/fs1%tank_fs1  share.nfs.sec.sys.rw            @10.0.0.15:@10.0.0.16  local

The property is successfully updated on the share file system type. Lets see if it also enabled the NFS export.

bash

zfs get share tank/fs1

Still disabled, that is perfect. Now lets see what happens on an enabled NFS export.

bash
zfs set share.nfs.sec.sys.ro=@10.0.0.15:@10.0.0.16 tank/fs1/sub1%tank_fs1_sub1

Now we have added 2 new IPs to have read only access to this export. Lets check its effect.

bash
zfs get share.nfs.sec.sys.all tank/fs1/sub1%tank_fs1_sub1
NAME                         PROPERTY                        VALUE                  SOURCE
tank/fs1/sub1%tank_fs1_sub1  share.nfs.sec.sys.none                                 default
tank/fs1/sub1%tank_fs1_sub1  share.nfs.sec.sys.ro            @10.0.0.15:@10.0.0.16  local
tank/fs1/sub1%tank_fs1_sub1  share.nfs.sec.sys.root                                 default
tank/fs1/sub1%tank_fs1_sub1  share.nfs.sec.sys.root_mapping                         default
tank/fs1/sub1%tank_fs1_sub1  share.nfs.sec.sys.rw            @10.0.0.15             local

The properties were successfully updated. Lets check the file system to see if the share properties were brought over.

bash
zfs get share tank/fs1/sub1
NAME           PROPERTY  VALUE  SOURCE
tank/fs1/sub1  share     name=tank_fs1_sub1,path=/tank/fs1/sub1,prot=nfs,sec=sys,ro=@10.0.0.15:@10.0.0.16,rw=@10.0.0.15  local

That looks perfect, and lets ask NFS what it knows.

bash
showmount -e
export list for server1:
/tank/fs1/sub1                      @10.0.0.15,@10.0.0.16,@10.0.0.15

Wonderful! That is exactly what we wanted. Now one final thing.

Removing NFS Export Configuration Permanently

Now since there is this whole different type of file system to store the share configuration, removing it becomes incredibly easy.

bash

zfs destroy tank/fs1%tank_fs1

That pretty much wraps up everything.