After initial install and at first boot a quick look at the ZFS filesystems
will reveal that atime was erroneously set 'off'. There can not be a valid
reason to disable one of the three UNIX file timestamps 'off'. I would
question the value of xattr=off also however that isn't as serious as atime.
There is infact a good reason to disable atime. ZFS is a CoW (Copy-on-Write) filesystem, so when the atime property is set to on, all snapshots will grow as the atime attribute of each object is modified. This causes what are supposed to be zero-cost snapshots to resume space, and causes read-only workloads to generate amplification as additional metadata must be read for it to be updated to be written out with a modified atime and the checksum of the entire tree of blocks to be updated.
The 'zfs auto' part of the installer sets atime on ZFS to disabled on purpose, for all datasets other than /var/mail.
(In reply to Allan Jude from comment #1)
That seems like a very weak argument to disable one of the three key timestamps in the UNIX world. Disk space is cheap and so is CPU whereas end to end data integrity is essential and precious. I would argue that growing snapshots is exactly correct and they should grow and will grow precisely because that is what snapshots are supposed to do. The disable of "atime" argument may as well be followed by the disable of "ctime" if that were possible.
The zfs manpage could perhaps be updated to reflect that the installer
disables this feature by default. Currently we see :
atime=on | off
Controls whether the access time for files is updated when they are
read. Turning this property off avoids producing write traffic when
reading files and can result in significant performance gains, though
it might confuse mailers and other similar utilities. The default
value is on.
The default in 11.2 and 12.0beta2 and beta3 is to disable the UNIX atime
feature silently and thus the user will never know. The manpage should
reflect this information as the "default value is on" will lead anyone to
think there is a failure in their system after initial install.