Created attachment 174200 [details] Program used to reproduce the issue It seems that ZFS does not honor fsync() done on zvols. This issue affects iSCSI targets backed by ZFS volumes, which are forced to run with sync=always to ensure consistency. Running sync=always has a drastic performance impact. To reproduce: - Log in via remote SSH console (important). - Create a ZFS pool and a zvol. - Format zvol with UFS. - Mount zvol UFS partition. - Cd into mount point. - Compile and run the attached program. - Physically remove power to the computer. - Write down the last number shown in the SSH console window. - Reboot computer and log in. - Mount zvol partition if needed. - Cd into mount point. - Display contents of "output.txt" - Compare last line of "output.txt" with last line shown in SSH console. Expected output: If the SSH console's last number was 11291, the last number in "output.txt" should be 11291 or 11292. Actual output: The last line in "output.txt" is a lot further behind the output in the SSH console. Here's the commands I used to set up the UFS partition: History for 'tank0': 2016-08-29.19:39:35 zpool create tank0 /dev/ada0s2 2016-08-29.19:43:57 zpool import tank0 2016-08-29.19:44:50 zfs create -V 500M tank0/testvol gpart add -t freebsd-ufs -a 1M /dev/zvol/tank0/testvol newfs -U /dev/zvol/tank0/testvolp1 mkdir test mount /dev/zvol/tank0/testvolp1 test The supplied program shows the expected output if run directly on a "native" UFS partition, as well as when run on a zvol using ZFS-on-Linux (on a Debian machine).
Created attachment 174201 [details] Screenshot showing the discrepancy due to lack of fsync()
(In reply to Asbjørn from comment #0) I meant to write "istgt iSCSI targets backed by ZFS volumes".
When sync=always, the results are as expected, the issue is when sync=standard, which, according to the documentation should respect fsync(). As mentioned, the issue is present with iSTGT using the zvol, so this is not a UFS-on-zvol issue as far as I can see.
Reproduced it on FreeBSD 11.0.
do I understand from IRC correctly, that you do NOT have this problem when using ctld?
I have _not_ yet tested ctld. The confirmation I mentioned above for 11.0 is the supplied test program using UFS on a zvol as described in the initial entry of the bug report. I have not tested istgt on 11.0 yet, but given that the simple UFS program doesn't work I didn't see the point of testing that yet. I will test ctld next though, _if_ it works it may help to narrow down cause of the reported issue.
I tested ctld on FreeBSD 11.0, and it would seem it is not affected by this issue. So I can try to use ctld as a workaround instead of istgt (I understand performance should be better as well). Though the issue as reported in this bug is still unresolved and IMO worthy of being properly resolved.
fsync on zvols is ignored (probably because zvols are not real files). ioctl(fd, DIOCGFLUSH) should be used instead.