I live in time zone GMT + 1. While trying something for another issue (#217440), I created a file on a FAT32 formatted USB stick using my Sony PlayStation 4. The PlayStation 4 is showing the correct local time and I have verified that it has its time zone configured correctly as "UTC+01:00 Oslo". My laptop is showing the correct local time and I have the TZ env variable set correctly as "Europe/Oslo". I took a screenshot in the game Alien: Isolation on the PS4 and transferred it to my USB stick. I mounted the USB stick with # mount_msdosfs -L en_US.UTF-8 /dev/da1s1 /mnt/ as per the discussion in issue #217440 but when I list the directory contents, the time stamp of the files is off by one hour into the future compared to actual time when the file was created as shown below. Note also that the file name itself contains the correct time when the file was created. I transferred the file to my memory stick within minutes of having taken the screenshot. I don't have a windows system to create files from at the moment so I can't verify whether this is a problem with FreeBSD handling of FAT32 or if the incorrect timestamp was caused by the PS4. I have attached an image of a minimal FAT32 partition which contains the directory structure and the screenshot which was written to the USB stick by the PS4. The USB stick was prepared with # umount /dev/da1s1 $ gpart delete -i 1 /dev/da1 $ gpart destroy /dev/da1 # dd if=/dev/zero of=/dev/da1 bs=16m $ gpart create -s mbr /dev/da1 $ gpart add -t fat32 -s 4g -i 1 /dev/da1 # newfs_msdos -F32 /dev/da1s1 # sync $ gpart show da1 => 63 30277569 da1 MBR (14G) 63 8161 - free - (4.0M) 8224 8388608 1 fat32 (4.0G) 8396832 21880800 - free - (10G) Unplugged the USB stick, put it in the PS4, took a screenshot and transferred it to the USB stick. Put it back in the computer and made a minimal image of the FAT32 partition; # umount /dev/da1s1 $ dd if=/dev/da1s1 of=usbstick.img conv=sparse bs=16m count=1 $ hd usbstick.img > tmp.img $ xxd -r tmp.img > usbstick.img $ rm tmp.img $ ls -hal usbstick.img -rw-r--r-- 1 erikn erikn 1.5M Mar 5 12:10 usbstick.img To mount the image; # mdconfig -a -t vnode -f usbstick.img -u 0 # mount_msdosfs -L en_US.UTF-8 /dev/md0 /mnt/ Here we have the file with the mismatching timestamp. $ ls -al /mnt/PS4/SHARE/Screenshots/Alien_\ Isolation™/ total 768 drwxrwxrwx 1 root wheel 32768 Mar 5 13:07 ./ drwxrwxrwx 1 root wheel 32768 Mar 5 13:07 ../ -rwxrwxrwx 1 root wheel 305981 Mar 5 13:07 Alien_ Isolation™_20170305120653.jpg* Unmount image; # umount /dev/md0 # mdconfig -d -u 0
Image was about half of a megabyte too big to be attached. You can download a copy of the ~1.5MiB file from https://www.nordstroem.no/blob/7e/bc/32e9d7c-1534776.img
If I'm not mistaken, the FAT file system stores timestamps in local time. When that FAT file system is mounted in a Unix-like system (e.g., FreeBSD), the timestamp is interpreted as if it is UTC; that system cannot know what the timezone was on the system that wrote the FAT file system. So, since you are +1:00 from UTC, you see the timestamp one hour in the future.
(In reply to David Bright from comment #2) The discussion above presumes that your PlayStation 4 actually records the time as an MS-DOS/Windows system does. I don't know if that is actually the case.
There is definitely some timestamp bug here. In my UTC+2 timezone: $ date Sun Apr 29 03:23:58 SAST 2018 $ touch bsd-201804290323.txt $ stat -x bsd-201804290323.txt File: "bsd-201804290323.txt" Size: 0 FileType: Regular File Mode: (0755/-rwxr-xr-x) Uid: ( 1002/ user) Gid: ( 0/ wheel) Device: 0,139 Inode: 59620 Links: 1 Access: Sun Apr 29 02:00:00 2018 Modify: Sun Apr 29 05:23:58 2018 Change: Sun Apr 29 05:23:58 2018 Windows shows the correct modified/changed times of 03:23. Why is FreeBSD adding my time zone offset (2 hours) to the times when stat reads them, even though they are already in local time?
Created attachment 192889 [details] Use vfs_timestamp() instead of getnanotime() in msdosfs I've noticed msdosfs uses getnanotime() to generate timestamps, unlike ufs, ext2fs, devfs, nandfs, nfsclient, nfsserver, and tmpfs, which all use the better vfs_timestamp() function instead (which allows filesystem timestamp granularity to be configured using the vfs.timestamp_precision sysctl). Only autofs and msdosfs use getnanotime(). Patching msdosfs to use vfs_timestamp() instead, seems to also fix this bug here, with msdosfs timestamps having the timezone offset added to them a second time when read. Please find my patch attached.
(In reply to Damjan Jovanovic from comment #5) Change to use vfs_timestamp() should be fine by itself, but I cannot see how could it solve 'off by a hour' issue.
(In reply to Konstantin Belousov from comment #6) By default vfs.timestamp_precision is TSP_USEC, which returns microtime() instead of getnanotime(). The timezone for microtime() seems to be more correct than for getnanotime().
(In reply to Damjan Jovanovic from comment #7) This does not make sense, there is no 'timezone', and esp. there is no 'more correct' timezone. The difference between e.g. getmicrotime() and microtime() is that get return current clock hands without quering hardware to get the most precise possible value for the timecounter. So the typical precision of of getXXX() is 1/hz, while precision of XXX() is around the precision of hardware. But both variants return UTC time as known by kernel.
(In reply to Konstantin Belousov from comment #8) Thank you, I'll try that test again with a clean kernel rebuild. MICROTIME(9) does not document the timezone for the time functions.
(In reply to Damjan Jovanovic from comment #9) Something must have been wrong with my setup. With a clean kernel rebuild, I can no longer reproduce this bug, even without my patch. Sorry. That patch may still be worth committing regardless ;).
(In reply to Damjan Jovanovic from comment #10) Did you tweaked machdep.adjkerntz or machdep.wall_cmos_clock between tests ?
(In reply to Konstantin Belousov from comment #11) During my early tests, I did change CMOS clock to local time in the BIOS, and run tzsetup selecting that the CMOS clock is in local time, but I still got the same bad times after that. Most probably, between experimenting with several patches to the msdosfs code that passed utc=1 to fattime2timespec() and/or timespec2fattime(), and sometimes forgetting to run "make installkernel" after "make buildkernel", I ended up with a badly patched msdosfs binary, that kept giving bad times until I did a clean rebuild.
Commits base r333311 (-current) and base r334742 (MFC to stable/11) applied the suggested patch (or similar). Can this bug now be closed? Added pfg@ (who committed the change) to CC List to get his input on this question, too.
(In reply to David Bright from comment #13) The patch was committed but it is unrelated the bug, and that's why the PR was not mentioned in the commit. I think the issue might have been related to an update of the timezone database though. Can one of the affected users confirm the bug is gone?
Comment on attachment 192889 [details] Use vfs_timestamp() instead of getnanotime() in msdosfs Obsolete the patch as it was committed in r333311 and MFCd to 11/stable.
^Triage: apparently committed to all supported branches.
CLose the issue since there was no reply on the issue reappearing.