Bug 258679

Summary: www/chromium: Unable to download files with chromium-92.0.4515.159_2
Product: Ports & Packages Reporter: jimp
Component: Individual Port(s)Assignee: freebsd-chromium (Nobody) <chromium>
Status: New ---    
Severity: Affects Some People CC: bobb, freebsd, grahamperrin, lapo, marko.cupac, michael.hmich, mmatalka, rodrigo, tatsuki_makino
Priority: --- Flags: bugzilla: maintainer-feedback? (chromium)
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258232
Attachments:
Description Flags
Backtrace at rename and sendfile none

Description jimp 2021-09-22 15:30:28 UTC
On chromium-92.0.4515.159_2 I am unable to download files to alternate directories if they are on different filesystems.

I can download to ~/Downloads which is inside my home directory on the same disk, but not to a different directory I have on another ZFS dataset. This worked previously until the most recent major version of Chromium (92.x). Permissions are correct and my user can create files in the target without issue.

It's simple to reproduce by going to freebsd.org and attempting a download of a FreeBSD installer image. If I download to ~/Downloads it succeeds but if I attempt to place it into the directory I export to hypervisors (/data/apps/OS) it fails. The browser displays "Failed - Download Error" but does not log anything on the console or stderr. The browser leaves an empty file ending in ".crdownload" behind when it fails.

If I go to chrome://downloads/ and click "Resume" on the failed entry it may randomly work. Sometimes it works on a single retry sometimes it takes 10 or more tries before it succeeds. Each failed attempt results in a new empty ".crdownload" file

This happens in regular mode and in incognito mode. The problem also persists even if I set the main Download folder to the desired target location.

System is on 13.0 with all available updates applied.

FreeBSD <blah> 13.0-RELEASE-p4 FreeBSD 13.0-RELEASE-p4 #0: Tue Aug 24 07:33:27 UTC 2021     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64

This is not the same issue as bug #258232 as my dbus machine ID is present and consistent.

As a temporary workaround I can download files to my home dir and move them but it's a weird and arbitrary limitation that has not been present in the past, and it gets more irritating each time I have to take the extra steps that were unnecessary before.

There are others reporting similar issues on the -stable mailing list and in the comments on closed bug #25832
Comment 1 bobb 2021-09-30 15:27:07 UTC
I have the same problem "Failed - Download Error." with same chromium-92.0.4515.159_2.

FreeBSD xxx 13.0-RELEASE-p4 FreeBSD 13.0-RELEASE-p4 #0: Tue Aug 24 07:33:27 UTC 2021     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64

If I click "Resume" on the failed entry it randomly work. Each failed attempt results in a new empty ".crdownload" file.
Comment 2 Michael 2021-10-14 23:59:58 UTC
same here, since the upgrade thi days Chromium is not downloading anything no more, download failed is the comment, no further error 

doen't help, but another browser is working downloading the same file, so I mean no problem with my computer or link, this is an isolated Chromium problem

just in case I've recreated the machine-id what resolved the same problem in a former case weeks ago

hm@hm-fbsd:~ % ll /etc/machine-id
lrwxr-xr-x  1 root  wheel  24 Oct 14 19:12 /etc/machine-id@ -> /var/lib/dbus/machine-id




FreeBSD-13-Stable
Chromium Version 92.0.4515.159 (Official Build) (64-bit)
Comment 3 Michael 2021-10-18 22:41:37 UTC
may be this helps:

ERROR:file_util.cc(141)] Moving extension from : /tmp/scoped_dir6inz36/CRX_INSTALL to : /home/hm/.config/chromium/Default/Extensions/Temp/scoped_dirWjuUET/CRX_INSTALL failed.

[3770:108265:1018/192716.110500:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[3770:108264:1018/192717.219121:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[3770:108265:1018/192718.982382:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
last line repeated several times more

the destiny dir does not exist, and can not be created probably because the last existing dir is set to 600 by chromium's startup before it tries to create the dir tree and copy something 

setting it to 655 doesnt matter because it's reset to 600
Comment 4 Matthias Wolf 2021-10-20 10:53:51 UTC
Should be fixed in v94.
Comment 5 jimp 2021-10-21 12:23:26 UTC
No change on chromium-94.0.4606.81 for me. Same symptoms as 92. Can save to my own home directory but not to a location on another ZFS dataset.
Comment 6 Michael 2021-10-21 18:10:05 UTC
unfortunately download doesn´t download, start msgs changed


hm@hm-fbsd:/usr/src % chrome
[1652:101116:1021/150728.041194:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150728.041289:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150728.645635:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150728.862696:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150729.795534:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150732.233837:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150732.595580:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150736.098551:ERROR:chrome_browser_main_extra_parts_metrics.cc(238)] crbug.com/1216328: Checking default browser status started. Please report if there is no report that this ends.
[1652:101111:1021/150736.177362:ERROR:chrome_browser_main_extra_parts_metrics.cc(242)] crbug.com/1216328: Checking default browser status ended.
[1652:101102:1021/150736.325140:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150801.406396:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150803.215000:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150816.536614:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150816.545203:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150816.546307:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150820.384966:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150820.386379:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150821.953678:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101103:1021/150821.955783:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101203:1021/150822.013777:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101203:1021/150822.059647:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101116:1021/150824.144547:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101116:1021/150824.963960:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101116:1021/150824.974360:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101116:1021/150825.812498:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150827.069701:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150827.307218:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150827.848776:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101102:1021/150828.056880:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101203:1021/150829.375680:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101116:1021/150830.889403:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101111:1021/150831.047682:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150831.089406:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
[1652:101111:1021/150833.693483:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101111:1021/150834.010511:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101111:1021/150834.632676:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101203:1021/150834.890751:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101111:1021/150835.408028:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101203:1021/150836.380769:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101111:1021/150838.170543:ERROR:component_installer.cc(143)] Move failed.: Socket operation on non-socket (38)
[1652:101102:1021/150841.347867:ERROR:cert_verify_proc_builtin.cc(600)] No net_fetcher for performing AIA chasing.
Comment 7 Tatsuki Makino 2021-10-22 04:18:36 UTC
There is still a condition in my chromium-94.0.4606.81 that causes the download to fail.

mount -t nullfs /somewhere/with/enough/space ~/Downloads
and start download.

I think it's failing in the directory with the same condition that the combination of rm and cp is used to mv files to the ~/Downloads or download directory.
Comment 8 Tatsuki Makino 2021-10-22 05:15:00 UTC
I tried many things, but I get an error except where the following results match.

stat -f '%8Xd %N' -- ~ download_directory
Comment 9 Michael 2021-10-22 08:49:37 UTC
(In reply to Tatsuki Makino from comment #7)

I think that is a wild guess, you chose the download dir or have it configured and there it goes, as well as the temporary file, as well as the partial cdr file and the complete file, chromium is not moving download files backwards and forwards
Comment 10 Tatsuki Makino 2021-10-22 09:38:38 UTC
(In reply to Michael from comment #9)

Because no one, including me, generates diff of source code :)
Comment 11 Michael 2021-10-22 12:02:44 UTC
(In reply to Tatsuki Makino from comment #10)

the same version runs well on Linux, so it probably is not the source code, but some messup by the packager or missing dependency

I have downgraded to v92... because I need the working download for my work
Comment 12 Michael 2021-11-02 08:53:46 UTC
chromium 94.0.4606.81 is still not downloading, but there is a new issue which I haven't noticed before, it is downloading but only the the home dir (~) any other place not, even if there are the correct permissions set, as in a download dir not placed under the home dir
Comment 13 Marko Cupać 2021-11-09 09:16:10 UTC
(In reply to Michael from comment #12)

Nice find Michael! I always set my download dir to ~/Downloads. Downloads were failing even after latest update to chromium-94.0.4606.81_1. I thought tmpfs /tmp could contribute to the problem but no change when I removed tmpfs /tmp.

Changing download dir to ~/ results in functional downloads.

My subfolders under ~ are separate ZFS datasets.
Comment 14 Tatsuki Makino 2021-11-10 00:55:22 UTC
on FreeBSD 12.3-STABLE stable/12-n234193-89e293e5dcb4-dirty amd64

If ENV HOME has been changed...
/home/tatsuki is my default HOME.

#!/bin/tcsh
setenv HOME `mktemp -d` ; cd ~ ; chrome --incognito &

Download to /tmp/tmp.XXXXXXXX/Downloads was successful.
Download to /home/tatsuki/Downloads failed.

/var/tmp/tmpfs is tmpfs on /var/tmp/tmpfs (tmpfs, local, nosuid).

setenv HOME /var/tmp/tmpfs ; cd ~ ; chrome --incognito &

Download to /var/tmp/tmpfs/Downloads failed.
Download to /home/tatsuki/Downloads failed.

It looks like we have more strange conditions again :)
Comment 15 Tatsuki Makino 2021-11-10 01:14:28 UTC
I have a small request.
Can someone please try download file in combination with gtk3-3.24.29 or earlier?
I would do it too, but it would be later.
Comment 16 Tatsuki Makino 2021-11-10 04:18:28 UTC
(In reply to Tatsuki Makino from comment #15)

I tried to combine chromium-94.0.4606.81_2 and gtk3-3.24.29_1 or chromium-94.0.4606.81_2 and gtk3-3.24.27.
However, this problem was not resolved.
There was a gtk3 update around the same time that chromium started having this problem, but it is not related, maybe :)
Comment 17 gnikl 2021-11-10 16:55:13 UTC
(In reply to Tatsuki Makino from comment #14)
> /home/tatsuki is my default HOME.
The download failure is not directly related to your $HOME directory.

> Download to /tmp/tmp.XXXXXXXX/Downloads was successful.
> Download to /home/tatsuki/Downloads failed.
See bug #258232, comment #24. The successful download is a strong indication that the temporary file location is on the same filesystem as the target download location. Starting with v92 Chromium fails to notice if the download and target location use different filesystems. Now the question is how to find the the broken place in the Chromium sources...
Comment 18 Tatsuki Makino 2021-11-10 22:07:46 UTC
(In reply to gnikl from comment #17)
> Now the question is how to find the the broken place in the Chromium sources...
I have an idea about that. However, I will explain it later :)

Some of the things I have found using the idea are:

env "XDG_DATA_HOME=/tmp" chrome --incognito &

This will make it impossible to download to ~/Downloads. Also, it can probably be downloaded to /tmp, but I haven't tried that.
There are now more keywords to find the relevant code :)
Comment 19 Tatsuki Makino 2021-11-12 02:02:59 UTC
This is too exaggerated to be called an idea :) , but....

I ran chrome with truss logging.
> truss -d -o /tmp/truss.log -- chrome --incognito &

Then observe the truss log.
> less -n -+S +F -- /tmp/truss.log

In a line like the following, find the handle of the IPC? with the X server.
> 4.478558977 connect(22,{ AF_UNIX "/tmp/.X11-unix/X0" },106) = 0 (0x0)

This descriptor 22 produces a large amount of logging just by moving the mouse, but it can all be ignored.
> 25.314379644 recvmsg(22,0x7fffffffda00,0)        ERR#35 'Resource temporarily unavailable'
> 25.314406709 recvmsg(22,0x7fffffffda30,0)        ERR#35 'Resource temporarily unavailable'
> 25.314437758 poll({ 5/POLLIN 21/POLLIN 22/POLLIN },3,0) = 0 (0x0)

You can ignore the parts of it that you don't want, but if you download the featured file, you can easily find where it might be relevant.

> 23.004446278 open("/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",O_RDWR|O_CREAT|O_EXCL,0600) = 152 (0x98)
> 23.004597101 openat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",O_RDWR,00) = 152 (0x98)
> 23.009780133 access("/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64",F_OK) ERR#2 'No such file or directory'
> 23.054732785 access("/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64.crdownload",F_OK) = 0 (0x0)
> 23.055022511 access("/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",F_OK) ERR#2 'No such file or directory'
> 23.055528878 access("/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",F_OK) ERR#2 'No such file or directory'
> 23.055610948 openat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",O_WRONLY|O_CREAT|O_TRUNC,0666) = 151 (0x97)
> 23.055798888 fstatat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",{ mode=-rw-r--r-- ,inode=642050,size=0,blksize=32768 },0x0) = 0 (0x0)
> 23.055874646 fstatat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",{ mode=-rw-r--r-- ,inode=642050,size=0,blksize=32768 },0x0) = 0 (0x0)
> 23.055926311 fstatat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",{ mode=-rw------- ,inode=29856080,size=1811,blksize=32768 },0x0) = 0 (0x0)
> 23.056057619 rename("/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ","/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload") ERR#18 'Cross-device link'
> 23.056209671 access("/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",F_OK) = 0 (0x0)
> 23.056530051 fstatat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",{ mode=-rw-r--r-- ,inode=642050,size=0,blksize=32768 },AT_SYMLINK_NOFOLLOW) = 0 (0x0)
> 23.056931343 fstatat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",{ mode=-rw------- ,inode=29856080,size=1811,blksize=32768 },AT_SYMLINK_NOFOLLOW) = 0 (0x0)
> 23.057138161 fstatat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",{ mode=-rw------- ,inode=29856080,size=1811,blksize=32768 },0x0) = 0 (0x0)
> 23.057200189 fstatat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",{ mode=-rw-r--r-- ,inode=642050,size=0,blksize=32768 },0x0) = 0 (0x0)
> 23.057266794 openat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",O_RDONLY|O_NONBLOCK,00) = 152 (0x98)
> 23.057442543 openat(AT_FDCWD,"/home/tatsuki/Downloads/CHECKSUM.SHA512-FreeBSD-13.0-RELEASE-amd64 (1).crdownload",O_WRONLY|O_NONBLOCK|O_CREAT|O_TRUNC,0600) = 153 (0x99)
> 23.063141718 fstatat(AT_FDCWD,"/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ",{ mode=-rw------- ,inode=29856080,size=1811,blksize=32768 },AT_SYMLINK_NOFOLLOW) = 0 (0x0)
> 23.063323207 unlink("/home/tatsuki/.local/share/.org.chromium.Chromium.0kroOZ") = 0 (0x0)

According to this, there seems to be a code that proceeds as follows: create a temporary file, check for duplicate filenames, add a (1) before the last period because the filenames were duplicated, and start over.

The rename is also failing, but I am also concerned about this sendfile failure.
> 23.057592727 sendfile(0x98,0x99,0x0,0x713,0x0,0x7fffd6de9f48,0x7fff00000000) ERR#38 'Socket operation on non-socket'

There's an even better way. Does anyone have a built chrome with the following settings enabled?
WITH_DEBUG=yes or WITH_DEBUG_PORTS=www/chromium
www_chromium_SET=DEBUG

I still need more time to prepare it on my end, as I did not have enough space to build it on disk :)
Comment 20 Michael 2021-11-12 19:45:14 UTC
seems you try to breed a new egg ...

the source code is the same source code package which is used on Linux where no such problem exist

I think that's enough to say the problem is not in the source

when you have time and knowledge, better you spend some minutes with the freebsd specific patches or Makefile
Comment 21 gnikl 2021-11-12 20:09:36 UTC
(In reply to Tatsuki Makino from comment #19)
> The rename is also failing, but I am also concerned about this sendfile failure.
> 23.057592727 sendfile([...]) ERR#38 'Socket operation on non-socket'
Thank you! That is exactly the problem: sendfile returns an error and the retry logic does not kick in since the errno value is not in the list of the permitted values. Adding ENOTSOCK fixes the download operation for v92. I don't have a ccached v94 yet, thus I can only assume that the attached patch will work for v94. Please try the the patch (save as eg. files/patch-zza) and report back.

TL;DR: The v92 port was the first version that had the sendfile optimization on (Free)BSD. However, the modification was broken for any arch where ssize_t != off_t. This got fixed with the v94 port. I really wonder if activating sendfile() was a sensible decision though. Contrary to Linux the FreeBSD sendfile call limits the out descriptor to a socket. According to the Linux man page the out descriptor can be any file there.

-- cut --
--- base/files/file_util_posix.cc~	2021-11-12 20:13:11.151633000 +0100
+++ base/files/file_util_posix.cc	2021-11-12 20:13:20.289125000 +0100
@@ -1290,8 +1290,8 @@ bool CopyFileContentsWithSendfile(File& 
   // file sizes and file offsets will not have changed. A slow fallback and
   // proceed without issues.
   retry_slow = (copied == 0 && res < 0 &&
-                (errno == EINVAL || errno == ENOSYS || errno == EPERM));
-
+                (errno == EINVAL || errno == ENOSYS || errno == EPERM || errno == ENOTSOCK));
+//fprintf(stderr, "CopyFileContentsWithSendfile: errno=%d\n", errno);
   return res >= 0;
 }
 #endif  // defined(OS_LINUX) || defined(OS_CHROMEOS) || defined(OS_ANDROID) || defined(OS_BSD)
-- cut --
Comment 22 Tatsuki Makino 2021-11-14 00:48:26 UTC
Created attachment 229483 [details]
Backtrace at rename and sendfile

(In reply to Tatsuki Makino from comment #19)
> WITH_DEBUG=yes or WITH_DEBUG_PORTS=www/chromium
> www_chromium_SET=DEBUG

This is what happens when you build www/chromium with these settings:

The 32GB of physical memory is just barely enough.

# du -c -h -d 0 -- `make -C /usr/ports/www/chromium/ -V WRKDIR`
218G    /mnt/tmp/ports/work/usr/ports/www/chromium/work
218G    total

# make -C /usr/ports/www/chromium/ check-orphans
====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: %%DATADIR%%/libimmediate_crash_test_helper.so
Error: Missing: %%DATADIR%%/libmalloc_wrapper.so
Error: Missing: %%DATADIR%%/libtest_shared_library.so
===> Error: Plist issues found.
*** Error code 1
 
Stop.
make: stopped in /usr/ports/www/chromium/

--8<----

check-orphans may have made some mistakes by me.

Thank you for the comment #21 patch.
I haven't used it yet, but for now I'll put here the backtrace I was able to get.
It can be used to determine where to patch and so on :)
Comment 23 Tatsuki Makino 2021-11-15 01:03:05 UTC
(In reply to gnikl from comment #21)

I built and installed chromium-94.0.4606.81_2 with errno == ENOTSOCK added like the patch.
As a result, the download will no longer fail under conditions where the download directory is cross-device with XDG_DATA_HOME directory.
Thank you.
Comment 24 gnikl 2021-11-15 17:52:26 UTC
(In reply to Tatsuki Makino from comment #22)
> This is what happens when you build www/chromium with these settings:
> The 32GB of physical memory is just barely enough.
Is this about the (final) link step? A non-debug build works fine for me with $WRKDIR on a memory-backed tmpfs with an equal amount of RAM.

> # du -c -h -d 0 -- `make -C /usr/ports/www/chromium/ -V WRKDIR`
> 218G    /mnt/tmp/ports/work/usr/ports/www/chromium/work
That is enormous but then not that surpising with C++.

> check-orphans may have made some mistakes by me.
I don't think so. Maybe these files are gone given how fast the chromium sources change...

> I'll put here the backtrace I was able to get.
Those backtraces show the same callstack: first a rename is tried and if that fails a copy is done.

> It can be used to determine where to patch and so on :)
That information is in the patch itself, thats why I suggested to save as files/patch-zza :) 

(In reply to Tatsuki Makino from comment #23)
> I built and installed chromium-94.0.4606.81_2 with errno == ENOTSOCK added
> like the patch.
> As a result, the download will no longer fail under conditions where the
> download directory is cross-device with XDG_DATA_HOME directory.
> Thank you.
You are welcome. Good to hear that the modification fixes the issue for you as well. In the meantime I rebuilt v94 using ccache (now a "fresh" build takes only about 23 minutes ;) and I can confirm that the issue is gone.
Comment 25 Jan Beich freebsd_committer 2021-11-15 22:42:04 UTC
(In reply to gnikl from comment #21)
> Contrary to Linux the FreeBSD sendfile call limits the out descriptor to a socket

Maybe FreeBSD >= 13.0 can optimize via https://man.freebsd.org/copy_file_range/2
like cp(1) did since base c01816a97fd5 (see also later fixes).
Comment 26 Tatsuki Makino 2021-11-15 23:35:51 UTC
(In reply to gnikl from comment #24)
> > The 32GB of physical memory is just barely enough.
> Is this about the (final) link step? A non-debug build works fine for me with $WRKDIR on a memory-backed tmpfs with an equal amount of RAM.

Yes.
ld.lld used about 26GB :)
And then it was killed by kernel :)
The debug version requires 32GB+ of memory and about 220GB of storage space to build.

> I don't think so. Maybe these files are gone

We may be able to delete those files from the pkg-plist.
After someone has confirmed exactly what it is :)

> Those backtraces

The backtraces were brought for those who are exploring the code.
However, you had already brought a patch to help solve this problem :)

> I rebuilt v94 using ccache
I am deleting ${WRKDIR}/.build_done.chromium._usr_local and doing make build :)


(In reply to Jan Beich from comment #25)

What should we do about this problematic file whose name is file_util_"posix".cc? :)
Maybe that needs to be fixed to a larger extent?

#if defined(OS_LINUX) || defined(OS_CHROMEOS) || defined(OS_ANDROID)
bool CopyFileContentsWithSendfile(File& infile,
    original sendfile code...
}
#elif defined(OS_BSD)
bool CopyFileContentsWithSendfile(File& infile,
# if version which copy_file_range exists
    copy_file_range
# else
    retry_slow = value for always fallback
# endif
}
#endif
Comment 27 Tatsuki Makino 2021-11-17 08:25:22 UTC
Other workaround, I'll make the backtrace logs I brought with me useful :)

It is a backtrace that would run CopyDirectory after rename fails.
A patch has been applied to use sendfile for CopyFileContents in frame #2.
So, removing the patch "/usr/ports/www/chromium/files/patch-base_files_file__util.cc" and rebuilding it would be a workaround that would bring it back to the original code.
Comment 28 Rodrigo Osorio freebsd_committer 2021-11-17 09:44:59 UTC
(In reply to gnikl from comment #21)
Applying @gnikl patch also fix the issue for me.
Comment 29 Rodrigo Osorio freebsd_committer 2021-11-17 14:25:05 UTC
(In reply to Rodrigo Osorio from comment #28)

For testing purposes I shared the patched package build for amd64 here: https://people.freebsd.org/~rodrigo/chromium/
Comment 30 gnikl 2021-11-17 17:42:32 UTC
(In reply to Tatsuki Makino from comment #27)
> removing the patch "files/patch-base_files_file__util.cc" and rebuilding it
> would be a workaround that would bring it back to the original code.
Good observation! Removing that file is in a way a simpler fix than adding another patch. Now a port maintainer should comment how to procede.
Comment 31 Michael 2021-11-17 19:46:59 UTC
today its working, let's see tomorrow ;) 

thanks
Comment 32 Tatsuki Makino 2021-11-23 06:12:57 UTC
(In reply to Tatsuki Makino from comment #27)
I'm sorry if anyone thought the method in this comment would solve the problem.
I have built chromium-94.0.4606.81_2 this way.

rm -r -f /usr/ports/www/chromium
portsnap extract www/chromium/
rm -f /usr/ports/www/chromium/files/patch-base_files_file__util.cc
portmaster www/chromium

This chromium will start downloading to the cross-device directory, but will fail when it finishes.
The remaining file, named *.crdownload, is partially missing.

It may be that renaming to the correct name is failing, but I have not investigated it yet.
Comment 33 Tatsuki Makino 2021-11-23 07:22:21 UTC
(In reply to Tatsuki Makino from comment #32)
This comment #32 of mine seems to be a different issue from this one.
And it is difficult to reproduce. (The file size may be a condition for it to occur, but I'm not sure.)

So please don't think about it :)
Comment 34 Michael 2021-11-23 12:26:23 UTC
(In reply to Tatsuki Makino from comment #32)

well, normally you do make clean in the port dir and then portsnap fetch and to compile that thing make and/or make install ...
before compiling you mess with the patches
Comment 35 Tatsuki Makino 2021-11-26 00:05:19 UTC
(In reply to Tatsuki Makino from comment #32)

#21 patched and not debug version was also built.

rm -r -f /usr/ports/www/chromium
portsnap extract www/chromium/
make -C /usr/ports/www/chromium/ clean
make -C /usr/ports/www/chromium/ patch
apply comment #21 patch by my hand
portmaster www/chromium

The problems I reported in my #32-#33 comments also occur in this build.
Perhaps there is a problem with the server where the file I used to download is located or CDN. Or below the 4th layer on my side :)

* Apply the patch by gnikl in comment #21
* Remove the patch file in files/patch-base_files_file__util.cc before building

Both of these are valid workarounds for this problem.