Bug 237369

Summary: ports-mgmt/pkg: pkg delete removes required NLS directories
Product: Ports & Packages Reporter: Stefan Eßer <se>
Component: Individual Port(s)Assignee: freebsd-pkg (Nobody) <pkg>
Status: Open ---    
Severity: Affects Some People CC: pkg
Priority: --- Flags: koobs: maintainer-feedback? (pkg)
koobs: merge-quarterly?
Version: Latest   
Hardware: Any   
OS: Any   

Description Stefan Eßer freebsd_committer 2019-04-18 21:04:24 UTC
A port I maintain (math/gh-bc) installs POSIX message catalog files below ${PREFIX}/share/nls/ .

While the installation succeeds first time, it fails after the package has been de-installed, since the directory ${PREFIX}/share/nls/C has been removed.

That directory is not directly used in the package, but it is the symlink target of ${PREFIX}/share/nls/en_US.US-ASCII where a message catalog is installed by the port/package and removed by pkg delete.

To reproduce:

Make sure there is no message catalog file in ${PREFIX}/share/nls/C , then:

pkg install gh-bc
pkg delete gh-bc

---> the empty directory ${PREFIX}/share/nls/C has been deleted

pkg install gh-bc

---> the installation fails due to a file that is to be installed in ${PREFIX}/share/nls/en_US.US-ASCII/ which points to the no longer existing ${PREFIX}/share/nls/C (which is not automatically created).

IMHO, pkg delete should not delete ${PREFIX}/share/nls/C (which is not even contained in pkg-plist of the port) when this directory has become empty (after the last message catalog has been deleted).

As is, the de-installation of the gh-bc port will cause surprising breakage on users' systems (and the cause is not obvious from the error messages of the failed installation attempt, see last line of the following log):

# ls -l /usr/local/share/nls/C /usr/local/share/nls/en_US.US-ASCII
lrwxr-xr-x  1 root  wheel  1 May 25  2011 /usr/local/share/nls/en_US.US-ASCII -> C

total 0

# pkg add /usr/packages/All/gh-bc-1.2.4.txz 
Installing gh-bc-1.2.4...
Extracting gh-bc-1.2.4: 100%

# pkg delete gh-bc
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:

Number of packages to be removed: 1

Proceed with deinstalling packages? [y/N]: y
[1/1] Deinstalling gh-bc-1.2.4...
[1/1] Deleting files for gh-bc-1.2.4: 100%

# ls -l /usr/local/share/nls/C /usr/local/share/nls/en_US.US-ASCII
ls: /usr/local/share/nls/C: No such file or directory
lrwxr-xr-x  1 root  wheel  1 May 25  2011 /usr/local/share/nls/en_US.US-ASCII -> C

# pkg add /usr/packages/All/gh-bc-1.2.4.txz 
Installing gh-bc-1.2.4...
Extracting gh-bc-1.2.4:  69%
pkg: Fail to create hardlink: /usr/local/share/nls/en_US.US-ASCII/.bc.cat.cbNr0pGVY6yN:No such file or directory
Extracting gh-bc-1.2.4: 100%

Failed to install the following 1 package(s): /usr/packages/All/gh-bc-1.2.4.txz
Comment 1 Baptiste Daroussin freebsd_committer 2019-04-19 11:38:36 UTC
before starting why is /usr/local/share/nls/en_US.US-ASCII a symlink? In the ports tree I cannot see anything making it a symlink.

When I install the port:
# file /usr/local/share/nls/en_US.US-ASCII
/usr/local/share/nls/en_US.US-ASCII: directory

That said when I look at the port itself and its plist. there is nothing under  /usr/local/share/nls/C at all so pkg will not remove it. (btw it does not even install it from the test I am doing.

am I missing something?
Comment 2 Stefan Eßer freebsd_committer 2019-04-20 11:32:47 UTC
Sorry for not making the upgraded port available - after writing the PR I was afraid it could cause problems for users that caused them some effort to understand and fix ...

I have just committed the version with NLS files (1.2.8) and have further analyzed the problem. My attempt to fix the issue by adding a @postunexec command failed, since these commands are executed before the directory is removed. (I tried "@postunexec /bin/mkdir -p %%LOCALBASE%%/share/nls/C").

I can work around this issue by adding a preexec command that creates the missing directory, but this will not help other ports that install files into share/nls.

The actual problem is that the port uses message catalog files that are meant to be portable to quite a number of platforms. These are named according to typical locale names, but the install script takes care to only install to existing nls directories (does not create directories with names not common for the platform like "en_US.cat" or "en_US.utf8" in the case of FreeBSD).

The problem is in the automatic cleanup of empty directories performed by pkg delete if the directory considered is actually a symlink. 

It would suffice to have pkg keyword like "@precious" or "@keep" to declare some files (or directories, or sub-trees) as not to be cleaned up on uninstall, but I think this is a general problem that should be solved in "pkg".

The cause of the problem is that bc.cat gets installed to "share/nls/en_US.US-ASCII", which actually is a symlink to "share/nls/C" (the actual directory).

On pkg deletion, the pkg command sees that "share/nls/en_US.US-ASCII" has become empty and tries to delete that directory - but it in fact deletes the symlink target "share/nls/C".

If "pkg" was careful to create directories required to install files to before trying to access them, "share/nls/C" could be re-created (in this case by "mkdir share/nls/en_US.US-ASCII/") then the problem could be worked around.

But IMHO, "pkg" should not delete empty directories if the last path component is a symlink. I.e. do not perform the equivalent of "rmdir share/nls/en_US-ASCII/" but rather "rmdir share/nls/en_US-ASCII".

Deleting the symlink indstead of its target would obviously also be wrong: That would lead to creation of a directory instead of re-creation of the symlink, when a file is to be installed within ...

IMHO there is a mis-match between "rmdir share/nls/en_US.US-ASCII/" and "mkdir -p share/nls/en_US.US-ASCII" (but the problem is in the rmdir, not the mkdir).
And there are two solutions (besides adding a keyword like @keep):

- Perform both rmdir and mkdir on the target of the symlink.

- Perform both rmdir and mkdir on the symlink (and ignore the error for operating on a non-directory).

I'd prefer to not delete, since I consider directories below share/nls as "infrastructure", even though they are not covered by an mtree definition (AFAIK).

Having "share/nls/C" re-appear after a mkdir "share/nls/en_US.US-ASCII/" seems surprising - but perhaps the idea to have "POSIX", "en_US.US-ASCII", and "en_US.US_ASCII" all being symlinks to "nls/share/C" is the real problem (with "en_US.US_ASCII" being an outlier anyway - it does not exist for any other "en" region.

And BTW: It is funny to see that there is only "en_IE.UTF-8" and no directory for the corresponding ISO locales or US-ASCII. Maybe this was an oversight when this locale was created?
Comment 3 Baptiste Daroussin freebsd_committer 2019-04-21 19:32:19 UTC
I will test with your committed port and fix the issue if pkg really tried to delete a directory pointed at by a symlink onstead of the said symlink. Because yes that is a bug if that is the case, and thanks for reporting. I will come back to you as soon as I found time to give it a try