Bug 171250

Summary: ldd32 cannot find some i386 libraries
Product: Base System Reporter: David Naylor <naylor.b.david>
Component: amd64Assignee: freebsd-amd64 (Nobody) <amd64>
Status: Closed Overcome By Events    
Severity: Affects Only Me CC: dbn
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   

Description David Naylor 2012-09-02 13:50:06 UTC
ldd32 does not find libraries, yet ldconfig lists them as known.  

# ldconfig -32 -r | grep directories
        search directories: /usr/lib32:/usr/local/lib32:/usr/local/lib32/wine
# ldconfig -32 -r | grep 'lssl.6\|iconv.3'
        94:-lssl.6 => /usr/lib32/libssl.so.6
        124:-liconv.3 => /usr/local/lib32/libiconv.so.3
# ldd32 /usr/local/lib32/libcups.so.2 
/usr/local/lib32/libcups.so.2:
        libssl.so.6 => not found (0)
        libcrypto.so.6 => /usr/lib32/libcrypto.so.6 (0x281f4000)
        libcrypt.so.5 => /usr/lib32/libcrypt.so.5 (0x28351000)
        libm.so.5 => /usr/lib32/libm.so.5 (0x28376000)
        libiconv.so.3 => not found (0)
        libz.so.6 => /usr/lib32/libz.so.6 (0x28390000)
        libthr.so.3 => /usr/lib32/libthr.so.3 (0x283a4000)
        libc.so.7 => /usr/lib32/libc.so.7 (0x2806c000)

Fix: 

Explicitly include all library paths in LD_32_LIBRARY_PATH:
# env LD_32_LIBRARY_PATH=/usr/local/lib32:/usr/lib32 ldd32 /usr/local/lib32/libcups.so.2
/usr/local/lib32/libcups.so.2:
        libssl.so.6 => /usr/lib32/libssl.so.6 (0x281f4000)
        libcrypto.so.6 => /usr/lib32/libcrypto.so.6 (0x2823d000)
        libcrypt.so.5 => /usr/lib32/libcrypt.so.5 (0x2839a000)
        libm.so.5 => /usr/lib32/libm.so.5 (0x283bf000)
        libiconv.so.3 => /usr/local/lib32/libiconv.so.3 (0x28800000)
        libz.so.6 => /usr/lib32/libz.so.6 (0x283d9000)
        libthr.so.3 => /usr/lib32/libthr.so.3 (0x288f8000)
        libc.so.7 => /usr/lib32/libc.so.7 (0x2806c000)
How-To-Repeat: Install wine-fbsd64 from www.mediafire.com/wine_fbsd64 and follow instructions above.
Comment 1 Kostik Belousov 2012-09-02 16:57:55 UTC
On Sun, Sep 02, 2012 at 12:42:43PM +0000, David Naylor wrote:
> ldd32 does not find libraries, yet ldconfig lists them as known.  
> 
> # ldconfig -32 -r | grep directories
>         search directories: /usr/lib32:/usr/local/lib32:/usr/local/lib32/wine

You should also look at the DT_RPATH and DT_RUNPATH tags in your binary.
I suspect that there is some path hardcoded which points to the
native 64bit libraries dir.

> # ldconfig -32 -r | grep 'lssl.6\|iconv.3'
>         94:-lssl.6 => /usr/lib32/libssl.so.6
>         124:-liconv.3 => /usr/local/lib32/libiconv.so.3
> # ldd32 /usr/local/lib32/libcups.so.2 
> /usr/local/lib32/libcups.so.2:
>         libssl.so.6 => not found (0)

If DT_R*PATH specified native directory, dynamic linker found the
libssl.so.6 which is for the wrong architecture, and claimed the
dependency not satisfied.

>         libcrypto.so.6 => /usr/lib32/libcrypto.so.6 (0x281f4000)
>         libcrypt.so.5 => /usr/lib32/libcrypt.so.5 (0x28351000)
>         libm.so.5 => /usr/lib32/libm.so.5 (0x28376000)
>         libiconv.so.3 => not found (0)
>         libz.so.6 => /usr/lib32/libz.so.6 (0x28390000)
>         libthr.so.3 => /usr/lib32/libthr.so.3 (0x283a4000)
>         libc.so.7 => /usr/lib32/libc.so.7 (0x2806c000)
> 
> >How-To-Repeat:
> Install wine-fbsd64 from www.mediafire.com/wine_fbsd64 and follow instructions above.  
> >Fix:
> Explicitly include all library paths in LD_32_LIBRARY_PATH:
> # env LD_32_LIBRARY_PATH=/usr/local/lib32:/usr/lib32 ldd32 /usr/local/lib32/libcups.so.2
> /usr/local/lib32/libcups.so.2:
>         libssl.so.6 => /usr/lib32/libssl.so.6 (0x281f4000)

If my theory holds, ths explicit use of LD_32_LIBRARY_PATH helps because
the env var path has precedence over the path from tag.
Comment 2 David Naylor 2012-09-03 13:01:43 UTC
On Sunday, 2 September 2012 17:57:55 Konstantin Belousov wrote:
> On Sun, Sep 02, 2012 at 12:42:43PM +0000, David Naylor wrote:
> > ldd32 does not find libraries, yet ldconfig lists them as known.
> > 
> > # ldconfig -32 -r | grep directories
> > 
> >         search directories:
> >         /usr/lib32:/usr/local/lib32:/usr/local/lib32/wine
> 
> You should also look at the DT_RPATH and DT_RUNPATH tags in your binary.
> I suspect that there is some path hardcoded which points to the
> native 64bit libraries dir.


# elfdump -d libcups.so.2
<snip/>
entry: 9
        d_tag: DT_RPATH
        d_val: /usr/lib:/usr/local/lib
<snip/>
# elfdump -a libcups.so.2 | grep DT_RUNPATH
...

As you suspected the paths are hardcoded.  However, since this is a 32bit 
library (and compiled in a i386 chroot) on a 64bit system I would have 
expected the dynamic linker to translate /usr/lib to /usr/lib32, or to prepend 
it.  

> If my theory holds, ths explicit use of LD_32_LIBRARY_PATH helps because
> the env var path has precedence over the path from tag.


If my expectation, mentioned above, is wrong then please close this bug.  I'm 
happy to continue using LD_32_LIBRARY_PATH to enduce the correct behaviour.  

Thanks for your quick reply.
Comment 3 Kostik Belousov 2012-09-03 13:26:50 UTC
On Mon, Sep 03, 2012 at 02:01:43PM +0200, David Naylor wrote:
> On Sunday, 2 September 2012 17:57:55 Konstantin Belousov wrote:
> > On Sun, Sep 02, 2012 at 12:42:43PM +0000, David Naylor wrote:
> > > ldd32 does not find libraries, yet ldconfig lists them as known.
> > > 
> > > # ldconfig -32 -r | grep directories
> > > 
> > >         search directories:
> > >         /usr/lib32:/usr/local/lib32:/usr/local/lib32/wine
> > 
> > You should also look at the DT_RPATH and DT_RUNPATH tags in your binary.
> > I suspect that there is some path hardcoded which points to the
> > native 64bit libraries dir.
> 
> # elfdump -d libcups.so.2
> <snip/>
> entry: 9
>         d_tag: DT_RPATH
>         d_val: /usr/lib:/usr/local/lib
> <snip/>
> # elfdump -a libcups.so.2 | grep DT_RUNPATH
> ...
> 
> As you suspected the paths are hardcoded.  However, since this is a 32bit 
> library (and compiled in a i386 chroot) on a 64bit system I would have 
> expected the dynamic linker to translate /usr/lib to /usr/lib32, or to prepend 
> it.  
> 
> > If my theory holds, ths explicit use of LD_32_LIBRARY_PATH helps because
> > the env var path has precedence over the path from tag.
> 
> If my expectation, mentioned above, is wrong then please close this bug.  I'm 
> happy to continue using LD_32_LIBRARY_PATH to enduce the correct behaviour.  
> 
> Thanks for your quick reply.


I have a symphathy to the idea that ld-elf32.so.1 should translate well-known
pathes from DT_RPATH into their 32bit compat synonims. That is, /lib and
/usr/lib probably should be interpreted as /lib32 and /usr/lib32.

On the other hand, I do not know what to do with non-default pathes,
that is /usr/local/lib in your case. Please note that some library can
be find there for many reasons, and I cannot imagine a sane way to
translate to 32bit compat path without involving some additional config.

This is closely related to non-existent multiarch support in ports, which
cannot even start happens without working cc -m32.

I think your PR shall be postponed instead of being closed.
Comment 4 David Naylor 2012-09-03 13:39:32 UTC
On Monday, 3 September 2012 14:26:50 Konstantin Belousov wrote:
> On Mon, Sep 03, 2012 at 02:01:43PM +0200, David Naylor wrote:
> > On Sunday, 2 September 2012 17:57:55 Konstantin Belousov wrote:
> > > On Sun, Sep 02, 2012 at 12:42:43PM +0000, David Naylor wrote:
> > > > ldd32 does not find libraries, yet ldconfig lists them as known.
> > > > 
> > > > # ldconfig -32 -r | grep directories
> > > > 
> > > >         search directories:
> > > >         /usr/lib32:/usr/local/lib32:/usr/local/lib32/wine
> > > 
> > > You should also look at the DT_RPATH and DT_RUNPATH tags in your
> > > binary. I suspect that there is some path hardcoded which points to
> > > the native 64bit libraries dir.
> > 
> > # elfdump -d libcups.so.2
> > <snip/>
> > entry: 9
> > 
> >         d_tag: DT_RPATH
> >         d_val: /usr/lib:/usr/local/lib
> > 
> > <snip/>
> > # elfdump -a libcups.so.2 | grep DT_RUNPATH
> > ...
> > 
> > As you suspected the paths are hardcoded.  However, since this is a 32bit
> > library (and compiled in a i386 chroot) on a 64bit system I would have
> > expected the dynamic linker to translate /usr/lib to /usr/lib32, or to
> > prepend it.
> > 
> > > If my theory holds, ths explicit use of LD_32_LIBRARY_PATH helps
> > > because the env var path has precedence over the path from tag.
> > 
> > If my expectation, mentioned above, is wrong then please close this bug. 
> > I'm happy to continue using LD_32_LIBRARY_PATH to enduce the correct
> > behaviour.
> > 
> > Thanks for your quick reply.
> 
> I have a symphathy to the idea that ld-elf32.so.1 should translate
> well-known pathes from DT_RPATH into their 32bit compat synonims. That is,
> /lib and /usr/lib probably should be interpreted as /lib32 and /usr/lib32.


FYI, I do not have a /lib32 on my system, only /usr/lib32.  

Would not a reasonable first step be to simple append /usr/lib32 to the search 
path, thus if there is reason to find a valid library in the normal path it 
would be discovered first, but fall back to the compat directory if needed.  

> On the other hand, I do not know what to do with non-default pathes,
> that is /usr/local/lib in your case. Please note that some library can
> be find there for many reasons, and I cannot imagine a sane way to
> translate to 32bit compat path without involving some additional config.


The append method above, doing s/lib/lib32/, may be an acceptable attempt.  
The way I current do it with wine-fbsd64 packages is to create a shell script 
that added the non-default paths to LD_32_LIBRARY_PATH and `exec` to the 
binaries under bin32.  

> This is closely related to non-existent multiarch support in ports, which
> cannot even start happens without working cc -m32.
> 
> I think your PR shall be postponed instead of being closed.


Happy bug hunting...
Comment 5 Peter Jeremy 2012-09-04 12:40:56 UTC
On 2012-Sep-03 12:30:19 +0000, Konstantin Belousov <kostikbel@gmail.com> wrote:
> I have a symphathy to the idea that ld-elf32.so.1 should translate well-known
> pathes from DT_RPATH into their 32bit compat synonims. That is, /lib and
> /usr/lib probably should be interpreted as /lib32 and /usr/lib32.


That sounds like a good idea to me as well.

> On the other hand, I do not know what to do with non-default pathes,
> that is /usr/local/lib in your case. Please note that some library can
> be find there for many reasons, and I cannot imagine a sane way to
> translate to 32bit compat path without involving some additional config.


My suggestion is that we define /usr/local/lib32 as a standard
directory (and wire it into ld-elf32.so.1 in the same wap as /lib &
/usr/lib) and extend /etc/libmap32.conf to define directory name
mappings as well as dynamic object mappings to handle cases where
LOCALBASE is not /usr/local and non-standard library directories.

> This is closely related to non-existent multiarch support in ports, which
> cannot even start happens without working cc -m32.


I would disagree on this point.  There's no reason why we need a working
"cc -m32" in order to pkg_add an i386 package on an amd64 system.  I
notice there was some discussion regarding multi-arch ports on -current
in late March.

> I think your PR shall be postponed instead of being closed.


Agreed.

-- 
Peter Jeremy
Comment 6 David Naylor freebsd_committer freebsd_triage 2014-12-16 14:39:36 UTC
Any progress on this?  Can this bug be closed (as it doesn't actually cause any harm)?
Comment 7 David Naylor freebsd_committer freebsd_triage 2015-04-19 12:59:30 UTC
Timeout