Bug 180568 - r251668 breaks applications which depends on dlopen("libc.so") [regression]
Summary: r251668 breaks applications which depends on dlopen("libc.so") [regression]
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-07-15 07:20 UTC by Taku YAMAMOTO
Modified: 2024-10-10 20:34 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Taku YAMAMOTO 2013-07-15 07:20:00 UTC
After r251668 /usr/lib/libc.so becomes a plain ASCII text file which contains a linker script.

Unfortunately, now dlopen("libc.so") fails because it's not an ELF file, resulting applications which depend on dlopen("libc.so") get broken.

Applications which are broken by r251668 currently I know includes, but potentially not limited to:
 - Firefox (it boots, but exhibits broken behaviour)
 - Mono applications which P/Invoke libc functions (ends up with System.DllNotFoundException: libc.so)

Fix: 

Note this is only a workaround!

Put the following line into libmap.conf:
libc.so	libc.so.7
How-To-Repeat: Firefox case:
Launch Firefox with libmap.conf empty.
Firefox boots, but it won't load the previous session, URL bar won't navigate us anywhere, etc.

Mono applications which use libgdiplus, or anything which P/Invoke's libc's functions:
Launch such applications with libmap.conf empty.
They ends up with: System.DllNotFoundException: libc.so
Comment 1 Gleb Smirnoff freebsd_committer freebsd_triage 2013-07-16 13:46:10 UTC
On Mon, Jul 15, 2013 at 06:19:27AM +0000, Taku YAMAMOTO wrote:
T> >Description:
T> After r251672 /usr/lib/libc.so becomes a plain ASCII text file which contains a linker script.

r251672 is a comment only change, isn't it?


-- 
Totus tuus, Glebius.
Comment 2 Taku YAMAMOTO 2013-07-16 16:54:45 UTC
On Tue, 16 Jul 2013 16:46:10 +0400
Gleb Smirnoff <glebius@FreeBSD.org> wrote:

> On Mon, Jul 15, 2013 at 06:19:27AM +0000, Taku YAMAMOTO wrote:
> T> >Description:
> T> After r251672 /usr/lib/libc.so becomes a plain ASCII text file which contains a linker script.
> 
> r251672 is a comment only change, isn't it?
> 
> 
> -- 
> Totus tuus, Glebius.

Oops, it's actually r251668 which I wanted to mean.

-- 
-|-__   YAMAMOTO, Taku
 | __ <     <taku@tackymt.homeip.net>

      - A chicken is an egg's way of producing more eggs. -

[note: this PR was edited by linimon to reflect the proper Subject, per request]
Comment 3 Mark Linimon freebsd_committer freebsd_triage 2013-07-18 03:06:14 UTC
Responsible Changed
From-To: freebsd-bugs->jlh

Over to committer in question.
Comment 4 Jeremie Le Hen freebsd_committer freebsd_triage 2013-07-23 19:34:17 UTC
(Re-sending with the right GNATS email address.)

On Thu, Jul 18, 2013 at 02:09:50AM +0000, linimon@FreeBSD.org wrote:
> Old Synopsis: r251672 breaks applications which depends on dlopen("libc.so")
> New Synopsis: r251668 breaks applications which depends on dlopen("libc.so")
>
> Responsible-Changed-From-To: freebsd-bugs->jlh
> Responsible-Changed-By: linimon
> Responsible-Changed-When: Thu Jul 18 02:06:14 UTC 2013
> Responsible-Changed-Why:
> Over to committer in question.
>
> http://www.freebsd.org/cgi/query-pr.cgi?pr=180568

Thanks for reporting this.  I've done a quick sweep over firefox source
code but the latest release is more than 700 MB.  Can you manage to get
a stacktrace please?

--
Jeremie Le Hen

Scientists say the world is made up of Protons, Neutrons and Electrons.
They forgot to mention Morons.
Comment 5 dfilter service freebsd_committer freebsd_triage 2013-08-01 06:50:51 UTC
Author: jlh
Date: Thu Aug  1 05:50:42 2013
New Revision: 253853
URL: http://svnweb.freebsd.org/changeset/base/253853

Log:
  Include /usr/local/etc/libmap.d/ by default.
  
  PR:		180568
  Reviewed by:	bapt
  Obtained from:	kib
  MFC after:	3 days

Added:
  head/etc/libmap.conf   (contents, props changed)
Modified:
  head/etc/Makefile

Modified: head/etc/Makefile
==============================================================================
--- head/etc/Makefile	Thu Aug  1 04:50:46 2013	(r253852)
+++ head/etc/Makefile	Thu Aug  1 05:50:42 2013	(r253853)
@@ -22,6 +22,7 @@ BIN1=	crontab \
 	hosts.equiv \
 	inetd.conf \
 	libalias.conf \
+	libmap.conf \
 	login.access \
 	login.conf \
 	mac.conf \

Added: head/etc/libmap.conf
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/etc/libmap.conf	Thu Aug  1 05:50:42 2013	(r253853)
@@ -0,0 +1,2 @@
+# $FreeBSD$
+includedir /usr/local/etc/libmap.d
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 6 Jeremie Le Hen freebsd_committer freebsd_triage 2013-08-03 11:23:52 UTC
Hi Taku,

On Tue, Jul 23, 2013 at 08:34:17PM +0200, Jeremie Le Hen wrote:
> (Re-sending with the right GNATS email address.)
> 
> On Thu, Jul 18, 2013 at 02:09:50AM +0000, linimon@FreeBSD.org wrote:
> > Old Synopsis: r251672 breaks applications which depends on dlopen("libc.so")
> > New Synopsis: r251668 breaks applications which depends on dlopen("libc.so")
> >
> > Responsible-Changed-From-To: freebsd-bugs->jlh
> > Responsible-Changed-By: linimon
> > Responsible-Changed-When: Thu Jul 18 02:06:14 UTC 2013
> > Responsible-Changed-Why:
> > Over to committer in question.
> >
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=180568
> 
> Thanks for reporting this.  I've done a quick sweep over firefox source
> code but the latest release is more than 700 MB.  Can you manage to get
> a stacktrace please?

Can you please try the following in /etc/libmap.conf:

    [/usr/local/bin/firefox]
    libc.so	    libc.so.7

If it does not work, try prepending "/lib/" before "libc.so.7".

If that works, we will manage to provide a fix for 9.2-RELEASE, so we
have to be swift because the release is done in August.


If if still crashes, can you run firefox with ktrace and provide me with
the resulting file please?


-- 
Jeremie Le Hen

Scientists say the world is made up of Protons, Neutrons and Electrons.
They forgot to mention Morons.
Comment 7 Taku YAMAMOTO 2013-08-04 17:33:56 UTC
On Sat, 3 Aug 2013 12:23:52 +0200
Jeremie Le Hen <jlh@FreeBSD.org> wrote:

> Hi Taku,
> 
> On Tue, Jul 23, 2013 at 08:34:17PM +0200, Jeremie Le Hen wrote:
> > (Re-sending with the right GNATS email address.)
> > 
> > On Thu, Jul 18, 2013 at 02:09:50AM +0000, linimon@FreeBSD.org wrote:
> > > Old Synopsis: r251672 breaks applications which depends on dlopen("libc.so")
> > > New Synopsis: r251668 breaks applications which depends on dlopen("libc.so")
> > >
> > > Responsible-Changed-From-To: freebsd-bugs->jlh
> > > Responsible-Changed-By: linimon
> > > Responsible-Changed-When: Thu Jul 18 02:06:14 UTC 2013
> > > Responsible-Changed-Why:
> > > Over to committer in question.
> > >
> > > http://www.freebsd.org/cgi/query-pr.cgi?pr=180568
> > 
> > Thanks for reporting this.  I've done a quick sweep over firefox source
> > code but the latest release is more than 700 MB.  Can you manage to get
> > a stacktrace please?
> 
> Can you please try the following in /etc/libmap.conf:
> 
>     [/usr/local/bin/firefox]
>     libc.so	    libc.so.7

This is what I've already suggested as a *WORKAROUND* which has been working
well for a month.

But please see below...

> If it does not work, try prepending "/lib/" before "libc.so.7".
> 
> If that works, we will manage to provide a fix for 9.2-RELEASE, so we
> have to be swift because the release is done in August.

At least firefox is concerned, we've already have a patch since
ports/r319816 - it no longer resort to fallback to libc.so but loads libc.so.7
explicitly - it looks promising, though I haven't had a time to update my ports
tree to catch up and test with :(

> If if still crashes, can you run firefox with ktrace and provide me with
> the resulting file please?

Again, I've rather been concerned about applications in general depending
upon dlopen-able libc.so than particular ports which we can teach each
and every one not to do so.

-- 
-|-__   YAMAMOTO, Taku
 | __ <     <taku@tackymt.homeip.net>

      - A chicken is an egg's way of producing more eggs. -
Comment 8 Jeremie Le Hen freebsd_committer freebsd_triage 2013-08-04 18:35:30 UTC
+Cc: bapt@ and kib@

On Mon, Aug 05, 2013 at 01:33:56AM +0900, Taku YAMAMOTO wrote:
> On Sat, 3 Aug 2013 12:23:52 +0200
> Jeremie Le Hen <jlh@FreeBSD.org> wrote:
> > Can you please try the following in /etc/libmap.conf:
> > 
> >     [/usr/local/bin/firefox]
> >     libc.so	    libc.so.7
> 
> This is what I've already suggested as a *WORKAROUND* which has been working
> well for a month.

Oh sorry, I overlooked that.


> But please see below...
> 
> > If it does not work, try prepending "/lib/" before "libc.so.7".
> > 
> > If that works, we will manage to provide a fix for 9.2-RELEASE, so we
> > have to be swift because the release is done in August.
> 
> At least firefox is concerned, we've already have a patch since
> ports/r319816 - it no longer resort to fallback to libc.so but loads libc.so.7
> explicitly - it looks promising, though I haven't had a time to update my ports
> tree to catch up and test with :(

bapt@ should implement a simple framework in the ports to create a file
in /usr/local/etc/libmap.d/${PORTNAME}.conf.  This will allow port
maintainers to easily implement a workaround for this kind of softwares.


> > If if still crashes, can you run firefox with ktrace and provide me with
> > the resulting file please?
> 
> Again, I've rather been concerned about applications in general depending
> upon dlopen-able libc.so than particular ports which we can teach each
> and every one not to do so.

The core problem is that libc.so (or any other .so) is historically
used by both the linker and the runtime loader.  This worked well until
the linker could read an ld script from the .so.  Now we have to
decouple the rendez-vous point for the linker and rtld.

I've implemented a patch that can solve this problem, but it is not
perfect.  Basically, when installing a .so which contains an ld script,
he will also create a .so.last symlink pointing to the real shared
object (thus actually decouplin the rendez-vous point).  I modified rtld
to try the .so.last first if it exists.

http://people.freebsd.org/~jlh/rtld_last_so_suffix.patch

I think this is slightly hackish but it works; if your application is
linked to an older version of the lib, it will fail but just as it would
fail today anyway.

kib@ pointed that with the libmap.conf solution, your application will
still work even after the shared object major number is bumped
(libc.so.7 -> libc.so.8).

This is more work for maintainers but on the other hand this means that
this kind of problems must be solved now (I think there are probably
less than 10 in the whole port tree) and this will be solved correctly.

Whereas with my patch the problem doesn't exist and doen't have to be
fixed as long as the shared object major number is not bumped, but may
bite the user unexpectedly after a major FreeBSD upgrade (though he
would probably be answered to recompile the port or reinstall the
package).

Do you have any thought on that?

Regards,
-- 
Jeremie Le Hen

Scientists say the world is made up of Protons, Neutrons and Electrons.
They forgot to mention Morons.
Comment 9 Kostik Belousov 2013-08-05 09:29:26 UTC
On Sun, Aug 04, 2013 at 07:35:30PM +0200, Jeremie Le Hen wrote:
> +Cc: bapt@ and kib@
> 
> On Mon, Aug 05, 2013 at 01:33:56AM +0900, Taku YAMAMOTO wrote:
> > On Sat, 3 Aug 2013 12:23:52 +0200
> > Jeremie Le Hen <jlh@FreeBSD.org> wrote:
> > > Can you please try the following in /etc/libmap.conf:
> > > 
> > >     [/usr/local/bin/firefox]
> > >     libc.so	    libc.so.7
> > 
> > This is what I've already suggested as a *WORKAROUND* which has been working
> > well for a month.
> 
> Oh sorry, I overlooked that.
> 
> 
> > But please see below...
> > 
> > > If it does not work, try prepending "/lib/" before "libc.so.7".
> > > 
> > > If that works, we will manage to provide a fix for 9.2-RELEASE, so we
> > > have to be swift because the release is done in August.
> > 
> > At least firefox is concerned, we've already have a patch since
> > ports/r319816 - it no longer resort to fallback to libc.so but loads libc.so.7
> > explicitly - it looks promising, though I haven't had a time to update my ports
> > tree to catch up and test with :(
> 
> bapt@ should implement a simple framework in the ports to create a file
> in /usr/local/etc/libmap.d/${PORTNAME}.conf.  This will allow port
> maintainers to easily implement a workaround for this kind of softwares.
> 
> 
> > > If if still crashes, can you run firefox with ktrace and provide me with
> > > the resulting file please?
> > 
> > Again, I've rather been concerned about applications in general depending
> > upon dlopen-able libc.so than particular ports which we can teach each
> > and every one not to do so.
> 
> The core problem is that libc.so (or any other .so) is historically
> used by both the linker and the runtime loader.  This worked well until
> the linker could read an ld script from the .so.  Now we have to
> decouple the rendez-vous point for the linker and rtld.
> 
> I've implemented a patch that can solve this problem, but it is not
> perfect.  Basically, when installing a .so which contains an ld script,
> he will also create a .so.last symlink pointing to the real shared
> object (thus actually decouplin the rendez-vous point).  I modified rtld
> to try the .so.last first if it exists.
> 
> http://people.freebsd.org/~jlh/rtld_last_so_suffix.patch
> 
> I think this is slightly hackish but it works; if your application is
> linked to an older version of the lib, it will fail but just as it would
> fail today anyway.
> 
> kib@ pointed that with the libmap.conf solution, your application will
> still work even after the shared object major number is bumped
> (libc.so.7 -> libc.so.8).
> 
> This is more work for maintainers but on the other hand this means that
> this kind of problems must be solved now (I think there are probably
> less than 10 in the whole port tree) and this will be solved correctly.
> 
> Whereas with my patch the problem doesn't exist and doen't have to be
> fixed as long as the shared object major number is not bumped, but may
> bite the user unexpectedly after a major FreeBSD upgrade (though he
> would probably be answered to recompile the port or reinstall the
> package).
> 
> Do you have any thought on that?


I also do not like the '.so.last' hack since it creates undocumented
(in the ELF standard) behaviour deviating from the behaviour of the
dynamic loaders on other ELF systems.  Any issues resulting from
it would be very confusing and probably unpenetrable for the outside
developer.
Comment 10 dfilter service freebsd_committer freebsd_triage 2013-08-14 18:49:16 UTC
Author: jlh
Date: Wed Aug 14 17:49:08 2013
New Revision: 254332
URL: http://svnweb.freebsd.org/changeset/base/254332

Log:
  MFC r253853:
    Include /usr/local/etc/libmap.d/ by default.
  
    PR:             180568
    Reviewed by:    bapt
    Obtained from:  kib

Added:
  stable/9/etc/libmap.conf
     - copied unchanged from r253853, head/etc/libmap.conf
Modified:
  stable/9/etc/Makefile
Directory Properties:
  stable/9/etc/   (props changed)

Modified: stable/9/etc/Makefile
==============================================================================
--- stable/9/etc/Makefile	Wed Aug 14 16:15:14 2013	(r254331)
+++ stable/9/etc/Makefile	Wed Aug 14 17:49:08 2013	(r254332)
@@ -22,6 +22,7 @@ BIN1=	crontab \
 	hosts.equiv \
 	inetd.conf \
 	libalias.conf \
+	libmap.conf \
 	login.access \
 	login.conf \
 	mac.conf \

Copied: stable/9/etc/libmap.conf (from r253853, head/etc/libmap.conf)
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ stable/9/etc/libmap.conf	Wed Aug 14 17:49:08 2013	(r254332, copy of r253853, head/etc/libmap.conf)
@@ -0,0 +1,2 @@
+# $FreeBSD$
+includedir /usr/local/etc/libmap.d
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 11 dfilter service freebsd_committer freebsd_triage 2013-08-15 09:12:24 UTC
Author: jlh
Date: Thu Aug 15 08:12:16 2013
New Revision: 254357
URL: http://svnweb.freebsd.org/changeset/base/254357

Log:
  MFC r253853:
    Include /usr/local/etc/libmap.d/ by default.
  
    PR:             180568
    Reviewed by:    bapt
    Obtained from:  kib
  
  Approved by:      re (delphij)

Added:
  releng/9.2/etc/libmap.conf
     - copied unchanged from r254332, stable/9/etc/libmap.conf
Modified:
  releng/9.2/etc/Makefile
Directory Properties:
  releng/9.2/etc/   (props changed)

Modified: releng/9.2/etc/Makefile
==============================================================================
--- releng/9.2/etc/Makefile	Thu Aug 15 07:54:31 2013	(r254356)
+++ releng/9.2/etc/Makefile	Thu Aug 15 08:12:16 2013	(r254357)
@@ -22,6 +22,7 @@ BIN1=	crontab \
 	hosts.equiv \
 	inetd.conf \
 	libalias.conf \
+	libmap.conf \
 	login.access \
 	login.conf \
 	mac.conf \

Copied: releng/9.2/etc/libmap.conf (from r254332, stable/9/etc/libmap.conf)
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ releng/9.2/etc/libmap.conf	Thu Aug 15 08:12:16 2013	(r254357, copy of r254332, stable/9/etc/libmap.conf)
@@ -0,0 +1,2 @@
+# $FreeBSD$
+includedir /usr/local/etc/libmap.d
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
Comment 12 commit-hook freebsd_committer freebsd_triage 2014-12-18 21:03:14 UTC
A commit references this bug:

Author: bdrewery
Date: Thu Dec 18 21:02:56 UTC 2014
New revision: 374921
URL: https://svnweb.freebsd.org/changeset/ports/374921

Log:
  - Fix loading of libc.so on FreeBSD 10+ where it is an ldscript.

  PR:		ports/180568 [umbrella bug for dlopen("libco.so")]
  Submitted by:	Anthony Cornehl
  Sponsored by:	EMC / Isilon Storage Division

Changes:
  head/net/csocks/Makefile
Comment 13 Glen Barber freebsd_committer freebsd_triage 2015-07-08 18:32:13 UTC
To originators/assignees of this PR:

A commit to the tree references this PR, however the PR is still in a non-closed state.

Please review this PR and close as appropriate, or if closing the PR requires a merge to stable/10, please let re@ know as soon as possible.

Thank you.

Glen
Comment 14 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:43:40 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
Comment 15 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2019-01-20 03:53:16 UTC
Jeremie, can this PR be closed now?

Thanks