Bug 201299 - Merge Linux Emulation (lemul) to stable/10 branch for 10.2-RELEASE
Summary: Merge Linux Emulation (lemul) to stable/10 branch for 10.2-RELEASE
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.2-STABLE
Hardware: Any Any
: --- Affects Many People
Assignee: Dmitry Chagin
Keywords: feature
Depends on:
Reported: 2015-07-03 05:21 UTC by Johannes Jost Meixner
Modified: 2019-04-29 03:47 UTC (History)
15 users (show)

See Also:
koobs: maintainer-feedback+
koobs: mfc-stable10+

patch against stable/10 (10.2-BETA1, r285386) (846.04 KB, patch)
2015-07-12 00:51 UTC, meowthink
no flags Details | Diff
patch rerolled with svn (819.98 KB, patch)
2015-07-16 14:40 UTC, Nikolai Lifanov
no flags Details | Diff
patch: fixed and rerolled with svn (819.52 KB, patch)
2015-07-16 15:39 UTC, Nikolai Lifanov
no flags Details | Diff
patch against stable/10 (10.2-STABLE, r290203) (875.46 KB, patch)
2015-10-31 02:18 UTC, meowthink
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Johannes Jost Meixner freebsd_committer 2015-07-03 05:21:37 UTC
Dimitri Chagin worked on upgrading FreeBSD's emulation layer to support 64bit binaries.

This has made it into 11.0-CURRENT on May 24.

He has not had the time to merge it back between the start of the code slush, and the start of the code freeze.

I know at least three (big!) companies that have a usecase for having these lemul changes in 10.2-RELEASE.

The relevant commits can be viewed on


How do we get it in?
Comment 1 Glen Barber freebsd_committer 2015-07-07 17:22:26 UTC
Since binary packages are built on the oldest supported release on the branch, it needs to be verified that this change does *not* break ABI compatibility with 10.1-RELEASE.
Comment 2 Johannes Jost Meixner freebsd_committer 2015-07-08 21:22:29 UTC
Judging by trasz@'s reply on stable to the thread that included this PR[1],
it should not. Of course, I am no release engineer nor particularly knowledgeable about ABI compatibility in general.

I would hope that  trazs@ and emaste@ as reviewers, or dchagin@ as author of the code in question, could comment on this.

[1] https://lists.freebsd.org/pipermail/freebsd-stable/2015-July/082663.html
Comment 3 meowthink 2015-07-12 00:51:51 UTC
Created attachment 158638 [details]
patch against stable/10 (10.2-BETA1, r285386)

Here's the lemul64 patch, including patches up to commit e3f0cc2(svn r284626).
I actually grabbed dchagin's branch and made some tweeks by dropping 9a1d749(r279335), 547eb9a(r279817) ...  and some other drops due to 8f08266(r271976) and 67db24d(r277610) not avaliable in stable/10.
Commit 11cee2e(r275121) has been merged as a prerequisite.
Only a simple helloworld program is tested, as I don't set an 64-bit linux base up yet.
Comment 4 Glen Barber freebsd_committer 2015-07-12 00:58:50 UTC
Again, ABI backwards compatibility with 10.1-RELEASE needs to be verified, since packages are built against the *oldest* supported release on the branch.

Has anyone done this?

If so, what testing has been done, and what package(s) were used for verification of ABI compatibility?
Comment 5 Edward Tomasz Napierala freebsd_committer 2015-07-12 09:34:06 UTC
Note that I was assuming - and talking - about the ABI in context of MFC, which is the FreeBSD kernel ABI, not Linux userspace ABI.

I don't know about userspace ABI - the one for actual apps - but any kind of breakage in that would be immediately visible, and I hadn't seen any reports of that.
Comment 6 Dmitry Chagin freebsd_committer 2015-07-12 10:46:31 UTC
As far as I understand Linuxulator changes does not break the kernel ABI. I can spend a bit of time to merge it, but i do not have hw and time to test it.
Comment 7 Glen Barber freebsd_committer 2015-07-12 15:47:12 UTC
Kernel binary interface = KBI.

The way this needs to be tested is someone(tm) needs to build the packages on 10.1-RELEASE-pN, and install the packages on a 10.2-BETA installation, and ensure the linuxulator-affected binaries run.

Portmgr, can we set aside one of the exp builders for this, and make the resultant package set available publicly?  Although there is no proposed patch yet to RE, we would need to provide this to apply to the 10.1 userland for the package build jails.
Comment 8 Glen Barber freebsd_committer 2015-07-12 15:50:45 UTC
(In reply to Glen Barber from comment #7)
> Portmgr, can we set aside one of the exp builders for this, and make the
> resultant package set available publicly?

For clarification, I do not mean public for general use, I mean having the package set installable for general testing of ABI backwards-compatibility.

In other words, the packages built for this case should *not* be signed and published with the production sets.
Comment 9 Edward Tomasz Napierala freebsd_committer 2015-07-13 10:04:07 UTC
In this case, building the packages is actually just repacking existing binaries
from RPMs; they end up being the exact same binaries regardless of the FreeBSD version there were built ("repacked") under.
Comment 10 Edward Tomasz Napierala freebsd_committer 2015-07-13 10:05:14 UTC
That said, it's better safe than sorry, so exp build is certainly a good idea anyway.
Comment 11 Glen Barber freebsd_committer 2015-07-13 10:41:38 UTC
I'm going to make one final attempt to make sure I am very clear on the problem here:

Someone(tm) needs to verify that binaries built on 10.1-RELEASE (as they are built for all supported 10.x releases) continue to run on a system patched with this change.

The official upstream FreeBSD package builders *always* build packages from the last supported release on the branch (10.1-RELEASE, in this case) for all supported architectures for which we provide upstream packages.

Packages built on 10.1-RELEASE *must* continue to run on 10.2-RELEASE, otherwise we have broken ABI compatibility, and this is unacceptible on a stable branch.

Speculation that they will run is insufficient.

We are in a release cycle now.  Verification is the only proof that is acceptable at this point.
Comment 12 Johannes Jost Meixner freebsd_committer 2015-07-13 17:07:57 UTC
(In reply to meowthink from comment #3)
You could test with


if you have an svn/git portstree with arc set up on https://reviews.freebsd.org,

do `arc patch D1746' in the project root.

To be fair, running 64bit ports is something we will address at the next two Developers Summits.
Comment 13 Johannes Jost Meixner freebsd_committer 2015-07-14 10:03:36 UTC
(In reply to meowthink from comment #3)
Testing out your patch now. :-)
Comment 14 Johannes Jost Meixner freebsd_committer 2015-07-14 11:29:50 UTC
(In reply to Johannes Jost Meixner from comment #13)
Doesn't build; fails like so:


That's with a patched 10-STABLE off earlier this afternoon.

Path: .
Working Copy Root Path: /usr/src
URL: svn://svn.freebsd.org/base/stable/10
Relative URL: ^/stable/10
Repository Root: svn://svn.freebsd.org/base
Revision: 285529
Node Kind: directory
Schedule: normal
Last Changed Author: gjb
Last Changed Rev: 285521
Last Changed Date: 2015-07-13 20:32:25 -0500 (Mon, 13 Jul 2015)

Can someone else please take a look at the patch, fix the build and attach it here again?

Comment 15 meowthink 2015-07-15 12:25:36 UTC
(In reply to Johannes Jost Meixner from comment #14)

My patch is generated against git repo. You may encourage problems when applying it with svn, probably because of $FreeBSD$ not extended in git. Please check out what's not applied clearly by:

find /usr/src -name *.rej
Comment 16 Nikolai Lifanov 2015-07-16 14:40:34 UTC
Created attachment 158849 [details]
patch rerolled with svn

The previous patch applied cleanly to a 10-STABLE tree for me.
Here I rerolled the same patch with svn diff.
Comment 17 Nikolai Lifanov 2015-07-16 15:39:53 UTC
Created attachment 158855 [details]
patch: fixed and rerolled with svn

I see what the problem is. I fixed the patch to build and install.
I'm about to test the i386 c6 userland packages built on 10.0-RELEASE.
Comment 18 Nikolai Lifanov 2015-07-16 15:45:25 UTC
I confirmed that on 10.2-BETA1 system, running kernel from 10-STABLE with the patch I posted applied, i386 CentOS 6 packages work, my i386 CentOS 6.6 chroot works, and I was able to create an x86_64 CentOS 7.1 chroot, which works as well.
Comment 19 Nikolai Lifanov 2015-07-20 20:27:18 UTC
Are there any obstacles to this being accepted for MFC to 10.2?
Comment 20 Glen Barber freebsd_committer 2015-07-20 20:56:49 UTC
No one has submitted the commit approval request to re@.
Comment 21 Piotr Kubaj freebsd_committer 2015-07-21 10:46:39 UTC
(In reply to Glen Barber from comment #20)
Can anyone do it, or does it have to be from a committer?
Comment 22 Glen Barber freebsd_committer 2015-07-21 14:34:48 UTC
(In reply to pkubaj from comment #21)
> (In reply to Glen Barber from comment #20)
> Can anyone do it, or does it have to be from a committer?

Yes, it must be from a committer, specifically the committer planning to commit the change to the tree.
Comment 23 Edward Tomasz Napierala freebsd_committer 2015-07-21 17:45:41 UTC
I assumed Johannes was going to to do this, but now it looks like it might not happen.  Damn.

Glen, realistically, how much time do we have before it's too late for 10.2?
Comment 24 Glen Barber freebsd_committer 2015-07-21 17:54:43 UTC
If someone(tm) is going to propose this to re@ for inclusion in 10.2-RELEASE, it needs to happen as soon as possible.  (Preferably before the next 10.2 builds start this Friday.)
Comment 25 Johannes Jost Meixner freebsd_committer 2015-07-21 19:06:30 UTC
(In reply to Edward Tomasz Napierala from comment #23)

I was hoping Dmitry was going to do this before code slush, then before code freeze. Ah well, they're keeping him really busy at his job.

My problem is this --- I don't know C. Otherwise I'd have long since tried to pull it off.
Comment 26 Edward Tomasz Napierala freebsd_committer 2015-07-23 20:43:08 UTC
Okay.  So, this situation sucks, but I've thought this over and I'd rather not MFC it myself.

It's not just that I'd be quite uncomfortable merging a megabyte of somebody else's changes to somebody else's subsystem.  It's also that while I've reviewed some of those, there were quite huge ones (like the whole amd64 support) that were not reviewed at all, and might end up being a security vulnerability.  It's also that Linuxulator is not something I'm using on a regular basis (or actually at all, apart from quick detour to do some cleanups there), and it's highly probable I'd miss something in testing.

Now, if someone wanted to put linuxulator port from 11 into ports, like we eg have audio/oss - I'll happily help with that.
Comment 27 Glen Barber freebsd_committer 2015-07-23 21:08:21 UTC
At this point, I am inclined not to allow this change this late in the release cycle, as several concerns have been raised in this ticket, as well as within re@ internally.

We are no longer accepting MFC requests to stable/10 until after the 10.2-RC1 builds have finished, so I've changed the affected version from 10.1-STABLE to 10.2-STABLE.
Comment 28 Johannes Jost Meixner freebsd_committer 2015-07-24 12:31:22 UTC
(In reply to Glen Barber from comment #27)
Indeed, this seems the reasonable plan of action at this point.
Oh well. 11.0-RELEASE then ;-)
Comment 29 Ed Maste freebsd_committer 2015-07-30 19:49:01 UTC
Or perhaps 10.3
Comment 30 Nikolai Lifanov 2015-07-30 19:50:31 UTC
stable/10 is thawed, so it can be merged ASAP for increased exposure, right?
Comment 31 Ed Maste freebsd_committer 2015-07-30 19:58:19 UTC
> stable/10 is thawed, so it can be merged ASAP for increased exposure, right?

There's a desire for additional testing / review in HEAD prior to merging still.
Comment 32 Namik Smith 2015-09-21 21:23:06 UTC
Are there any updates as to hopes for merging into 10.2-STABLE in the near future? There's certainly strong interest because of the void that exists on the security front in the docker community in addition to more general use cases (https://forums.freebsd.org/threads/docker-for-freebsd-10-2.53258/#post-299515). 

If testing is the issue, might Dmitry or someone else who can MFC chime in with thoughts as to a desired testing protocol ahead of a merge? Maybe we can round up some help.
Comment 33 meowthink 2015-10-31 02:18:55 UTC
Created attachment 162629 [details]
patch against stable/10 (10.2-STABLE, r290203)
Comment 34 Kubilay Kocak freebsd_committer freebsd_triage 2015-10-31 03:16:59 UTC
@Meowthink thank you!
Comment 35 Dmitry Chagin freebsd_committer 2016-01-09 18:59:50 UTC
merged at r293610
Comment 36 Kubilay Kocak freebsd_committer freebsd_triage 2019-04-29 03:47:08 UTC
Track merge to stable/10 in base r293609 and base r293610