Bug 237427 - lang/ldc: update to 1.22.0, unbreak on aarch64
Summary: lang/ldc: update to 1.22.0, unbreak on aarch64
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Jose Alonso Cardenas Marquez
URL:
Keywords:
: 244262 (view as bug list)
Depends on: 245805 248023
Blocks:
  Show dependency treegraph
 
Reported: 2019-04-20 23:58 UTC by Val Packett
Modified: 2020-11-15 15:23 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (acm)


Attachments
ldc115.patch (10.92 KB, patch)
2019-04-20 23:58 UTC, Val Packett
no flags Details | Diff
ldc115.patch v2 (10.66 KB, patch)
2019-04-21 14:31 UTC, Val Packett
no flags Details | Diff
Possible replacement/update of the previous patch (30.51 KB, patch)
2019-06-20 18:23 UTC, Diederik de Groot
no flags Details | Diff
ldc116.patch (34.54 KB, patch)
2019-07-09 20:22 UTC, Val Packett
no flags Details | Diff
ldc118.patch (32.96 KB, patch)
2019-11-25 20:22 UTC, Val Packett
no flags Details | Diff
ldc119.patch (35.58 KB, patch)
2020-01-11 19:01 UTC, Val Packett
no flags Details | Diff
v1 (29.79 KB, patch)
2020-07-16 17:52 UTC, Mikael Urankar
no flags Details | Diff
v1 (29.79 KB, patch)
2020-08-22 15:24 UTC, Mikael Urankar
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Val Packett 2019-04-20 23:58:17 UTC
Created attachment 203847 [details]
ldc115.patch

and use ninja to build the bootstrap compiler quickly and in parallel.

v1.15.0 relnotes: https://github.com/ldc-developers/ldc/releases/tag/v1.15.0

This version brings LLVM 8 support, however not in ltsmaster it seems:

FAILED: dmd2/impcnvgen 
: && /usr/bin/c++     CMakeFiles/impcnvgen.dir/dmd2/impcnvgen.c.o  -o dmd2/impcnvgen   && :
ld: error: undefined symbol: llvm::DisableABIBreakingChecks

Might be worth it to use LLVM 7 for bootstrap and 8 for the final compiler. And to rebuild the final compiler with itself because the lowmem option (see relnotes) sounds pretty cool. But I'm not very interested in all that right now, that's just notes for later.
Comment 1 Jose Alonso Cardenas Marquez freebsd_committer freebsd_triage 2019-04-21 01:25:49 UTC
What version of FreeBSD are you using? Did you test it on FreeBSD 12? I see that you removed two patch files and they are necessary for ldc compiles/works fine on FreeBSD >=12

On another hand, I can't commit ldc 1.15.x update until that all ports that depends of this compiler work fine with it
Comment 2 Val Packett 2019-04-21 14:24:20 UTC
(In reply to Jose Alonso Cardenas Marquez from comment #1)

I tested on CURRENT.

These patches did not apply, I thought they were upstreamed, but this must've been an error on my side, it makes no sense that they would merge only one of the files.
Comment 3 Val Packett 2019-04-21 14:31:41 UTC
Created attachment 203864 [details]
ldc115.patch v2

Regenerated dirent patch instead of removing
Comment 4 Val Packett 2019-05-01 15:53:24 UTC
ping
Comment 5 Diederik de Groot 2019-06-15 11:32:16 UTC
@Greg_V: 

The `ld: error: undefined symbol: llvm::DisableABIBreakingChecks` is still remains when building the bootstrap compiler. This might have already been resolved in 1.16.0-beta2. For 1.15.0 it might be better to stick with llvm70 (as a quick fix). Won't be long for 1.16.0 to be released anyway.

Also did you upstream the patches for setjmp.d, ucontext.d and gammafunction.d ?

BTW: You could do: 
    USES=ninja
and replace:
    ${SETENV} ${CONFIGURE_ENV} ${CMAKE_BIN} -GNinja .
with
    ${SETENV} ${CONFIGURE_ENV} ${CMAKE_BIN} ${CMAKE_ARGS} .

See: /usr/ports/Mk/Uses/ninja.mk
Comment 6 Val Packett 2019-06-15 13:29:44 UTC
(In reply to Diederik de Groot from comment #5)

Yes, I mentioned that in the comment at the beginning of the patch. It's been upstreamed quite some time ago https://github.com/dlang/druntime/pull/2269 and merged into ldc's master. But wasn't cherry-picked into bootstrap/ltsmaster (you can see the patch here is only for ltsmaster). I pinged them two months ago to cherry-pick it, guess I need to send another reminder :)

And yeah, the patch currently uses llvm70 already
Comment 7 Diederik de Groot 2019-06-16 18:32:37 UTC
(In reply to Greg V from comment #6)

To be able to compile ltsmaster with llvm80 would require quite a number of patches. So llvm70 seems the better choice for the bootstrap compiler. If need be we would also download a freebsd precompiled bootstrap compiler (as is done for other languages like ada/go/gcc etc.

If you do want to compile ltsmaster with llvm70 and 1.15.0 with ${LLVM_DEFAULT}, then it might be a good idea to split the Makefile into two steps (Makefile and Makefile.bootstrap) (to ease maintenance). Logically the divide between bootstrap and end compiler will only increase over time. For 1.15.0 it might still be the easier way to just force llvm70 (for now).
Comment 8 Diederik de Groot 2019-06-20 18:23:24 UTC
Created attachment 205239 [details]
Possible replacement/update of the previous patch

Split the Makefile in two parts to ease bootstrapping in the future
Search for pre-existing, usable d-compiler for bootstrapping purposes
Use FLAVORS for llvm70 and llvm80 (which can both be installed at the same time)
Comment 9 Diederik de Groot 2019-06-20 19:23:10 UTC
(In reply to Diederik de Groot from comment #8)

(I reused part of Greg_v's previous patch)

Notes:
- When no previously installed suitable d-compiler can be found, it automatically falls back to building the ltsmaster based bootstrap compiler.
- the ltsmaster bootstrap compiler is build with llvm70 (for now, as it is not yet made compatible with llvm80).
- Both flavors can be installed at the same time (under /usr/local/llvmXX-ldc (similar llvmXX itself).
- When no /usr/local/bin/ldc2 binary is find, a link is created to the llvmXX-ldc/bin/ldc2, to make the d compiler easier to be found.
- WIP: Mk/Uses/dcompiler.mk (See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230517), so that other d ports can be made easier. And ldc2/gdc can be used to bootstrap lang/dmd,

@Greg_V: Hopefully i did not offend by supplying a potential replacement for your patch and and link it to your bug-report.
Comment 10 Val Packett 2019-06-20 19:44:12 UTC
(In reply to Diederik de Groot from comment #9)

Nice. Ideally, poudriere would somehow be instructed to build ltsmaster, build current ldc, and then rebuild current ldc with itself..

btw, I received a report of an "intrinsics error on memcpy when linking tilix" with llvm70 on 12-stable, and reverting to llvm60 as a fix. (I asked for the error message.)

also re: dcompiler.mk (or ports framework in general): all Meson-built D programs need SSP_UNSAFE=yes because Meson currently likes to shove C flags into D compiler args https://github.com/mesonbuild/meson/issues/5369
Comment 11 Diederik de Groot 2019-06-20 19:57:02 UTC
(In reply to Greg V from comment #10)

Funny little thing i noticed. Currently ldc will build find with ltsmaster or any d-compiler with 0.79.1 > druntime < 0.84.0. But not with ldc-0.15.1 (or higher). So at the moment you cannot build 0.15.1 with 0.15.1 :-). I guess i will have to create an upstream issue for that.

re: meson. Does that mean the dcompile.mk would have to have SSP_UNSAFE=yes in it (by default) ?

Have not tried compiling with poudriere or meson so far. I thought it would be wise to first check the waters, in regard to pushing a patch to someone else's bug-report (yours in this case). If you like my solution, you can set your patch to obsolete.
Comment 12 Diederik de Groot 2019-06-20 20:08:23 UTC
(In reply to Greg V from comment #10)
re:tilix
Adding llvm60 as another flavor should not be an issue (i think)

I was planning to add a couple of arguments the dcompiler.mk script something like 
USES=dcompiler:gdc9
USES=dcompiler:llvm60-ldc or USES=dcompiler:ldc,llvm60
USES=dcompiler:minver=0.15.1,mindrt=20850  (not sure if this is possible)

So when they are in place, that would take care of the tilix requirement.
Comment 13 Val Packett 2019-07-07 22:36:18 UTC
Some D news:

- a (WIP) patch for LLVM 8 support in ltsmaster was published https://github.com/ldc-developers/ldc/pull/3092
- I have discovered that the ino64 (freebsd12) patch in ports missed dirent/readdir..
- and I made a 32-bit-inode versioned symbols patch to replace it: https://github.com/dlang/druntime/pull/2668
- I also published ldc patches for enabling dead code elimination via gc-symbols and making LTO with LLD work without a path to the gold LTO plugin: https://github.com/ldc-developers/ldc/pull/3105 https://github.com/ldc-developers/ldc/pull/3106


> re: meson. Does that mean the dcompile.mk would have to have SSP_UNSAFE=yes in it (by default) ?

Maybe we should just patch meson..

I'll look into sending a pull request to meson.
Comment 14 Val Packett 2019-07-09 20:22:19 UTC
Created attachment 205633 [details]
ldc116.patch

Back to packaging, here's 1.16.0,
tested on -CURRENT both amd64 and aarch64,
includes the bootstrap makefile split from Diederik de Groot's version (but not the flavors),
includes my patches for backtrace, lto, gc-sections, FreeBSD 11 ABI versioned symbols for building on >=12.

- I don't really like llvmXX flavors..
  - old llvm versions are pkg bloat, everything should move to latest LLVM when possible;
  - 'pkg install ldc' is a nicer command;
  - more serious reason: 'LIB_DEPENDS=libphobos2-ldc-shared.so:lang/ldc' in dynamically linked ports would not work! the port would have to know the ldc flavor and ugh this is too complex.
  - (maybe USES=dcompiler could manage the complexity, but we have one version currently, let's have just one version for as long as possible maybe?)
- I'm not sure where you got that DRUNTIME_MAXVER thing from — I've never had a problem building current (not bootstrap) ldc with itself.
- for some reason I've ended up the final compiler linked dynamically to bootstrap's phobos/druntime once.. but can't reproduce now even with no special flags that make the bootstrap only build static libs
- the check for existing D compilers spams 'sh: --version: not found' (for every not-installed compiler) and I haven't fixed that
Comment 15 Diederik de Groot 2019-11-21 02:53:25 UTC
Greg you updates are perfectly fine by me. Flavors are not needed, but i thought it would make a nice addition. I did not know that not everbody like them.

The DRUNTIME_MIN/MAX versions was need if you are bootstrapping from gdc or dmd2 which is also possible. Using a dmd2 druntime newer then the ldc2 caused some problems on some of my builds, so i added this flag.

With the new FreeBSD releases the ino64/ABI issue should be out of the way (right ?).

Any progress on getting the ldc116.patch accepted. Should we update it to 1.18.0 which is out already (FYI, just changing the version number and updating pkg-plist suffices).

Can we change the flag: 'Only affects me; above, as that is not really true ?
Comment 16 Val Packett 2019-11-24 12:16:14 UTC
(In reply to Diederik de Groot from comment #15)

> With the new FreeBSD releases the ino64/ABI issue should be out of the way (right ?).

Only after 11 goes EoL. Right now packages for 11 are still built.

> Should we update it to 1.18.0 which is out already

Yes. I'll update soon :)

> Can we change the flag: 'Only affects me; above

Does anyone care about it? Seems like one of those default bugzilla things that no one really needs.
Comment 17 Diederik de Groot 2019-11-24 19:30:45 UTC
(In reply to Greg V from comment #16)

> Only after 11 goes EoL. Right now packages for 11 are still built.
:-)

>> Should we update it to 1.18.0 which is out already
> Yes. I'll update soon :)
Super

>> Can we change the flag: 'Only affects me; above
>Does anyone care about it? Seems like one of those default bugzilla things that no one really needs.
I don't have a clue. But I don't understand why some patches take so very very long, to even be looked at. I was hoping this might be it :-) (Idle hope maybe).
Comment 18 Val Packett 2019-11-25 20:22:03 UTC
Created attachment 209422 [details]
ldc118.patch

Updated to 1.18.0. Using LLVM 9 now.

Tested on aarch64 and amd64, built by lts bootstrap, by 1.16.0 and by self.

Removed BROKEN_powerpc64 because some support was merged upstream (was done on old ABI, not sure if ELFv2 would work, but let's let someone on POWER retest anyway).

> I don't understand why some patches take so very very long, to even be looked at

Well, usually it's the maintainer not having the time or not paying attention.. We have maintainer-timeout for such cases.
Comment 19 Val Packett 2019-11-25 20:25:04 UTC
testing bootstrapping from GDC and DMD would be appreciated :)

Also forgot to mention:

> the ino64/ABI issue should be out of the way

right now we always use the 11 (32-bit) functions via versioned symbols, my PR for that was merged into upstream druntime https://github.com/dlang/druntime/pull/2668
Comment 20 Diederik de Groot 2019-11-25 20:41:05 UTC
(In reply to Greg V from comment #19)

> testing bootstrapping from GDC and DMD would be appreciated :)
I have already done that in my new Mk/Uses/dlang.mk used on dragonflybsd for the moment. Waiting until this ino issues is resolved to port it back.

You can have a look here: https://github.com/DragonFlyBSD/DeltaPorts/pull/918

> right now we always use the 11 (32-bit) functions via versioned symbols, my PR for that was merged into upstream druntime https://github.com/dlang/druntime/pull/2668

I think that is a good solution. So actually the issue should be resolved with the updated patch (right).
Comment 21 Val Packett 2020-01-11 19:01:31 UTC
Created attachment 210633 [details]
ldc119.patch

1.19 now.
Comment 22 Mikael Urankar freebsd_committer freebsd_triage 2020-07-16 17:52:37 UTC
Created attachment 216494 [details]
v1

update to 1.22.0 based on Greg's patch
Comment 23 Mikael Urankar freebsd_committer freebsd_triage 2020-07-16 17:55:37 UTC
(In reply to Mikael Urankar from comment #22)
OK on 11.4, 12.1 and 13 amd64
OK all consumers (with patch from bug #248023 and bug #245805)
Comment 24 Mikael Urankar freebsd_committer freebsd_triage 2020-07-16 17:57:31 UTC
*** Bug 244262 has been marked as a duplicate of this bug. ***
Comment 25 Mikael Urankar freebsd_committer freebsd_triage 2020-08-22 15:24:31 UTC
Created attachment 217446 [details]
v1

update to 1.23.0
Comment 26 commit-hook freebsd_committer freebsd_triage 2020-09-29 13:49:32 UTC
A commit references this bug:

Author: mikael
Date: Tue Sep 29 13:49:25 UTC 2020
New revision: 550507
URL: https://svnweb.freebsd.org/changeset/ports/550507

Log:
  lang/ldc: update to 1.23.0

  Changelog from 1.14.0 to 1.23.0 can be found here:
  https://github.com/ldc-developers/ldc/releases

  PR:		237427
  Approved by:	acm (maintainer timeout)

Changes:
  head/lang/ldc/Makefile
  head/lang/ldc/distinfo
  head/lang/ldc/files/fbsd12-runtime-druntime-src-core-sys-freebsd-sys_event.d
  head/lang/ldc/files/fbsd12-runtime-druntime-src-core-sys-freebsd-sys_mount.d
  head/lang/ldc/files/fbsd12-runtime-druntime-src-core-sys-posix-sys_stat.d
  head/lang/ldc/files/fbsd12-runtime-druntime-src-core-sys-posix-sys_types.d
  head/lang/ldc/files/fbsd12-runtime-druntime-src-core-sys-posix_dirent.d
  head/lang/ldc/files/patch-cmake_Modules_FindLLVM.cmake
  head/lang/ldc/files/patch-driver_linker-gcc.cpp
  head/lang/ldc/files/patch-driver_targetmachine.cpp
  head/lang/ldc/files/patch-ltsmaster_cmake_Modules_FindLLVM.cmake
  head/lang/ldc/files/patch-runtime_CMakeLists.txt
  head/lang/ldc/pkg-descr
  head/lang/ldc/pkg-plist
Comment 27 Mikael Urankar freebsd_committer freebsd_triage 2020-09-29 13:50:53 UTC
thanks!
Comment 28 commit-hook freebsd_committer freebsd_triage 2020-11-15 15:23:50 UTC
A commit references this bug:

Author: mikael
Date: Sun Nov 15 15:23:05 UTC 2020
New revision: 555229
URL: https://svnweb.freebsd.org/changeset/ports/555229

Log:
  lang/ldc: fix build on aarch64

  PR:		237427
  Submitted by:	Greg V
  Approved by:	portmgr (tier-2 blanket)

Changes:
  head/lang/ldc/files/patch-ltsmaster_driver_targetmachine.cpp
  head/lang/ldc/files/patch-ltsmaster_runtime_druntime_src_core_sys_posix_setjmp.d
  head/lang/ldc/files/patch-ltsmaster_runtime_druntime_src_core_sys_posix_ucontext.d
  head/lang/ldc/files/patch-ltsmaster_runtime_phobos_std_internal_math_gammafunction.d