Bug 237213 - [NEW PORT] devel/mingw-w64 cross compilers
Summary: [NEW PORT] devel/mingw-w64 cross compilers
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2019-04-12 04:24 UTC by Damjan Jovanovic
Modified: 2021-07-12 22:12 UTC (History)
12 users (show)

See Also:


Attachments
MinGW-w64 for Windows cross-development using LLVM/Clang (469.84 KB, patch)
2020-03-01 08:18 UTC, Theron Tarigo
no flags Details | Diff
partial port of i686 mingw-w64 (210.60 KB, patch)
2020-11-22 09:45 UTC, Damjan Jovanovic
no flags Details | Diff
complete mingw-w64 port (415.76 KB, patch)
2020-11-26 15:14 UTC, Damjan Jovanovic
no flags Details | Diff
complete mingw-w64 port, version 2 (129.32 KB, patch)
2020-12-17 14:19 UTC, Damjan Jovanovic
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Damjan Jovanovic 2019-04-12 04:24:31 UTC
Hi

Can someone please make a port for the mingw-w64 project that cross-compiles Windows binaries from *nix?

Note that this is NOT THE SAME software project as the now largely abandoned mingw project; mingw-w64 was made by different developers with a different setup, and (despite the name) can build both Win32 and Win64 binaries unlike the original mingw project that only supported Win32.

I need this to develop Wine-Mono on FreeBSD, and it is generally useful in all cases where mingw was useful. Please provide both the Win32 and Win64 ports.

Links:
https://mingw-w64.org/doku.php/download
https://sourceforge.net/p/mingw-w64/wiki2/Cross%20Win32%20and%20Win64%20compiler/

Thank you so much!
Comment 1 Naram Qashat 2019-09-06 02:13:04 UTC
I've been wanting to do this on-and-off for probably a few years now, but the documentation for it, which you linked, is pretty lackluster and a bit confusing at points. I've not had the time to really dive into it much either.
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-16 11:55:01 UTC
@Naram / Akinori-san, is this something either one of you can take care of?

Normally, Bugzilla is not used to take new port requests, in favour of creating entries in the wiki [1]

[1] https://wiki.freebsd.org/WantedPorts

If we can get some buy-in to create these ports, I'm happy to keep the issue open.

Separately there is also the question of maintainer/maintenance. Our mingw* ports aren't in the best shape with regard to maintenance (I may be wrong with regard to versions being up to date however)
Comment 3 Naram Qashat 2019-09-16 12:58:37 UTC
You're right about the current mingw ports not being in the best shape, I haven't devoted time to them as I feel that the current mingw ports are outdated and stale. Maybe it's just my opinion on them. I'd love to have the mingw-w64 project in the ports tree instead of what we currently have, but as I mentioned previously, I have not devoted any time to looking into how to get those into the ports tree either.
Comment 4 Alex S 2019-11-10 08:20:47 UTC
Might be useful for Proton as well.

(In reply to Kubilay Kocak from comment #2)

> If we can get some buy-in to create these ports
That doesn't sound encouraging. Who's permission we should ask first? I'd hate to put 100+ hours into something only to see it stalled at review/merge level.
Comment 5 Alex S 2019-11-10 08:27:50 UTC
(In reply to Alex S from comment #4)

* whose

(f* Bugzilla and it's lack of edit function)
Comment 6 Theron Tarigo 2020-02-29 01:40:05 UTC
(In reply to Alex S from comment #4)
Uhoh.  Why 100+ hours for this?  Why should this get bogged in review (I don't see anything likely controversial here)?

> If we can get some buy-in to create these ports

I think all this means is someone needs to volunteer to do the work.  I'm looking into it a bit right now.

My experience with not getting things accepted is failing to make enough noise to get committers' attention (or some would say, failing to become a committer myself).
Comment 7 Alex S 2020-02-29 10:35:48 UTC
(In reply to Theron Tarigo from comment #6)

See https://aur.archlinux.org/packages/mingw-w64-gcc and https://github.com/ValveSoftware/Proton/blob/proton_5.0/build-mingw-w64.sh for reference.

You can also look at my half-assed attempt at building mingw-w64: https://gist.github.com/shkhln/bfefeb1196707559a5de0e1290edc37a. Although I'm not sure if that's actually useful.

> Why 100+ hours for this?

Getting reasonably familiar with the gcc building process, waiting for compilation to finish (multiple times, of course), debugging compilation errors (such as undefined reference to `___chkstk_ms' for example).
Comment 8 Theron Tarigo 2020-03-01 08:18:34 UTC
Created attachment 212067 [details]
MinGW-w64 for Windows cross-development using LLVM/Clang

This ended up being not difficult, albeit using LLVM toolchain, thanks to https://github.com/mstorsjo/llvm-mingw .

Of course getting everything done, compiling a working Windows 64bit exe, and seemingly ready to upload the patch only took half the time, other half has been wasted tracking down ridiculous behaviors whereby Clang tries to use GCC-4.x junk instead of its own libraries if the ancient mingw32-* packages are left installed... At least the fix ended up being simple.

When will these developers learn that adding automated workaround disasters and not documenting them prominently wastes more time than simple errors in the edge-cases they are meant to "fix"?

Needs commit or review (I'll put it in the mailing list if no one sees this).
Comment 9 Alex S 2020-03-01 08:52:52 UTC
Oh, I wasn't aware of that project. Clang is much easier to deal with. Good job, I suppose.
Comment 10 Theron Tarigo 2020-03-01 09:08:12 UTC
(In reply to Alex S from comment #9)
We shall see.  As noted in that project's README, existing MinGW projects with portability shortcomings will have problems with the usual GNU vs LLVM toolchain differences.

That won't affect me since I start my projects on FreeBSD and then port to other platforms, but someone else could look into supporting more GNU tools without necessarily switching to GCC.
Comment 11 Damjan Jovanovic 2020-08-15 03:15:19 UTC
I really needed GCC-based mingw for building Wine and its unit tests, and found a way to get Debian Buster's mingw-w64 version 8.3 working using our Linuxulator and debootstrap:

https://forums.freebsd.org/threads/mingw-w64-from-linuxulated-debian-buster-building-wine-pe.76580/
Comment 12 Damjan Jovanovic 2020-11-22 09:45:36 UTC
Created attachment 219875 [details]
partial port of i686 mingw-w64

So that Debian emulated mingw-w64 crashes a lot and I got sick of it.

Here is a partial port from NetBSD's pkg-src's mingw-w64 (https://github.com/NetBSD/pkgsrc/tree/trunk/cross/mingw-w64) to our ports. So far binutils, headers, gcc-bootstrap and crt successfully build. Unfortunately, the last and most important piece, the full gcc build, fails with:

ld: error: unable to find library -lc

while building libgcc, and I have no idea why. If anyone can help, it would be much appreciated.

Apply patch.
cd /usr/ports/devel/i686-w64-mingw32-gcc
make stage
Comment 13 Alex S 2020-11-22 10:52:44 UTC
(In reply to Damjan Jovanovic from comment #12)

Try USE_GCC=10.
Comment 14 Alex S 2020-11-22 11:33:44 UTC
> #		--enable-initfini-array \
> CONFIGURE_ARGS+=--target=${GCC_TARGET} --disable-nls --enable-languages=c,lto,c++,objc,obj-c++,fortran \

This isn't nice either.
Comment 15 Alex S 2020-11-22 14:47:47 UTC
(In reply to Alex S from comment #14)

I think i686-w64-mingw32-crt needs SSP_UNSAFE=yes.

> PKGNAMEPREFIX=	i686-w64-ming32-

Another typo.
Comment 16 Damjan Jovanovic 2020-11-23 02:01:11 UTC
Thank you Alex.

USE_GCC=10 and SSP_UNSAFE=yes give different results, but all break in different ways:

gcc-     |              |                               |
bootstrap|crt           |gcc                            |RESULT
---------+--------------+-------------------------------+----------------------
clang    |default       |clang with                     |some __stack_check
         |              |--enable-gnu-indirect-function |failure
         |              |--enable-initfini-array        |
---------+--------------+-------------------------------+----------------------
clang    |default       |clang                          |ld: error: unable to
         |              |                               | find library -lc
---------+--------------+-------------------------------+----------------------
clang    |default       |gcc                            |C++ errors
---------+--------------+-------------------------------+----------------------
clang    |USE_GCC=10 and|                               |CRT fails to link
         |SSP_UNSAFE=yes|                               |
---------+--------------+-------------------------------+----------------------
clang    |default and   |gcc                            |C++ errors
         |SSP_UNSAFE=yes|                               |
---------+--------------+-------------------------------+----------------------
gcc      |default and   |gcc                            |C++ errors
         |SSP_UNSAFE=yes|                               |
---------+--------------+-------------------------------+----------------------

Those C++ errors happen while building gcov, and look like this:

/usr/local/bin/ld: gcov.o: in function `void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) [clone .isra.0]':
gcov.c:(.text+0xf4a): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/local/bin/ld: gcov.c:(.text+0xf76): undefined reference to `std::__throw_logic_error(char const*)'
/usr/local/bin/ld: gcov.o: in function `output_line_beginning(__sFILE*, bool, bool, bool, long, unsigned int, char const*, char const*, unsigned int)':
gcov.c:(.text+0x233e): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)'
/usr/local/bin/ld: gcov.c:(.text+0x241c): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long)'

and many more.
Comment 17 Alex S 2020-11-23 02:59:46 UTC
(In reply to Damjan Jovanovic from comment #16)

Let me try to explain comment #14 again:

# ... \            <-- commented out line
CONFIGURE_ARGS=... <-- continuation of the same line, CONFIGURE_ARGS is also commented out
Comment 18 Damjan Jovanovic 2020-11-23 06:35:27 UTC
It builds successfully when I fix that line, even using Clang for all the parts, and compiles executables that work in Wine.

Thank you so so so much Alex!

      .ssSSSSss.
    .ER'      `AM.
  .ST'          `CS.
 .E'  .S.    .S.  `S.
.L'   SSS    SSS   `S.
S'    `S'    `S'    `S
S                    S
S                    S
S.  s.          .s   S
`S. `"s.      .s"'  S'
 `S.  `"ss..ss"'  .S'
  `SS.    ~~    .SS'
    `SS.      .SS'
      `SSssssSS'

Let me do some cleanup, testing, tweaking, and the ports targeting Win64, then I'll submit a patch for inclusion in Ports.
Comment 19 Damjan Jovanovic 2020-11-26 15:14:26 UTC
Created attachment 220001 [details]
complete mingw-w64 port

Here is my complete mingw-w64 Port, which I am submitting with me as the maintainer.

You can install devel/mingw-w64-gcc, with FLAVOR=i686 to target Win32 or FLAVOR=amd64 to target Win64. The GCC binaries are named i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc, like on other *nix platforms.

binutils is patched to expose its i686-w64-mingw32 and x86_64-w64-mingw32 targets, which it already had but we never used. mingw-w64 provides the headers and CRT, while GCC 10 provides the bootstrap GCC (to build the CRT) and full GCC. All 5 Ports build and work on both amd64 and i386 hosts.

The PE build of the whole of i386-wine (5 million lines of code) was successfully compiled with the i686-mingw-w64-gcc port, and worked fully with all the Windows applications I tested.

*******
WARNING
*******
Other Ports, whose software auto-detects mingw-w64 and automatically uses it, will break with this Port installed, as they may start producing files unexpected by their pkg-plist.

Eg. Wine, since version 5.0, preferentially makes PE builds with mingw-w64 (instead of ELF builds with native CC), and will build successfully but probably fail to package/install as its pkg-plist expects .dll.so files while it produces .dll with mingw-w64 instead. You can (1) force Wine to ignore mingw-w64 with "./configure --without-mingw", or better yet (2) update the Wine ports to use different pkg-plist files depending on whether mingw-w64 is used. (In my test, I built Wine directly from source, not the Port.)
Comment 20 Gerald Pfeifer freebsd_committer 2020-11-26 23:12:33 UTC
As a minor detail, new ports don't carry PORTREVISION.

Is there a specific reason you don't autogenerate pkg-plist like the
lang/gcc* ports do?

Wouldn't it be easier to have one pkg-plist, not one per architecture?
Substitutions might help with that, and if there is a specific issue
you ran into, perhaps I (or others) have an idea?

More generally, how hard would it be to use the existing lang/gcc10
port, like you have been doing with binutils?


(Note: I simply don't have the bandwidth to do a final review or commit,
merely trying to help - and hoping someone will step up for the other 
aspects.)
Comment 21 Damjan Jovanovic 2020-11-27 05:50:29 UTC
(In reply to Gerald Pfeifer from comment #20)

> As a minor detail, new ports don't carry PORTREVISION.

Thank you, fixed.

> Is there a specific reason you don't autogenerate pkg-plist like the
> lang/gcc* ports do?

Because I learned from the bad pkg-plist practices in devel/binutils instead.

> More generally, how hard would it be to use the existing lang/gcc10
> port, like you have been doing with binutils?

It looked to me like lang/* are for host-targeting compilers, while devel/* are for cross-compilers (devel/powerpc64-gcc, devel/gcc-arm-embedded, etc.). If that's not the case, then maybe lang/gcc10 should be used as the GCC for mingw-w64.
Comment 22 commit-hook freebsd_committer 2020-11-29 11:31:53 UTC
A commit references this bug:

Author: gerald
Date: Sun Nov 29 11:31:30 UTC 2020
New revision: 556564
URL: https://svnweb.freebsd.org/changeset/ports/556564

Log:
  Wine is now able to use mingw-w64 to build components. When mingw-w64
  is installed on FreeBSD, users might inadvertedly use that when we have
  not set up things properly yet (and it probably should be an option to
  choose, at least initially). For example, mingw-w64 produces .dll files
  instead of the current .dll.so files, breaking pkg-plist.

  So for now explicitly disable the use of mingw-w64. In a next step,
  once mingw-w64 is available, we probably should make this an option.

  PR:		237213
  Reported by:	Damjan Jovanovic <damjan.jov@gmail.com>
  MFH:		2020Q4 (blanket: build issue)

Changes:
  head/emulators/wine-devel/Makefile
Comment 23 commit-hook freebsd_committer 2020-12-11 20:38:59 UTC
A commit references this bug:

Author: gerald
Date: Fri Dec 11 20:38:16 UTC 2020
New revision: 557762
URL: https://svnweb.freebsd.org/changeset/ports/557762

Log:
  MFH: r556564

  Wine is now able to use mingw-w64 to build components. When mingw-w64
  is installed on FreeBSD, users might inadvertedly use that when we have
  not set up things properly yet (and it probably should be an option to
  choose, at least initially). For example, mingw-w64 produces .dll files
  instead of the current .dll.so files, breaking pkg-plist.

  So for now explicitly disable the use of mingw-w64. In a next step,
  once mingw-w64 is available, we probably should make this an option.

  PR:		237213
  Reported by:	Damjan Jovanovic <damjan.jov@gmail.com>

Changes:
_U  branches/2020Q4/
  branches/2020Q4/emulators/wine-devel/Makefile
Comment 24 Damjan Jovanovic 2020-12-17 14:19:11 UTC
Created attachment 220663 [details]
complete mingw-w64 port, version 2

Here is version 2 of the patch adding mingw-w64 ports.

PORTREVISION has been removed. pkg-plists have been merged or removed. GCC now comes from a FLAVOR of lang/gcc10, so you can install everything through eg. "FLAVOR=i686_w64_mingw32 make install" in lang/gcc10.
Comment 25 commit-hook freebsd_committer 2020-12-25 12:16:59 UTC
A commit references this bug:

Author: gerald
Date: Fri Dec 25 12:16:27 UTC 2020
New revision: 559237
URL: https://svnweb.freebsd.org/changeset/ports/559237

Log:
  Back port r556564 | gerald | 2020-11-29 from emulators/wine-devel:

    Wine is now able to use mingw-w64 to build components. When mingw-w64
    is installed on FreeBSD, users might inadvertedly use that when we have
    not set up things properly yet (and it probably should be an option to
    choose, at least initially). For example, mingw-w64 produces .dll files
    instead of the current .dll.so files, breaking pkg-plist.

    So for now explicitly disable the use of mingw-w64. In a next step,
    once mingw-w64 is available, we probably should make this an option.

  PR:		237213
  Reported by:	Damjan Jovanovic <damjan.jov@gmail.com>
  MFH:		2020Q4 (blanket: build issue)

Changes:
  head/emulators/wine/Makefile
Comment 26 Alex S 2020-12-26 12:00:15 UTC
I would be much better to throw WIP patches at https://reviews.freebsd.org.

(In reply to Damjan Jovanovic from comment #24)

I had to do the following adjustments to compile gcc:

@@ -32,7 +32,7 @@
 BUILD_DEPENDS+=	${LOCALBASE}/bin/as:devel/binutils
 RUN_DEPENDS+=	${LOCALBASE}/bin/as:devel/binutils
 .else
-BUILD_RUN_DEPS=	${FLAVOR:C/_/-/g}-as:devel/binutils@${FLAVOR} \
+BUILD_RUN_DEPS=	${TARGETARCH}-as:devel/binutils@${FLAVOR} \
 		${LOCALBASE}/${TARGETARCH}/mingw/include/windows.h:devel/mingw-w64-headers@${FLAVOR:S/_w64_mingw32//} \
 		${LOCALBASE}/${TARGETARCH}/mingw/lib/libkernel32.a:devel/mingw-w64-crt@${FLAVOR:S/_w64_mingw32//}
 BUILD_DEPENDS=	${BUILD_RUN_DEPS} \
@@ -54,8 +54,7 @@
 GCC_VERSION=	${PORTVERSION}
 SUFFIX=		${PORTVERSION:C/([0-9]+).*/\1/}
 .if ${FLAVOR} != native
-TARGETARCHAMD64=${FLAVOR:S/amd64/x86_64/}
-TARGETARCH=	${TARGETARCHAMD64:C/_/-/g}
+TARGETARCH=	${FLAVOR:C/_/-/g:S/amd64/x86_64/}
 .endif
 SSP_UNSAFE=	yes
 CFLAGS:=	${CFLAGS:N-mretpoline}
@@ -202,7 +201,7 @@
 	${RLN} ${STAGEDIR}${INSTALL_BASE}/bin/${FLAVOR:C/_/-/g}-$F${SUFFIX} \
 		${STAGEDIR}${PREFIX}/bin/${FLAVOR:C/_/-/g}-$F${SUFFIX}
 .endfor
-	${RLN} ${STAGEDIR}${INSTALL_BASE}/bin/${FLAVOR:C/_/-/g}-gcc-${PORTVERSION} \
+	${RLN} ${STAGEDIR}${INSTALL_BASE}/bin/*-gcc-${PORTVERSION} \
 		${STAGEDIR}${PREFIX}/bin/${FLAVOR:C/_/-/g}-gcc-${PORTVERSION}
 	cd ${STAGEDIR}${PREFIX} ; if [ -d ${TARGEARCH}/bin ]; then \
 	    ${FIND} ${TARGETARCH}/bin -type f -o -type l >>${WRKDIR}/PLIST.lib ;\

Different prefixes for gcc and binutils (amd64-w64-mingw32- and x86_64-w64-mingw32 respectively) seems to confuse Wine's build scripts, so something should be done about that as well.
Comment 27 Alex S 2020-12-26 12:05:56 UTC
(In reply to Alex S from comment #26)

* It