Bug 247352 - devel/emscripten Introduce emscripten-devel port for bundled llvm and binaryen
Summary: devel/emscripten Introduce emscripten-devel port for bundled llvm and binaryen
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Yuri Victorovich
Depends on:
Reported: 2020-06-17 18:59 UTC by Karsten Pedersen
Modified: 2020-07-09 09:39 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (yuri)

An initial port diff for emscripten-devel (649.01 KB, patch)
2020-06-17 18:59 UTC, Karsten Pedersen
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Karsten Pedersen 2020-06-17 18:59:37 UTC
Created attachment 215667 [details]
An initial port diff for emscripten-devel


Please find attached a port Makefile for emscripten-devel.

This is very similar to the original emscripten port (written by yuri@freebsd.org) but it builds and bundles in the latest llvm (with just Host;WebAssembly targets).

I feel this would be useful because emscripten is very dependent on the latest upstream llvm and if we can separate this from our version which may need to lag behind due to architecture related reasons this port will remain unaffected.

You can see in the port it uses the very latest versions from github (as revision hashes). I recently had my minor change pulled into the emscripten repo recently so it might be one or two revisions out by now.

This port also handles the generation of ~/.emscripten a little differently. Normally the first time the compiler is run it generates this file and exits, causing issues with our build server. This generates the file and then carries on with the compilation.

This port also avoids hard coding paths in the .emscripten file making it easier to use multiple versions. I added a CONFLICTS to the current emscripten but with a few tweaks we could make both run side by side.

Finally my future goal is to create an emscripten-ports-devel port which bundles up the few (9?) ports for various emscripten libraries and adds them to ${EMSCRIPTEN_PREFIX}/system/local. The current solution downloads them as needed which I find a little bit too "web dev" for my liking. I prefer deterministic builds. This initial work can be seen here: https://github.com/osen/emscripten-devel

Many thanks,

Comment 1 Yuri Victorovich freebsd_committer 2020-07-09 00:53:47 UTC
Hi Karsten,

Thanks for the new port.

The port builds in a very long time because it doesn't parallelize the llvm build.

I am also not convinced that it is necessary to rebuild llvm just for this. The devel/emscripten port intentionally links to the latest devel/llvm-devel which is supposed to be the very latest version of llvm. The effort of bundling an even newer version repeats the effort of updating devel/llvm-devel: why not just update devel/llvm-devel instead?

Also if there are particular improvements that you have in mind, such improvements should be in both devel/emscripten and devel/emscripten-devel to simplify the maintenance of the ports.

Comment 2 Karsten Pedersen 2020-07-09 09:39:31 UTC
Hi Yuri,

Thanks for getting back to me. OK, yes I can see your reasoning behind not bundling in an LLVM. I can also see both the emscripten port and llvm-devel are both pretty current and maintained so it looks not to be necessary.

I will target my "emscripten-ports" port against your Emscripten port. Unless perhaps you would consider including these few additional parts in with your port?


Basically all I aim to be doing is compile them using the emcmake wrapper and installing them into the "emscripten/system/local" directory. My concern about Emscripten is it is very "webdev-like", meaning they like to depend on dragging in dependencies from the internet (via emsdk or emcc flags) and I personally prefer something a little bit more deterministic. Certainly there are too few ports to warrant a separate FreeBSD port for each one.

Speaking of which, this comment might not be the place for it but for the -O3 optimisation level to work in the compiler we need a bunch of nodejs modules to be downloaded. At the moment this does not work in the port (because the directory is read-only at this point). It used to be a Java thing but now it is some optimiser node.js Javascript module.

Basically we need to fetch all the "stuff" in from here: https://github.com/emscripten-core/emscripten/blob/master/package.json as part of the build process.

I propose adding

$ npm install --ignore-scripts

Somewhere in the port makefile, perhaps under the build step. It would be great if we could do it at the distfile fetching stage but I have yet to find out a way. I tried downloading the package.json file first, executing the npm install and zip it up but it was fairly naff.

Many thanks,