Created attachment 240168 [details] patch introducing mlkit Adds mlkit, Standard ML compiler. Tested locally on poudriere. Builds and runs fine. Not sure everything in pkg-plist is as it should be though.
Thank you for your contribution! Here are some issues I found: Instead of adding the license file to PORTDOCS, set LICENSE_FILE= ${WRKSRC}/path/to/the/license. If there are multiple licenses, each license needs to be configured and its LICENSE_FILE set up. Please set DISTVERSION instead of PORTVERSION unless DISTVERSION doesn't work for you. Why is the port "ONLY_FOR_ARCH=amd64"? Please add ONLY_FOR_ARCH_REASON to explain the reason for this restriction. You having only tested the port for this architecture is not sufficient reason to restrict it, you need to know that it will not work on other architectures. Please include bsd.port.option.mk only if your makefile needs to evaluate option-specific macros. This doesn't seem the case with your makefile. Why did you set GH_TAGNAME to a git commit? The USE_GITHUB mechanism is capable of working with tag names. You should simply set DISTVERSIONPREFIX= v and leave out GH_TAGNAME. This will do the right thing. I also see that the Makefile is for the previous version. Please check if you can update it to the most recent version. I also noticed that the variables are in the wrong order. Please fix the order in accordance with the Porter's Handbook. The portclippy utility can help you with that. Are you sure CONFIGURE_SHELL is needed? Are you sure DISABLE_MAKE_JOBS is needed? Each installed file must only be listed once. Either in PORTDOCS/PLIST_FILES or in the pkg-plist file. Do not list files twice. Check if you can make the demo and test files conditional on an option. There are standard option names TEST and EXAMPLES for these. Please check these issues. I can proceed with a build test once you have addressed them.
Please also check if you can hook up TEST_TARGET or do-test (as appropriate) so it's easy to do a smoke test on your port. OPTIONS_DEFAULT can be omitted if empty. GH_PROJECT can be omitted if equal to PORTNAME. Does your port depend on a specific version of gcc or is any version fine? Would it also work with the base system compiler? This way, an extra dependency could be avoided.
Thanks for the feedback. mlkit doesn't seem to support anything else than amd64, that's why there's ONLY_FOR_ARCH=amd64 https://github.com/melsman/mlkit/issues/115#issuecomment-1326435741 CONFIGURE_SHELL is there, because mlkit uses patterns for install target that are not supported by /bin/sh and there's error: install: basis/*.{sml,sig,mlb}: No such file or directory DISABLE_MAKE_JOBS is needed, because mlkit_basislibs target depends on mlkit target. I get the following error without it: /usr/local/bin/bash: line 1: ../bin/mlkit: No such file or directory gmake[2]: *** [Makefile:58: mlkit_basislibs] Error 127 I don't think this requires any specific version of gcc. It does try to invoke gcc specifically though. I can try and see if cc works instead.
(In reply to serpent7776 from comment #3) Thank you for your response! > mlkit doesn't seem to support anything else than amd64, that's why there's ONLY_FOR_ARCH=amd64 https://github.com/melsman/mlkit/issues/115#issuecomment-1326435741 Cool! This is the kind of thing you should write in ONLY_FOR_ARCHS_REASON. > DISABLE_MAKE_JOBS is needed, because mlkit_basislibs target depends on mlkit target. I get the following error without it: You could try to patch the build scripts to make mlkit_basislibs depend on mlkit. But leaving in the DISABLE_MAKE_JOBS is fine as a stop gap measure. Make sure to report this problem up stream. > I don't think this requires any specific version of gcc. It does try to invoke gcc specifically though. I can try and see if cc works instead. Cool! Keep me posted on what you find. Ideally, gcc should be depended on by defining USE_GCC= yes and then configuring the port to use ${CC} and ${CXX} as the compilers (which USE_GCC will set to the gcc port selected).
Created attachment 242300 [details] patch v2 I've updated the patch. Hopefully it addresses the issues. Mlkit contains multiple license files. I hope I configured them correctly. I had to disable one of the tests, because it was giving me nondeterministic results. Also updated the mlkit to 4.7.3.
(In reply to serpent7776 from comment #5) Thank you for your updated patch. Here are some questions I have: - it's USE_GCC, not USE_gcc. I can fix that on commit. - The test target is not run by default and should not be made dependent on an option. It's purpose is to permit testing the port at build time. - Your port now no longer installs any test files, not even if the TEST option is set. So why is this option provided? Your original patch installed a bunch of test files, so I'm a bit confused as to what your intent is. If your intent was to install test files when "make test" is executed, please don't do that. Instead install them if the TEST option is provided, perhaps in a separate install target (such as do-install-TEST-on, which is run only if the TEST option is enabled). - please don't hard code a path to bash as the configure shell. I can patch this to just say CONFIGURE_SHELL= bash. Will proceed with a build test once I have a response to these questions.
Hi, Looks good overall, you don't need to wrap TEST_TARGET and as Robert mentioned if "make test" doesn't "need" any functionality from the framework you don't need to make it an option. While cosmetic you possibly want to push version data to the build, preferably so it's reproducable looking at https://github.com/melsman/mlkit/blob/master/configure.ac#L48 . These variables might also be used to generate version data in the binaries. In general we don't want to depend on git for building. While I guess it's preference USES= shebangfix uses full path so I think it's fine for consistency however it should use LOCALBASE and not PREFIX. Best regards, Daniel
(In reply to Daniel Engberg from comment #7) The port does not use shebangfix. It sets CONFIGURE_SHELL to run the configure script with bash. If the user has a bash binary installed from outside ${PREFIX}, the port makefile will not try to build a bash binary as the bash command is present but then cannot run the configure script as ${PREFIX}/bin/bash does not exist. Thus using a full path for CONFIGURE_SHELL is incorrect here unless we also use a full path for the BUILD_DEPENDS line. But then, why do it when it works just fine with just the command name?
Maybe the TEST option should be removed altogether. The intent was to just run tests provided by mlkit. I removed test related files from pkg-plist, because I don't believe they are necessary to be in the package. I made TEST_TARGET dependant on the option, because I got warning from portlint: WARN: Makefile: TEST is listed in OPTIONS_DEFINE, but no PORT_OPTIONS:MTEST appears. and I noticed it's done like that is some other port. I'm not sure what to do about the canfigure.ac file. Should I just patch it to hardcode some values instead of querying git/date commands?
(In reply to serpent7776 from comment #9) > Maybe the TEST option should be removed altogether. The intent was to just run > tests provided by mlkit. I removed test related files from pkg-plist, because I > don't believe they are necessary to be in the package. The correct way is to only add these files to the package if the TEST option is set. See § 8.1 Porter's Handbook for how to do that. You can at the same time hook up the test suite to the do-test target, but do it independently of the TEST option. > I'm not sure what to do about the canfigure.ac file. Should I just patch it to hardcode some values instead of querying git/date commands? Yes, that works. Bonus points if it includes PKGVERSION somewhere and that it was built by FreeBSD ports and who the maintainer is and that sort of stuff. You can use REINPLACE_CMD for such dynamic replacements.
(In reply to Robert Clausecker from comment #10) I'm not sure what you mean by 'hook up the test suite to the do-test target', do you mean something like this? do-test: @cd ${WRKSRC} && ${SETENV} make test Is it better than defining this? TEST_TARGET= test
(In reply to serpent7776 from comment #11) Setting TEST_TARGET=test is sufficient. The default do-test target basically runs cd ${WRKSRC} && ${MAKE} ${TEST_TARGET} (with some extra bells and whistles I've omitted for clarity).
Created attachment 242333 [details] patch v3 I updated the patch with the fixes.
Looks good! But oh boy, that thing wants 20 GB of RAM to compile. Not sure if our ports build cluster is configured to have that much per jail. Is it possible to make it use less RAM? I've also noticed that there are orphaned files. You can check for thesewith "make check-plist". One workaround is to remove these in post-install: lang/mlkit$ make check-plist ====> Checking for pkg-plist issues (check-plist) ===> Parsing plist ===> Checking for items in STAGEDIR missing from pkg-plist Error: Orphaned: lib/mlkit/basis/ARRAY.sig.log Error: Orphaned: lib/mlkit/basis/ARRAY2.sig.log Error: Orphaned: lib/mlkit/basis/ARRAY_SLICE.sml.log Error: Orphaned: lib/mlkit/basis/Array.sml.log Error: Orphaned: lib/mlkit/basis/Array2.sml.log Error: Orphaned: lib/mlkit/basis/ArraySlice.sml.log Error: Orphaned: lib/mlkit/basis/BIT_FLAGS.sml.log Error: Orphaned: lib/mlkit/basis/BOOL.sig.log Error: Orphaned: lib/mlkit/basis/BYTE.sig.log Error: Orphaned: lib/mlkit/basis/BitFlags.sml.log Error: Orphaned: lib/mlkit/basis/Bool.sml.log Error: Orphaned: lib/mlkit/basis/Byte.sml.log Error: Orphaned: lib/mlkit/basis/ByteSlice.sml.log Error: Orphaned: lib/mlkit/basis/ByteTable.sml.log Error: Orphaned: lib/mlkit/basis/CHAR.sig.log Error: Orphaned: lib/mlkit/basis/COMMAND_LINE.sml.log Error: Orphaned: lib/mlkit/basis/Char.sml.log Error: Orphaned: lib/mlkit/basis/CommandLine.sml.log Error: Orphaned: lib/mlkit/basis/DATE.sig.log Error: Orphaned: lib/mlkit/basis/Date.sml.log Error: Orphaned: lib/mlkit/basis/FileSys.sml.log Error: Orphaned: lib/mlkit/basis/GENERAL.sig.log Error: Orphaned: lib/mlkit/basis/General.sml.log Error: Orphaned: lib/mlkit/basis/INET_SOCK.sig.log Error: Orphaned: lib/mlkit/basis/INTEGER.sml.log Error: Orphaned: lib/mlkit/basis/INT_INF.sml.log Error: Orphaned: lib/mlkit/basis/INT_INF_REP.sml.log Error: Orphaned: lib/mlkit/basis/IO.sml.log Error: Orphaned: lib/mlkit/basis/Initial.sml.log Error: Orphaned: lib/mlkit/basis/Initial2.sml.log Error: Orphaned: lib/mlkit/basis/Int.sml.log Error: Orphaned: lib/mlkit/basis/Int31.sml.log Error: Orphaned: lib/mlkit/basis/Int32.sml.log Error: Orphaned: lib/mlkit/basis/Int63.sml.log Error: Orphaned: lib/mlkit/basis/Int64.sml.log Error: Orphaned: lib/mlkit/basis/IntInf.sml.log Error: Orphaned: lib/mlkit/basis/IntInfRep.sml.log Error: Orphaned: lib/mlkit/basis/LIST.sig.log Error: Orphaned: lib/mlkit/basis/LIST_PAIR.sml.log Error: Orphaned: lib/mlkit/basis/List.sml.log Error: Orphaned: lib/mlkit/basis/ListPair.sml.log Error: Orphaned: lib/mlkit/basis/MATH.sig.log Error: Orphaned: lib/mlkit/basis/MONO_ARRAY.sml.log Error: Orphaned: lib/mlkit/basis/MONO_ARRAY2.sml.log Error: Orphaned: lib/mlkit/basis/MONO_ARRAY_SLICE.sml.log Error: Orphaned: lib/mlkit/basis/MONO_VECTOR.sml.log Error: Orphaned: lib/mlkit/basis/MONO_VECTOR_SLICE.sml.log Error: Orphaned: lib/mlkit/basis/Math.sml.log Error: Orphaned: lib/mlkit/basis/NET_HOST_DB.sig.log Error: Orphaned: lib/mlkit/basis/NetHostDB.sml.log Error: Orphaned: lib/mlkit/basis/OPTION.sig.log Error: Orphaned: lib/mlkit/basis/OS.sml.log Error: Orphaned: lib/mlkit/basis/OSError.sml.log Error: Orphaned: lib/mlkit/basis/OS_FILE_SYS.sml.log Error: Orphaned: lib/mlkit/basis/OS_IO.sml.log Error: Orphaned: lib/mlkit/basis/OS_PATH.sml.log Error: Orphaned: lib/mlkit/basis/OS_PROCESS.sml.log Error: Orphaned: lib/mlkit/basis/Option.sml.log Error: Orphaned: lib/mlkit/basis/PACK_REAL.sml.log Error: Orphaned: lib/mlkit/basis/PACK_WORD.sml.log Error: Orphaned: lib/mlkit/basis/POSIX.sig.log Error: Orphaned: lib/mlkit/basis/POSIX_ERROR.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_FILE_SYS.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_IO.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_PROCENV.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_PROCESS.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_SIGNAL.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_SYS_DB.sml.log Error: Orphaned: lib/mlkit/basis/POSIX_TTY.sml.log Error: Orphaned: lib/mlkit/basis/Pack32Big.sml.log Error: Orphaned: lib/mlkit/basis/Pack32Little.sml.log Error: Orphaned: lib/mlkit/basis/PackRealBig.sml.log Error: Orphaned: lib/mlkit/basis/PackRealLittle.sml.log Error: Orphaned: lib/mlkit/basis/Path.sml.log Error: Orphaned: lib/mlkit/basis/Posix.sml.log Error: Orphaned: lib/mlkit/basis/Process.sml.log Error: Orphaned: lib/mlkit/basis/RANDOM.sig.log Error: Orphaned: lib/mlkit/basis/REAL.sig.log Error: Orphaned: lib/mlkit/basis/Random.sml.log Error: Orphaned: lib/mlkit/basis/Real.sml.log Error: Orphaned: lib/mlkit/basis/RealArray2.sml.log Error: Orphaned: lib/mlkit/basis/RealArrayVector.sml.log Error: Orphaned: lib/mlkit/basis/RealTable.sml.log Error: Orphaned: lib/mlkit/basis/SOCKET.sig.log Error: Orphaned: lib/mlkit/basis/STRING.sig.log Error: Orphaned: lib/mlkit/basis/STRING_CVT.sml.log Error: Orphaned: lib/mlkit/basis/STR_BASE.sml.log Error: Orphaned: lib/mlkit/basis/SUBSTRING.sig.log Error: Orphaned: lib/mlkit/basis/Socket.sml.log Error: Orphaned: lib/mlkit/basis/StrBase.sml.log Error: Orphaned: lib/mlkit/basis/String.sml.log Error: Orphaned: lib/mlkit/basis/StringCvt.sml.log Error: Orphaned: lib/mlkit/basis/Substring.sml.log Error: Orphaned: lib/mlkit/basis/TEXT.sig.log Error: Orphaned: lib/mlkit/basis/TIME.sig.log Error: Orphaned: lib/mlkit/basis/TIMER.sig.log Error: Orphaned: lib/mlkit/basis/TableSlice.sml.log Error: Orphaned: lib/mlkit/basis/Text.sml.log Error: Orphaned: lib/mlkit/basis/Time.sml.log Error: Orphaned: lib/mlkit/basis/Timer.sml.log Error: Orphaned: lib/mlkit/basis/UNIX.sig.log Error: Orphaned: lib/mlkit/basis/Unix.sml.log Error: Orphaned: lib/mlkit/basis/VECTOR.sig.log Error: Orphaned: lib/mlkit/basis/VECTOR_SLICE.sml.log Error: Orphaned: lib/mlkit/basis/Vector.sml.log Error: Orphaned: lib/mlkit/basis/VectorSlice.sml.log Error: Orphaned: lib/mlkit/basis/WORD.sig.log Error: Orphaned: lib/mlkit/basis/Word.sml.log Error: Orphaned: lib/mlkit/basis/Word16.sml.log Error: Orphaned: lib/mlkit/basis/Word31.sml.log Error: Orphaned: lib/mlkit/basis/Word32.sml.log Error: Orphaned: lib/mlkit/basis/Word63.sml.log Error: Orphaned: lib/mlkit/basis/Word64.sml.log Error: Orphaned: lib/mlkit/basis/Word8.sml.log Error: Orphaned: lib/mlkit/basis/WordN.sml.log Error: Orphaned: lib/mlkit/basis/io/bin-io.sig.log Error: Orphaned: lib/mlkit/basis/io/bin-io.sml.log Error: Orphaned: lib/mlkit/basis/io/bin-prim-io.sml.log Error: Orphaned: lib/mlkit/basis/io/bin-stream-io.sig.log Error: Orphaned: lib/mlkit/basis/io/cleaner.sig.log Error: Orphaned: lib/mlkit/basis/io/cleaner.sml.log Error: Orphaned: lib/mlkit/basis/io/imperative-io.sig.log Error: Orphaned: lib/mlkit/basis/io/imperative-io.sml.log Error: Orphaned: lib/mlkit/basis/io/prim-io.sig.log Error: Orphaned: lib/mlkit/basis/io/prim-io.sml.log Error: Orphaned: lib/mlkit/basis/io/stream-io.sig.log Error: Orphaned: lib/mlkit/basis/io/stream-io.sml.log Error: Orphaned: lib/mlkit/basis/io/text-io.sig.log Error: Orphaned: lib/mlkit/basis/io/text-io.sml.log Error: Orphaned: lib/mlkit/basis/io/text-prim-io.sml.log Error: Orphaned: lib/mlkit/basis/io/text-stream-io.sig.log Error: Orphaned: lib/mlkit/basis/wordtables.sml.log Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/CODEMIRROR-LICENSE Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/GPL-LICENSE Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/MLKit-LICENSE Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/MLton-LICENSE Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/SMLNJ-LIB-LICENSE Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/license/SMLNJ-LICENSE I can add on commit to fix this: post-install: ${FIND} ${STAGEDIR}${PREFIX}/lib/mlkit/basis -name \*.log -delete ${RM} -r ${STAGEDIR}${DOCSDIR}/license Is that okay with you? As for the test suite, it mostly passes, but there are some errors towards the end: gmake[2]: Entering directory '/usr/home/main.ports/lang/mlkit/work/mlkit-4.7.3/test/explicit_regions' ../../bin/kittester "../../bin/mlkit -no_gc -no_basislib --maximum_inline_size 0 -er -no_opt" --logdirect all.tst .. ERR: compile log er1.sml.log not equal to er1.sml.log.ok ... ERR: compile log er2.sml.log not equal to er2.sml.log.ok ... ERR: compile log er3.sml.log not equal to er3.sml.log.ok ... ERR: compile log er4.sml.log not equal to er4.sml.log.ok ... ERR: compile log er5.sml.log not equal to er5.sml.log.ok ... ERR: compile log er6.sml.log not equal to er6.sml.log.ok ... ERR: compile log er7.sml.log not equal to er7.sml.log.ok ... ERR: compile log er8.sml.log not equal to er8.sml.log.ok ... ERR: compile log er9.sml.log not equal to er9.sml.log.ok ... ERR: compile log er10.sml.log not equal to er10.sml.log.ok ... ERR: compile log er11.sml.log not equal to er11.sml.log.ok ... ERR: compile log er12.sml.log not equal to er12.sml.log.ok ... ERR: compile log er13.sml.log not equal to er13.sml.log.ok ... ERR: compile log er14.sml.log not equal to er14.sml.log.ok ... ERR: compile log er15.sml.log not equal to er15.sml.log.ok ... ERR: compile log er16.sml.log not equal to er16.sml.log.ok ... ERR: compile log er17.sml.log not equal to er17.sml.log.ok ... ERR: unexpected compile time failure for param.sml .. ERR: unexpected compile time failure for param1.sml .. ERR: unexpected compile time failure for tup.sml .. ERR: unexpected compile time failure for tup2.sml .. ERR: unexpected compile time failure for rec.sml .. ERR: unexpected compile time failure for string.sml .. ERR: unexpected compile time failure for ref.sml .. ERR: unexpected compile time failure for con0.sml .. ERR: unexpected compile time failure for con1.sml .. ERR: unexpected compile time failure for call.sml .. ERR: unexpected compile time failure for call2.sml [28/28] ***TEST FAILED: there were 28 errors. Please let me know if you would like for this to be committed as is or if you want to fix this up first.
Another warning: $ make stage-qa ====> Running Q/A tests (stage-qa) Warning: 'bin/mlkit' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STRIP_CMD} You can fix this one by adding post-install: ${STRIP_CMD} ${STAGEDIR}/bin/mlkit
mlkit uses mlton for compilation and it may use something like 0.7 of available system memory. On my 16GB machine it uses ~11GB. It probably can be adjusted if needed. I specifically excluded *.sml.log files, because I don't think they are needed. So post-install additions should be fine. I'm worried about the test failures. All tests are passing on my machine. Can I see some of the failed *.sml.log files? (e.g. er17.sml.log, call.sml.log, call2.sml.log)
The failing files all seem to have this content: $ cat er17.sml.log * Error: A library install directory must be provided in an environment variable SML_LIB or as a path-definition in either the system wide path-map /usr/local/etc/mlkit/mlb-path-map or in your personal path-map ~/.mlkit/mlb-path-map.. * Exiting!
We usually build ports by building X ports at once where each port is built with one thread. Thus your port trying to use a fraction of total system memory during build is suboptimal. If possible, please supply a patch to limit the memory use to some reasonable amount (e.g. 4 GB).
Created attachment 242410 [details] patch v4 I updated the patch. This should limit heap usage to 4GB. It also fixes the tests issues.
Looks good! No more unit test failures! Will commit with minor formatting changes with my next batch. Thank you for taking your time to bring this to perfection.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=ebc62e34c69afc2a45e052599af60def8cb192ba commit ebc62e34c69afc2a45e052599af60def8cb192ba Author: Serpent7776 <serpent7776@gmail.com> AuthorDate: 2023-02-09 18:31:43 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-05-29 23:24:04 +0000 lang/mlkit: Standard ML Compiler The MLKit is a compiler toolkit for the Standard ML language, including The MLKit with Regions, which features a native backend for the x64 architecture, based on region inference, and SMLtoJs, which features a JavaScript backend targeting web browsers. The two compilers share the same frontend and compilation management scheme. The MLKit covers all of Standard ML, as defined in the 1997 edition of The Definition of Standard ML and supports most of the Standard ML Basis Library. WWW: https://elsman.com/mlkit/ PR: 269566 lang/Makefile | 1 + lang/mlkit/Makefile (new) | 53 + lang/mlkit/distinfo (new) | 3 + lang/mlkit/files/patch-Makefile.in (new) | 23 + lang/mlkit/files/patch-Makefiledefault (new) | 11 + lang/mlkit/files/patch-src_Runtime_Spawn.c (new) | 24 + .../files/patch-src_Runtime_gen__syserror.c (new) | 27 + lang/mlkit/files/patch-test_all.tst (new) | 11 + lang/mlkit/files/patch-test_posix.sml (new) | 10 + lang/mlkit/pkg-descr (new) | 9 + lang/mlkit/pkg-plist (new) | 3579 ++++++++++++++++++++ 11 files changed, 3751 insertions(+)
Thank you for your contribution.
Yay, thank you for the guidance.