Created attachment 196734 [details] hadoop3 shar SHAR attached adds a preliminary variant devel/hadoop3 to the portstree. What isn't done yet: * porting over setsid/ssid patches from hadoop2 * rc.d scripts need a lot of work * etc/ samples need a lot of work * testing actual map/reduce through YARN & friends * lots of polishing, testing What's done: * testing that namenode + datanode can play nice together * the package builds... to some degree What would be a great idea before committing: * renaming hadoop to hadoop1 * renaming hadoop3 to hadoop * ensuring dependencies match I am hoping to do a lot of these items within the next month(s) but would appreciate any help/patches/feedback given.
Two more things: * the work/m2 directory needs to be hosted somewhere else, and DISTFILES :maven should adjusted * once that's done, the maven call in do-build should have `--offline` appended
I wasn't aware that it's possible to create a shar file that does not create the directories required ? [...] x - hadoop3/files/zkfc.in /tmp/had: cannot create hadoop3/files/zkfc.in: No such file or directory Hmm. Any idea ?
Ok, I can cope with mkdir in small numbers 8-) testbuild@work after I try to silence portlint by butching the makefile 8-}
testbuild failed on 11.1a, see http://people.freebsd.org/~pi/logs/devel__hadoop3-111.txt
Fails with the same problem on cur: http://people.freebsd.org/~pi/logs/devel__hadoop3-cur.txt
Port is still being worked on, ETA end of October.
Created attachment 203866 [details] Hadoop 3.2 shell archive Bump to Hadoop 3.2 - builds fine (for me, in poudriere testport) - after adding configuration [1], namenode and datanode can be started - I've carried over the sedsid->ssid patches from devel/hadoop2 - I've had to import and extend https://jira.apache.org/jira/browse/MAPREDUCE-6417 [1] https://hadoop.apache.org/docs/r3.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html
portlint has some comments. One of them is that patch filenames longer than 100 characters are FATAL. I have no idea on how to handle this. testbuilds@work.
Build fails on 11.2amd64, but as it's a new port, I guess that can be tolerated. https://people.freebsd.org/~pi/logs/devel__hadoop3-112-1555997211.txt Build fails on current-i386, but I guess that's also expected. I'll ask portmgr about the overlong filenames.
(In reply to Johannes Jost Meixner from comment #7) > - I've carried over the sedsid->ssid patches from devel/hadoop2 sysutils/ssid installs as setsid too since ports r482969, so this seems unnecessary and some of the patches could be dropped.
Created attachment 203936 [details] CRF on hadoop 3.2 - remove setsid/ssid patches - fix @dir in pkg-plist / user:group info - remove bogus .orig files from pkg-plist Once I get a working arcanist setup on the buildbox I will close this PR and move everything to Phabricator. Might make reviewing / commenting things a lot easier - especially given that there are still a few renames and new ports forthcoming. Thanks tobik@ and lwhsu@ for the feedback!
(In reply to Kurt Jaeger from comment #9) I'm guessing the build failure is due to different libc versions. I removed the USE_GCC line so it would build with Clang. Might have to add some USES+= compiler magic!
testbuilds@work
builds on current, test-build on 12 @work
Builds on 12.0, fails to build on 11.2: https://people.freebsd.org/~pi/logs/devel__hadoop3-112.txt Is it supposed to build on 11.2 or is that an expected failure ?
(In reply to Kurt Jaeger from comment #15) So far I've really only tested it on 13.0-CURRENT, as that's what my buildbox runs.
On 12amd64 there's this plist error: > ===> Checking for items in STAGEDIR missing from pkg-plist > Error: Orphaned: @dir %%HADOOP_LOGDIR%% > ===> Checking for items in pkg-plist which are not in STAGEDIR >Error: Missing: @dir %%HADOOP_LOGDIR%%