Created attachment 196734 [details]
SHAR attached adds a preliminary variant devel/hadoop3 to the portstree.
What isn't done yet:
* porting over setsid/ssid patches from hadoop2
* rc.d scripts need a lot of work
* etc/ samples need a lot of work
* testing actual map/reduce through YARN & friends
* lots of polishing, testing
* testing that namenode + datanode can play nice together
* the package builds... to some degree
What would be a great idea before committing:
* renaming hadoop to hadoop1
* renaming hadoop3 to hadoop
* ensuring dependencies match
I am hoping to do a lot of these items within the next month(s) but would appreciate any help/patches/feedback given.
Two more things:
* the work/m2 directory needs to be hosted somewhere else, and DISTFILES :maven should adjusted
* once that's done, the maven call in do-build should have `--offline` appended
I wasn't aware that it's possible to create a shar file that does not
create the directories required ?
x - hadoop3/files/zkfc.in
/tmp/had: cannot create hadoop3/files/zkfc.in: No such file or directory
Hmm. Any idea ?
Ok, I can cope with mkdir in small numbers 8-)
testbuild@work after I try to silence portlint by butching the makefile 8-}
testbuild failed on 11.1a, see
Fails with the same problem on cur:
Port is still being worked on, ETA end of October.
Created attachment 203866 [details]
Hadoop 3.2 shell archive
Bump to Hadoop 3.2
- builds fine (for me, in poudriere testport)
- after adding configuration , namenode and datanode can be started
- I've carried over the sedsid->ssid patches from devel/hadoop2
- I've had to import and extend https://jira.apache.org/jira/browse/MAPREDUCE-6417
portlint has some comments. One of them is that patch filenames longer
than 100 characters are FATAL. I have no idea on how to handle this.
Build fails on 11.2amd64, but as it's a new port, I guess that can be tolerated.
Build fails on current-i386, but I guess that's also expected.
I'll ask portmgr about the overlong filenames.
(In reply to Johannes Jost Meixner from comment #7)
> - I've carried over the sedsid->ssid patches from devel/hadoop2
sysutils/ssid installs as setsid too since ports r482969, so this seems
unnecessary and some of the patches could be dropped.
Created attachment 203936 [details]
CRF on hadoop 3.2
- remove setsid/ssid patches
- fix @dir in pkg-plist / user:group info
- remove bogus .orig files from pkg-plist
Once I get a working arcanist setup on the buildbox I will close this PR and move everything to Phabricator. Might make reviewing / commenting things a lot easier - especially given that there are still a few renames and new ports forthcoming.
Thanks tobik@ and lwhsu@ for the feedback!
(In reply to Kurt Jaeger from comment #9)
I'm guessing the build failure is due to different libc versions.
I removed the USE_GCC line so it would build with Clang. Might have to add some
builds on current, test-build on 12 @work
Builds on 12.0, fails to build on 11.2:
Is it supposed to build on 11.2 or is that an expected failure ?
(In reply to Kurt Jaeger from comment #15)
So far I've really only tested it on 13.0-CURRENT, as that's what my buildbox runs.
On 12amd64 there's this plist error:
> ===> Checking for items in STAGEDIR missing from pkg-plist
> Error: Orphaned: @dir %%HADOOP_LOGDIR%%
> ===> Checking for items in pkg-plist which are not in STAGEDIR
>Error: Missing: @dir %%HADOOP_LOGDIR%%