Bug 267982 - New port: sysutils/hammer2 for HAMMER2 file system (read-only support)
Summary: New port: sysutils/hammer2 for HAMMER2 file system (read-only support)
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Koichiro Iwao
URL: https://www.freshports.org/sysutils/h...
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-25 08:12 UTC by kusumi.tomohiro
Modified: 2022-12-12 03:04 UTC (History)
4 users (show)

See Also:
grahamperrin: merge-quarterly-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description kusumi.tomohiro 2022-11-25 08:12:55 UTC
HAMMER2 file system for FreeBSD (read-only support).
https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/sys/vfs/hammer2/DESIGN

Consists of hammer2.ko kernel module, sbin/* binaries and man pages.
WWW: https://github.com/kusumi/freebsd_hammer2

Patch:
- https://www.dragonflybsd.org/~tkusumi/diff/freebsd/hammer2/ports/hammer2.shar.txt
- https://www.dragonflybsd.org/~tkusumi/diff/freebsd/hammer2/ports/0001-sysutils-hammer2-New-port.patch

Notes:
- Also see discussions in https://reviews.freebsd.org/D37354/new/ .
- Makefile is based on what's suggested in https://wiki.freebsd.org/Ports/SimpleGithub .
- The upstream is myself, and MAINTAINER is temporarily set to my NetBSD address.
- Some ports split kernel and user into a separate port (e.g. -kmod and -utils), but since upstream contains both in a single repository, I put everything in sysutils/hammer2.
- Compiles and tested on amd64 and i386. The code isn't arch dependent.
Comment 1 kusumi.tomohiro 2022-11-25 08:19:14 UTC
Also available on GitHub.
https://github.com/kusumi/freebsd-ports/commits/hammer2_1
Comment 2 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-25 09:47:49 UTC
Failed to build to me. Does it have some special requirements? 


--- hammer2_ondisk.o ---
hammer2_ondisk.c:69:59: error: too few arguments provided to function-like macro invocation
        NDINIT(ndp, LOOKUP, FOLLOW | LOCKLEAF, UIO_SYSSPACE, path);
                                                                 ^
/usr/src/sys/sys/namei.h:221:9: note: macro 'NDINIT' defined here
#define NDINIT(ndp, op, flags, segflg, namep, td)                       \
        ^
hammer2_ondisk.c:69:2: error: use of undeclared identifier 'NDINIT'
        NDINIT(ndp, LOOKUP, FOLLOW | LOCKLEAF, UIO_SYSSPACE, path);
        ^
2 errors generated.
*** [hammer2_ondisk.o] Error code 1
Comment 3 kusumi.tomohiro 2022-11-25 10:17:06 UTC
Looks like 7e1d3eefd410ca0fbae5a217422821244c3eeee4 from "Thu Nov 25 21:43:06 2021" removed an argument from NDINIT() and friends.

I'm using -CURRENT src and so does build tests in https://github.com/freebsd/freebsd-src/pull/627 .

Then the question is what's the expectation for a kernel code ports in this case ?
I could probably support both kAPI using ifdefs on kernel version or something.
Comment 4 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-25 12:52:15 UTC
(In reply to kusumi.tomohiro from comment #3)

I performed a build test on 13-STABLE (__FreeBSD_version == 1301509). Actually, I succeeded to build it with 1400072.

See also: https://docs.freebsd.org/en/books/porters-handbook/versions/

Generally, ports need to be able to be built on all supported versions (and CURRENT). Can you update upstream code to support 12, 13 and CURRENT?

If not possible, mark as IGNORE on the specific version using ${OSREL}.

See www/chromium/Makefile for example.

.if ${OSREL} == "12.3"
IGNORE=         does not compile, libc++ too old
.endif
Comment 5 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-25 12:58:26 UTC
Using ${OSVERSION} would be better. See net/onedrive/Makefile.
Not fixing in upstream but in local patch in ports tree is also fine.
Comment 6 kusumi.tomohiro 2022-11-25 22:38:52 UTC
Updated sysutils/hammer2 to use upstream v1.0.1 tag.

It now compiles with both 6 arguments (last one being unused) and 5 arguments version of NDINIT().
No local patches as the fix is in upstream.

Upstream:
https://github.com/kusumi/freebsd_hammer2/releases/tag/v1.0.1

Patch:
https://www.dragonflybsd.org/~tkusumi/diff/freebsd/hammer2/ports/v1.0.1/

GitHub branch:
https://github.com/kusumi/freebsd-ports/commits/hammer2_2
Comment 7 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-26 04:38:12 UTC
1.0.1 test build succeeded on CURRENT and 13-STABLE but failed on 12-STABLE.
Can you fix it? If difficult, just let me know. 

--- mount_hammer2.o ---
mount_hammer2.c:54:2: error: use of undeclared identifier 'MNT_NOCOVER'
        MOPT_STDOPTS,
        ^
././mntopts.h:100:2: note: expanded from macro 'MOPT_STDOPTS'
        MOPT_NOCOVER,                                                   \
        ^
././mntopts.h:68:37: note: expanded from macro 'MOPT_NOCOVER'
#define MOPT_NOCOVER            { "cover",      1, MNT_NOCOVER, 0 }
                                                   ^
mount_hammer2.c:54:2: error: use of undeclared identifier 'MNT_EMPTYDIR'
././mntopts.h:101:2: note: expanded from macro 'MOPT_STDOPTS'
        MOPT_EMPTYDIR
        ^
././mntopts.h:69:41: note: expanded from macro 'MOPT_EMPTYDIR'
#define MOPT_EMPTYDIR           { "emptydir",   0, MNT_EMPTYDIR, 0 }
                                                   ^
2 errors generated.
Comment 8 kusumi.tomohiro 2022-11-26 06:00:02 UTC
There is another error when compiled against 12-stable src tree, due to nonexistent kernel header at the time.

---
In file included from hammer2_admin.c:46:
./hammer2.h:64:10: fatal error: 'sys/gsb_crc32.h' file not found
#include <sys/gsb_crc32.h>
         ^~~~~~~~~~~~~~~~~
1 error generated.
*** Error code 1
---

# git log --pretty="%h" -- sys/sys/gsb_crc32.h | cat | tail -1
f89d2072795
# git tag --contains f89d2072795
release/13.0.0
release/13.1.0

The one you mentioned is also due to nonexistent mount option MNT_NOCOVER and MNT_EMPTYDIR at the time.


These compile-time errors can most likely be addressed via #ifdef conditionals, however since file system is a type of code depends on kernel i/f or kernel itself, I see no point in trying to support all past FreeBSD versions. Even if it did *compile* with older versions, there is no guarantee this kernel subsystem works as expected at runtime for various reasons.

So let's drop everything below 13. If you could tell me a directive to write in Makefile to drop 12 or older, I could put that.
Comment 9 kusumi.tomohiro 2022-11-26 08:36:32 UTC
Tried to see if it's trivial to fix compilation for 12-stable.

For kernel side, the <sys/gsb_crc32.h> error I mentioned above can be worked around with "#if __FreeBSD_version >= 1300032", but then there are more compile-time errors around VFS due to things didn't exist 4 years ago.

So no, it's not realistic to support 12 or older kernels.

If it were merged to src (instead of ports), there would basically be no such thing supporting HAMMER2 on older kernels. So just because this exists as out-of-tree kernel module doesn't mean we want/need to care about older kernels.
Comment 10 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-28 00:54:36 UTC
Thanks, I'm fine with dropping support of 12.
Comment 11 commit-hook freebsd_committer freebsd_triage 2022-11-28 00:55:54 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=e510f45d474dc7bcd7d097a69ebfe30cee4fce68

commit e510f45d474dc7bcd7d097a69ebfe30cee4fce68
Author:     Tomohiro Kusumi <tkusumi@netbsd.org>
AuthorDate: 2022-11-25 06:48:22 +0000
Commit:     Koichiro Iwao <meta@FreeBSD.org>
CommitDate: 2022-11-28 00:52:35 +0000

    sysutils/hammer2: New port: HAMMER2 file system for FreeBSD

    HAMMER2 file system for FreeBSD (read-only support).
    https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/sys/vfs/hammer2/DESIGN

    Consists of hammer2.ko kernel module and sbin/* binaries.

    WWW: https://github.com/kusumi/freebsd_hammer2

    PR:             267982
    Tested by:      meta

 sysutils/Makefile                |  1 +
 sysutils/hammer2/Makefile (new)  | 38 ++++++++++++++++++++++++++++++++++++++
 sysutils/hammer2/distinfo (new)  |  3 +++
 sysutils/hammer2/pkg-descr (new) |  2 ++
 sysutils/hammer2/pkg-plist (new) |  9 +++++++++
 5 files changed, 53 insertions(+)
Comment 12 Koichiro Iwao freebsd_committer freebsd_triage 2022-11-28 01:01:58 UTC
Committed, thanks! Feel free to add me to CC list when you submit an update for this port. Then I can catch your PR soon.
Comment 13 kusumi.tomohiro 2022-11-28 01:35:57 UTC
Thanks for your work.
Comment 14 kusumi.tomohiro 2022-11-30 09:55:34 UTC
https://pkg-status.freebsd.org/beefy15/data/131i386-default/0772795015a6/logs/errors/hammer2-1.0.1.log

This happens with FreeBSD 13.X on i386 (and likely other 32 bit arch), due to readdir cookies having u_long type until this commit last year expanded it to 64 bit.

https://github.com/freebsd/freebsd-src/commit/b214fcceacad6b842545150664bd2695c1c2b34f

Things like DragonFly and NetBSD have had 64 bit cookies, so this was never a problem.

>  This doesn't matter for any in-tree file systems, but it matters for some FUSE file systems that use 64-bit directory cookies.

HAMMER2 also uses 64 bit cookies.
I'll try to address this in a few days.
Comment 15 Graham Perrin freebsd_committer freebsd_triage 2022-12-11 10:48:43 UTC
Thanks. 

Merge to quarterly?

<https://www.freshports.org/sysutils/hammer2/#packages>
Comment 16 Koichiro Iwao freebsd_committer freebsd_triage 2022-12-11 12:35:31 UTC
(In reply to Graham Perrin from comment #15)
AFAIK new ports are usually not added to quarterly.
Comment 17 Graham Perrin freebsd_committer freebsd_triage 2022-12-11 13:01:07 UTC
(In reply to Koichiro Iwao from comment #16)

Oh, that's a surprise. 

Please, is it documented somewhere? 

There's no mention of quarterly in the FreeBSD Porter's Handbook.
Comment 18 Koichiro Iwao freebsd_committer freebsd_triage 2022-12-12 00:26:31 UTC
(In reply to Graham Perrin from comment #17)

It is reasonable not to add new ports for the purpose of the quarterly branch. Actually, it is not prohibited to do that however I don't think this is not the case to add. If there's strong motivation to add this, we can ask for portmgr's opinion.

In either case, this port will be added quarterly after 2 weeks and a bit. As a new quarterly branch 2023Q1.


https://wiki.freebsd.org/Ports/QuarterlyBranch
Comment 19 Graham Perrin freebsd_committer freebsd_triage 2022-12-12 03:04:13 UTC
(In reply to Koichiro Iwao from comment #18)

Thanks for the explanations. 

<https://wiki.freebsd.org/Bugzilla/TriageTraining> updated.