Bug 259527 - net/seaweedfs: update to 3.26 and port refactoring
Summary: net/seaweedfs: update to 3.26 and port refactoring
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Dave Cottlehuber
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-29 12:42 UTC by Boris Korzun
Modified: 2022-11-16 08:44 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (dch)


Attachments
seaweedfs.patch (53.32 KB, patch)
2021-10-29 12:42 UTC, Boris Korzun
no flags Details | Diff
seaweedfs.patch (52.43 KB, patch)
2021-11-12 13:16 UTC, Boris Korzun
no flags Details | Diff
seaweedfs.diff (54.92 KB, patch)
2021-12-07 08:04 UTC, Boris Korzun
no flags Details | Diff
seaweedfs.diff (13.42 KB, patch)
2021-12-22 05:04 UTC, Boris Korzun
dch: maintainer-approval-
Details | Diff
trimmed rc.d & makefiles, provisional update to 2.90 (27.07 KB, patch)
2022-02-23 10:08 UTC, Dave Cottlehuber
no flags Details | Diff
seaweedfs-306.diff (76.09 KB, patch)
2022-05-26 11:57 UTC, Boris Korzun
no flags Details | Diff
seaweedfs-324.diff (10.78 KB, patch)
2022-08-29 15:38 UTC, Boris Korzun
no flags Details | Diff
seaweedfs-326.diff (10.56 KB, patch)
2022-09-09 09:11 UTC, Boris Korzun
drtr0jan: maintainer-approval?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Boris Korzun 2021-10-29 12:42:48 UTC
Created attachment 229120 [details]
seaweedfs.patch

Updated to 2.75.

And also changes:
- removed auto-creation /var/db/${PORTNAME} and /var/log/${PORTNAME} from Makefile;
- weed executable moved from sbin/weed to bin/weed (weed is running under nonprivilged user usually).
- rc.conf variables seaweedfs_syslog_* renamed to seaweedfs_syslog_output_* making them consistent with the rest of the ports tree;
- added rc.conf variable seaweedfs_syslog_output_tag for defining a syslog tag;
- default value of rc.conf variable seaweedfs_options changed to "server -dir=${seaweedfs_dir}";
- dropped daemon(8) syslog support verification;
- dropped extra rc commands reload and monitor as inoperative;
- dropped auto-creation ${seaweedfs_dir}/data and ${seaweedfs_dir}/metadata from rc-script as unnecessary.
- added auto-creation ${seaweedfs_dir} to rc-script.

Verified by portfmt/portclippy and poudriere testport. Also updated pkg-message.in and UPDATING.

What do you think about moving weed from sbin to bin? And about dropping auto-creation data and metadata dirs?
Comment 1 Boris Korzun 2021-11-12 13:16:28 UTC
Created attachment 229447 [details]
seaweedfs.patch

Update to 2.77
Comment 2 Boris Korzun 2021-12-07 08:04:41 UTC
Created attachment 229953 [details]
seaweedfs.diff

Update to 2.81
Comment 3 Dave Cottlehuber freebsd_committer freebsd_triage 2021-12-21 07:58:34 UTC
thanks for the patch & updates! I've been afk due to work for a spot.

I'll commit the distinfo & version changes immediately, and give the
rest some time to review.

The directory creation was definitely required in the past, how
does /var/{db,log}/* data and metadata get added for a new install
if not by us? if weed now auto-creates the *data dirs then that's
good news.

making settings consistent is good;
Comment 4 Dave Cottlehuber freebsd_committer freebsd_triage 2021-12-21 08:25:33 UTC
making rc* settings consistent is good; but I'm not sure yet about moving the
directories around. Can you elaborate a bit on why this helps your setup?

I'd originally wanted the port to make getting started with seaweedfs a lot
simpler, it's very confusing for newcomers.
Comment 5 Boris Korzun 2021-12-21 08:35:51 UTC
(In reply to Dave Cottlehuber from comment #4)

Current versions of seaweedfs don't use data and metadata directories. There's no a mention in the sources too.

These directories are always empty.

The changes aren't confusing for newcomers and don't depend from it.
Comment 6 commit-hook freebsd_committer freebsd_triage 2021-12-21 23:18:35 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=5baa6d516f41b5d4243eee1f02a21dea23790db6

commit 5baa6d516f41b5d4243eee1f02a21dea23790db6
Author:     Boris Korzun <drtr0jan@yandex.ru>
AuthorDate: 2021-12-21 07:34:11 +0000
Commit:     Dave Cottlehuber <dch@FreeBSD.org>
CommitDate: 2021-12-21 23:02:31 +0000

    net/seaweedfs: update to 2.81

    PR:             259527
    Sponsored by:   SkunkWerks, GmbH

 net/seaweedfs/Makefile      |  40 +++++----
 net/seaweedfs/Makefile.deps | 104 +++++++++++-----------
 net/seaweedfs/distinfo      | 206 +++++++++++++++++++++-----------------------
 3 files changed, 173 insertions(+), 177 deletions(-)
Comment 7 Boris Korzun 2021-12-22 05:04:05 UTC
Created attachment 230297 [details]
seaweedfs.diff

(In reply to commit-hook from comment #6)

Thx fo commit, but there is no need to create /var/log/seaweedfs dir. Seaweedfs doesn't use this directory. The directory is always empty.

Upgrade to 2.82.

And also changes:
- removed auto-creation /var/db/${PORTNAME} and /var/log/${PORTNAME} from Makefile;
- rc.conf variables seaweedfs_syslog_* renamed to seaweedfs_syslog_output_* making them consistent with the rest of the ports tree;
- added rc.conf variable seaweedfs_syslog_output_tag for defining a syslog tag;
- default value of rc.conf variable seaweedfs_options changed to "server -dir=${seaweedfs_dir}";
- dropped daemon(8) syslog support verification;
- dropped extra rc commands reload and monitor as inoperative;
- dropped auto-creation ${seaweedfs_dir}/data and ${seaweedfs_dir}/metadata from rc-script as unnecessary.
- added auto-creation ${seaweedfs_dir} to rc-script.
Comment 8 Dave Cottlehuber freebsd_committer freebsd_triage 2021-12-22 09:34:35 UTC
awesome! yes clearly there's been significant changes in code in the last few months. The directory structure depends on what backend DB is chosen, and I think we can just add that as comment to pkg-message.

I did notice that there is not a clean upgrade from 2.76 to 2.81, at least with leveldb backend, but I will follow that up with chrisf directly, I don't think we can do anything about it here.
Comment 9 Boris Korzun 2021-12-22 10:42:39 UTC
(In reply to Dave Cottlehuber from comment #8)

I've checked the sources of v2.19 (the first version seaweedfs in FreeBSD Ports) and haven't found mentions for /var/log/seaweedfs and metadata dirs.

What backend do you mean?

And what problem with upgrade from 2.76 to 2.81 do you have?
Comment 10 Dave Cottlehuber freebsd_committer freebsd_triage 2022-02-23 09:27:17 UTC
Comment on attachment 230297 [details]
seaweedfs.diff

Boris I do appreciate the patch & tidy up of the port, however some of
the change feels more based on personal preference. Moving things around
breaks real stuff for real users, and unless there's a gross violation
of hier(7) then POLA should apply, and people should expect their ports
and settings to continue working after updating.

Do we need to make the config changes? In what way is the current setup
broken, or impacting users somehow?

I'm just doing a 2.90 update now, so I'll include my patch here for
comment.
Comment 11 Dave Cottlehuber freebsd_committer freebsd_triage 2022-02-23 10:08:40 UTC
Created attachment 232036 [details]
trimmed rc.d & makefiles, provisional update to 2.90

Has this build failure (on arm64) 

===>  Building for seaweedfs-2.90
(cd /tmp/usr/ports/net/seaweedfs/work/seaweedfs-2.90;  for t in ./weed; do  out=$(/usr/bin/basename $(echo ${t} |  /usr/bin/sed -Ee 's/^[^:]*:([^:]+).*$/\1/' -e 's/^\.$/seaweedfs/'));  pkg=$(echo ${t} |  /usr/bin/sed -Ee 's/^([^:]*).*$/\1/' -e 's/^seaweedfs$/./');  echo "===>  Building ${out} from ${pkg}";  /usr/bin/env XDG_DATA_HOME=/tmp/usr/ports/net/seaweedfs/work  XDG_CONFIG_HOME=/tmp/usr/ports/net/seaweedfs/work  XDG_CACHE_HOME=/tmp/usr/ports/net/seaweedfs/work/.cache  HOME=/tmp/usr/ports/net/seaweedfs/work TMPDIR="/tmp" PATH=/usr/local/libexec/ccache:/tmp/usr/ports/net/seaweedfs/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  "  MANPREFIX="/usr/local" CCACHE_DIR="/root/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" CGO_ENABLED=1  CGO_CFLAGS="-I. -I/usr/local/include"  CGO_LDFLAGS="-L/usr/local/lib"  GOARM= GOPATH="/portdistfiles/go/net_seaweedfs"  GOBIN="/tmp/usr/ports/net/seaweedfs/work/bin"  GO111MODULE=on  GOFLAGS=-modcacherw  GOSUMDB=sum.golang.org GO_NO_VENDOR_CHECKS=1 GOPROXY=off /usr/local/bin/go build -v -buildmode=exe -trimpath -ldflags=-s -mod=vendor  -o /tmp/usr/ports/net/seaweedfs/work/bin/${out}  ${pkg};  done)
===>  Building weed from ./weed
weed/filer/etcd/etcd_store.go:10:2: cannot find package "." in:
	/tmp/usr/ports/net/seaweedfs/work/seaweedfs-2.90/vendor/go.etcd.io/etcd/client/v3
*** Error code 1


I'll need to look into this further. Help welcomed!
Comment 12 Boris Korzun 2022-02-24 08:46:52 UTC
(In reply to Dave Cottlehuber from comment #11)

The build error is appeared because you've changed etcd_io_etcd behavior in Makefile.deps. I've restored the etcd_io_etcd part of Makefile.deps and build is successful.

Why are you moved weed binary from bin directory to sbin directory?
Why are you removed SEAWEEDFS_USER and SEAWEEDFS_GROUP from the Makefile?
Comment 13 Boris Korzun 2022-02-24 09:31:11 UTC
(In reply to Dave Cottlehuber from comment #10)
The changes doesn't feel based on personal preference. They follow the principles of consistent and unification with the rest of the ports tree.

1) Renaming seaweedfs_syslog_* options to seaweedfs_syslog_output_* has backward compatibility for people are used its. The port will continue working after updating.
2) Adding seaweedfs_syslog_output_tag variable doesn't break port behavior.
3) Dropping auto-creation of ${seaweedfs_dir}/metadata doesn't break port behavior. There're no users who used the dir (the dir is undocumented).
4) Changing seaweedfs_options and dropping auto-creation of ${seaweedfs_dir}/data also has backward compatibility for people are used their.

What is the principle of POLA you meaning?
Comment 14 Boris Korzun 2022-05-26 11:57:11 UTC
Created attachment 234228 [details]
seaweedfs-306.diff

Upgrade to 3.06.

And also changes:
- removed auto-creation /var/db/${PORTNAME} and /var/log/${PORTNAME} from Makefile;
- rc.conf variables seaweedfs_syslog_* renamed to seaweedfs_syslog_output_* making them consistent with the rest of the ports tree;
- added rc.conf variable seaweedfs_syslog_output_tag for defining a syslog tag;
- default value of rc.conf variable seaweedfs_options changed to "server -dir=${seaweedfs_dir}";
- dropped daemon(8) syslog support verification;
- dropped extra rc commands reload and monitor as inoperative;
- dropped auto-creation ${seaweedfs_dir}/data and ${seaweedfs_dir}/metadata from rc-script as unnecessary;
- added auto-creation ${seaweedfs_dir} to rc-script;
- added port options for ELASTIC, GOCDK, HDFS, SQLITE AND YDB.
Comment 15 Boris Korzun 2022-08-29 15:38:10 UTC
Created attachment 236220 [details]
seaweedfs-324.diff

Upgrade to 3.24.

And also changes:
- removed auto-creation /var/db/${PORTNAME} and /var/log/${PORTNAME} from Makefile;
- rc.conf variables seaweedfs_syslog_* renamed to seaweedfs_syslog_output_* making them consistent with the rest of the ports tree;
- added rc.conf variable seaweedfs_syslog_output_tag for defining a syslog tag;
- default value of rc.conf variable seaweedfs_options changed to "server -dir=${seaweedfs_dir}";
- dropped daemon(8) syslog support verification;
- dropped extra rc commands reload and monitor as inoperative;
- dropped auto-creation ${seaweedfs_dir}/data and ${seaweedfs_dir}/metadata from rc-script as unnecessary;
- added auto-creation ${seaweedfs_dir} to rc-script;
- added port options for ELASTIC, GOCDK, SQLITE, TIKV and YDB.
Comment 16 Boris Korzun 2022-09-09 09:11:02 UTC
Created attachment 236451 [details]
seaweedfs-326.diff

Upgrade to 3.26.

And also changes:
- updated WWW in the Makefile;
- removed auto-creation /var/db/${PORTNAME} and /var/log/${PORTNAME} from Makefile;
- rc.conf variables seaweedfs_syslog_* renamed to seaweedfs_syslog_output_* making them consistent with the rest of the ports tree;
- added rc.conf variable seaweedfs_syslog_output_tag for defining a syslog tag;
- default value of rc.conf variable seaweedfs_options changed to "server -dir=${seaweedfs_dir}";
- dropped daemon(8) syslog support verification;
- dropped extra rc commands reload and monitor as inoperative;
- dropped auto-creation ${seaweedfs_dir}/data and ${seaweedfs_dir}/metadata from rc-script as unnecessary;
- added auto-creation ${seaweedfs_dir} to rc-script;
- added port options for ELASTIC, GOCDK, SQLITE, TIKV and YDB.
Comment 17 commit-hook freebsd_committer freebsd_triage 2022-11-13 12:02:49 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=8bb719ffa3c9e643c5048b046a1e0843195c8e49

commit 8bb719ffa3c9e643c5048b046a1e0843195c8e49
Author:     Dave Cottlehuber <dch@FreeBSD.org>
AuthorDate: 2022-11-13 12:02:38 +0000
Commit:     Dave Cottlehuber <dch@FreeBSD.org>
CommitDate: 2022-11-13 12:02:38 +0000

    net/seaweedfs: update to 3.26, pet port

    PR:             259527
    Sponsored by:   SkunkWerks, GmbH

 net/seaweedfs/Makefile             |  29 ++++++++--
 net/seaweedfs/distinfo             |  10 ++--
 net/seaweedfs/files/pkg-message.in |  21 +++++++
 net/seaweedfs/files/seaweedfs.in   | 113 ++++++++++++++++++-------------------
 net/seaweedfs/pkg-plist            |   2 -
 5 files changed, 105 insertions(+), 70 deletions(-)
Comment 18 Dave Cottlehuber freebsd_committer freebsd_triage 2022-11-13 12:03:19 UTC
hi Boris,

I have committed your last patch as-is.

do you want to pick up maintainership of this port? if
so just send a PR in with the MAINTAINER diff, and my ok.