Created attachment 235695 [details] The port shar Archiver with deduplication and snapshot freezing (aka: N zfs' snapshots in one .zpaq file), just like 7z "merged" with Time Machine FreeBSD version of the zpaqfranz port (OpenBSD here https://openports.se/archivers/zpaqfranz) --- zpaq is a free and open source incremental, journaling command-line archiver, designed and first implemented by Matt Mahoney. zpaq is faster and compresses better than most other popular archivers and backup programs, especially for realistic backups that have a lot of duplicate files and a lot of already compressed files. zpaqfranz is a more advanced fork of the original zpaq archiver (archives/paq) and continues to be maintained. --- This is the lastest version (55.8), not really sure if compile on non-Intel CPUs (I do not have any), but it should
Thanks for submitting a port! A few things require changes though: * Never set WRKSRC to WRKDIR. Instead, for an upstream distfile that doesn't create a subdir itself, set NO_WRKSUBDIR=yes. * Never define do-build and do-install targes if upstream already provides a build system that can do them (which is the case here). * Always remove -march=native (you will need to patch the upstream Makefile for that). Packages must only contain generic binary code that works on every machine. * Similar for CPU features, you're checking for SSE2 on the building machine, but if the executing machine doesn't have it, it will fail. In general, you can't assume SSE2 on i386. * The upstream Makefile installs the same binary twice. Remove one of them. If the second name is really strictly needed, replace it with a symlink. --- A simple fixed port could look like this: 1) remove everything in the port Makefile starting with WRKSRC= ... Instead just use this: NO_WRKSUBDIR= yes CFLAGS+= -DNOJIT 2) add a local patch for upstream's Makefile, removing the -march=natice and also the second install command. If necessary replace it with a symlink, e.g. like this: ln -s zpaqfranz ${DESTDIR}${BINDIR}/dir
Thank for you comments "* Never set WRKSRC to WRKDIR. Instead, for an upstream distfile that doesn't create a subdir itself, set NO_WRKSUBDIR=yes." It is my very first port, I have "cloned" some other, I'll check "* Never define do-build and do-install targes if upstream already provides a build system that can do them (which is the case here)." You can find here why there are "two" Makefiles https://forums.freebsd.org/threads/makefile-s-for-a-non-intel-only-port.86073/ One (the "inside .tar.gz") is, in fact, a Linux/FreeBSD/Solaris Makefile The second ("the port") is the FreeBSD specific, checking for "Intel" CPU and "others" (defining -DNOJIT in the compiling phase) Basically you need a conditional compilation (by activating -DNOJIT) on non-x86 machines. Maybe you can suggest me the best way * Similar for CPU features, you're checking for SSE2 on the building machine, but if the executing machine doesn't have it, it will fail. In general, you can't assume SSE2 on i386. It is true, but I do not know how to write a "multiple case" (yet) "* The upstream Makefile installs the same binary twice. Remove one of them. If the second name is really strictly needed, replace it with a symlink." The second name is really needed, because if the executable is named "dir" ... works just like Windows' dir :-) In fact two programs in one executable
PS the CPU check is from ancient zpaq /usr/ports/archivers/paq/Makefile
PS/2 CFLAGS+= -DNOJIT is a big nono, if not really necessary Because inside the source there is a JIT compiler from ZPAQL (esoteric compression language by Matt Mahoney) to x86 opcodes, much, much faster (of course) of the interpretated one. Short version: on non-Intel CPU, and not sse2, interpretation is required (-DNOJIT). On "modern" Intel CPUs, native machine code will be used by default About the symlink I'm not really convinced. I much prefer to leave both, so they are untied (delete one or the other, or both) The software has no dependency and practically does not require installation/deinstallation, it can be deleted directly without leaving "junk" If this is prohibited by FreeBSD policy I will simply remove "dir" in the FreeBSD version and leave it for others Incidentally, what is dir? It is a kind of Windows mini-dir, which shows the total size of the files Maybe try it, you will see that it is comfortable :) dir /s dir /os
(In reply to Franco Corbelli from comment #2) > It is my very first port, I have "cloned" some other, I'll check That's perfectly fine, I'm just telling you the correct way. You really can't know everything when you're just starting ;) > You can find here why there are "two" Makefiles [...] I commented in that thread myself, and I guess there's still some misunderstanding. Ports *always* use whatever upstream provides for building and installing (here, it's a Makefile in the upstream tarball). The do-build and do-install targets are *only* for situations where this is absolutely impossible. Here you just have an offending '-march=native' in the upstream Makefile, the way to go is to patch it out. For adding CFLAGS (like -DNOJIT), the upstream Makefile will already pick them up. > Maybe you can suggest me the best way See my previous comment. Removing everything starting with the line setting WRKSRC and replacing it with just the two lines I suggested will already work. > It is true, but I do not know how to write a "multiple case" (yet) All you could do here is provide an option to enable SSE2, so the ports user can opt to build it that way. It can't be enabled for the official package though. > In fact two programs in one executable That's a somewhat common thing, but installing the same executable twice is unnecessary and will never be done by a FreeBSD package, that's what symlinks are for.
(In reply to Franco Corbelli from comment #4) > PS/2 CFLAGS+= -DNOJIT is a big nono, if not really necessary > [...] You can enable SSE2 for amd64. You can't for i386 (except when adding a port option to do so). The latter would be a bit "advanced", and you can also add it later. > I much prefer to leave both, so they are untied (delete one or the other, > or both) > The software has no dependency and practically does not require > installation/deinstallation, it can be deleted directly without leaving "junk" This makes little sense when building a port. Having pkg manage installed files is kind of the point of it. It's not uncommon to have binaries do different things based on argv[0]. A symlink is the correct way to install such a thing.
1) on the symlink I do not know how to make via the Port's Make (do-install) 2) I do NOT want to use (in FreeBSD port manager) the "first" Makefile at all 3) -march=native therefore should not be called, and in fact does not seems to be (during the port make install) ===> Patching for zpaqfranz-55.8 ===> Configuring for zpaqfranz-55.8 ===> Building for zpaqfranz-55.8 c++ -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Dunix /tmp/zpaqfranz/work/zpaqfranz-55.8/zpaqfranz.cpp -o /tmp/zpaqfranz/work/zpaqfranz -static -pthread -lstdc++ -lm 4) the -DNOJIT I am thinking on something quick-and-dirty, just like that NO_WRKSUBDIR= yes ZPAQFLAGS= -Dunix -DNOJIT .include <bsd.port.options.mk> .if (${ARCH} == "amd64") ZPAQFLAGS= -Dunix .endif .if ((!empty(MACHINE_CPU:Msse2)) && (${ARCH} == "i386")) ZPAQFLAGS= -Dunix -msse2 .endif Is it feasible?
(In reply to Franco Corbelli from comment #7) > I do not know how to make via the Port's Make (do-install) I showed you how to do it by patching the upstream Makefile. > 2) I do NOT want to use (in FreeBSD port manager) the "first" Makefile at all This is against policies for good reasons. If upstream changes anything about how the software is built and installed, you don't want to catch up in the port every time. > Is it feasible? No, you're still assuming SSE2 as soon as your *build* machine supports it, but not every machine supported by the i386 architecture will be able to execute that.
OK, so I understand that it is not possible to gettwo birds with one stone Pity I will remove non-amd64 CPU support at all (ONLY_FOR_ARCHS= amd64) and make a FreeBSD-only Makefile
All you have to do is to make sure the build for i386 *with default options* doesn't use SSE2. ONLY_FOR_ARCHS is only for situations when a software is *known* to be broken on other architectures. Being less performant than on other architectures does not qualify as "broken". The simple solution would probably be to add -DNOJIT to CFLAGS for any architecture except amd64. Of course, you are free to add an extra OPTION, so users can decide to build it for i386 with SSE2.
To be even "surer" I will strip everything (for the FreeBSD) that is not amd64 only Thanks for the help I am doing this way
(In reply to Franco Corbelli from comment #11) I don't see where this should lead. If a software can be built for all architectures, it makes no sense to artificially restrict it. Just to give you a feeling, here's a q&d port Makefile that would work: --- PORTNAME= zpaqfranz DISTVERSION= 55.8 CATEGORIES= archivers MASTER_SITES= http://www.francocorbelli.it/zpaqfranz/ MAINTAINER= franco@francocorbelli.com COMMENT= Swiss army knife for the serious backup and disaster recovery manager LICENSE= BSD2CLAUSE MIT LICENSE_COMB= dual NO_WRKSUBDIR= yes post-patch: @${REINPLACE_CMD} 's|-march=native||' ${WRKSRC}/Makefile post-install: ${RM} ${STAGEDIR}${PREFIX}/bin/dir ${LN} -s ${PORTNAME} ${STAGEDIR}${PREFIX}/bin/dir .include <bsd.port.pre.mk> .if ${ARCH} != "amd64" CFLAGS+= -DNOJIT .endif .include <bsd.port.post.mk> ---
I prefer to make a specific release for FreeBSD, which compiles only on amd64 and is devoid of any hardware acceleration for SHA1 and BLAKE3, without the code for Windows, Solaris etc. Makefile(s) therefore become minimal and the source faster to download and compile. Anyone needing other functions can always download the "full" source from github and compile it
Created attachment 235803 [details] The current shar (amd64 only) This version should compile correctly on amd64 CPU in full compliance with FreeBSD's policies for ports Or, at least, I hope so :)
Actually, a policy about ARCH restrictions is: Only restrict them if the software is *known* to be broken on some arch. But then, this new port would (attempt to) build on any arch. You added ONLY_FOR_ARCHS to the *upstream* Makefile, where nothing would ever care about it. It's a variable of FreeBSD's port building framework, so only does anything in a port Makefile. Still I wonder why you insist on restricting anything? I gave you a simple example above how you could have your SSE2-dependent stuff on only amd64 and still have it build on any other arch. And if you want to take one step further, you can still add a port option enabling SSE2 on i386 as well, it just can't be on by default because official packages must run fine on any CPU of the respective ARCH.
Created attachment 235804 [details] The current shar (amd64 only) Prevents compilation on i386 (generally everything not amd64) root@bit32:/tmp/cz/zpaqfranz # make ===> zpaqfranz-55.9 is only for amd64, while you are running i386. *** Error code 1
> Only restrict them if the software is *known* to be broken on some arch. Not having the possibility (and time) to carry out extensive tests I prefer this way. It will be at least 3 years that I no longer verify the interpreted functionality, maybe more. Nothing prevents other users from making a new fork and proposing it on other architectures.
> Only restrict them if the software is *known* to be broken on some arch. AFAIK there aren't "bugs" BUT three+1 kinds of tests are missing, as I physically have no access to non-Intel machines to run them on The first is related to "endianness", in particular for the additional hashes and even more in particular for xxhash64 (the default one), but also all the other optional ones (less important, to be selectively disabled) The second is about memory alignment, in particular of BLAKE3 (I have implemented my own specific memory alignment function for run-time creation of objects that works fine on Intel) The third, linked to the first, is certify the compatibility of files (and the hashes created) between machines with different endianness. Translation. If the files created on the non-Intel machine contain the same data calculated by an Intel machine (on the same original files) The fourth (+1) is checking the non-latin handling of filenames (aka: utf-8, aka: russian, chinese etc) on internal routines. I do not think there are particular problems, but it is not sure (for example in the calculation of the length of long strings) **Worst scenarios** Case 1: The created files should be restorable on machines with different CPUs, BUT without checking the stored hashes Case 2: (BLAKE3) can prevent this algorithm from being used as a hash saved within files (the -blake3 optional switch) Case 3: Files may not be restorable on machines with different CPUs THEN I "stripped" from the proposed FreeBSD port the NOJIT support altogether, going back to amd64 By FreeBSD policy you have to *know* that something is broken, but this goes against MY policy, which can be read by writing zpaqfranz /? --- Doveryay, no proveryay; trust, but verify; fidarsi e' bene, non fidarsi e' meglio. --- Some kind of "non-intel" help is needed, it would be even better if I had ssh access to a non-Intel FreeBSD machine to run my tests directly, but as mentioned I don't know any non-Intel VPS to rent. --- This is not some kind of "hello world" software, which "maybe" can work, but, if it doesn't, no big deal. This is a very complex "piece", with some cutting-edge technologies inside, that must be trusted (or at least, trusted as possible). Certainly when someone use free software it is HIS problem whether it will work or not: the moment you "run" something, you accept every risks But I have a reputation that I intend to keep I hope to have clearly explained MY policy
Hi Franco, I now had another look. I let it sit for a while to give time thinking about it... First of all, about the architecture restriction again. Of course, as you're also the upstream author, you're in two different roles here. If you remove code you consider "untrusted", so it will (for now) only work on amd64, that's fine. It just leaves me a bit confused why you first submitted a version without that restriction (and were fine with using said "untrusted" code on i386 in case the *build* machine would not support SSE2)? So, in a nutshell, are you really sure about that? Testing the port, i noticed two more things: * You link statically. Normally, you only do that for distributing software outside of any "package management", and FreeBSD ports would typically patch upstream Makefiles not to do that if there's no really unavoidable reason. Now, as you are the upstream author yourself, would you mind changing your own Makefile so it *doesn't* link statically? (related question, why is C++ code compiled using ${CC}, so you have to explicitly link libstdc++?) * The 'dir' incarnation vanished completely. I'm still not sure whether it would do what I *assume* it would (behave differently depending on argv[0]), but *if* it does and this behavior is useful to the user, please just add a symlink. That's the typical practice, you can e.g. see it used by vim: $ file /usr/local/bin/gvim /usr/local/bin/gvim: symbolic link to vim
>First of all, about the architecture restriction again. Of course, as you're also >the upstream author, you're in two different roles here. If you remove code you >consider "untrusted", so it will (for now) only work on amd64, that's fine. >(...) >So, in a nutshell, are you really sure about that? I asked a user for help to try the software on non-Intel little endian (ARM) architectures and, as I expected, there was indeed a (minor) problem due to different compiler standards, fixed, in UTF-8 handling (note: with "UTF-8" I mean briefly the whole actual filename storage chain, I know "pretty" well what UTF-8 is, semantics are looser here) I then bought a Macintosh PowerPC, to do certification tests on BIG ENDIAN machines (where I expect more problems, due the advanced hashers used) I have to fix the relative cabling and then make it work (which is not trivial) Work in progress I then implemented a new self-test function (which does not exist in the FreeBSD release) which, in theory, should allow to verify (most, certainly not all) the main functions Basically the program now contains an archive of about 200KB (which weights down the executable but oh well), which is extracted, togheter with a special folder structure that is then created with pseudorandom files and, finally, a real script (.sh for non-Windows and .bat for Windows) is prepared by zpaqfranz I'm working on it, it's not trivial This will become something like zpaqfranz autotest -to /tmp/testfolder Then you run /tmp/testfolder/dotest.sh following the instructions. >Testing the port, i noticed two more things: >* You link statically. Normally, you only do that for distributing software >outside of any "package management", and FreeBSD ports would typically patch >upstream Makefiles not to do that if there's no really unavoidable reason. Now, >as you are the upstream author yourself, would you mind changing your own >Makefile so it *doesn't* link statically? (related question, why is C++ code >compiled using ${CC}, so you have to explicitly link libstdc++?) The issue of static linking is very complex and articulated. There are pros and cons to both choices. Personally, I believe that in 2022 the costs (of non-static linking) are much greater than the advantages. zpaqfranz is a kind of "Docker" or "backup-busybox" (of course I'm using a metaphor). In the very latest version I just implemented the equivalent of the pause command of Windows scripts to use them in *nix ones. Each platform has its own preferences: for example RedHat and "cousins" do not like them for many reasons (you can see them in the official documentation). Most of which are laughable, seems to be back to segments and overlays, when I was writing the first programs in 1985 Similarly on archlinux, but for a different reason (the PIE and RELO enforcing): the package-tester is "offended" and issues warnings,for no real reason. But they are automatic tools, and nobody likes warnings On MacOS binaries are practically impossible to link statically, Apple goes to great lengths to prevent this So why link statically ? Because it works. There are no "breaks" due to updates, upgrades or whatever, symlinks that "magically" disappear, collisions with other software, with the operating system or whatever A kind of little "Docker", remember? The software (of this kind) must works, and that's what counts the most, basically in all circumstances. I administer several servers, where it happens from time to time that updates forced by external reasons (EOL) lead systems to "break" something. It is not a huge problem, it can generally be corrected, perhaps by renting a KVM IP for a day in the worst case. But this "category" of software (not just zpaqfranz, I mean "backupper") are different: typically used for backup (and restore) of systems in production, where it is unthinkable to suspend backups, perhaps for days, because the X.Y library for some reason no longer makes your crontab-copy procedure runs after some kind of upgrade for the "magicalserver 3.1" to "magicalserver 3.1a". In some case it is impossible to do a rollback (you lose client-uploaded data) or are extremely expensive Short version: virtually zero security issues (I see no need for constantly updated libraries, it's NOT some kind of daemon interacting with internet that can be targeted). Virtually zero need for updating (essentially read-write to disk, malloc() and little more) so the very latest version of the library with new superdupefunction is just useless As mentioned it is a cost-benefit balance, there is no "perfect" solution On systems where it is mandatory not to do static linking ... I don't. If static linking is not allowed on FreeBSD, then let it be dynamic >* The 'dir' incarnation vanished completely. >...please just add a symlink. It is a minor problem: if I manage to make this port finalized (zpaqfranz), I will make another one, almost identical, where the executable will be called just... dir :) Why not a symlink ? The symlink operates simul stabunt vel simul cadent, and that's NOT what I want. Because it happens to remove the executable of zpaqfranz, keeping the one of dir. Why? Because is it normal to have DIFFERENT versions of zpaqfranz in different directories, not just /usr/local/bin Without of course jails or whatever (too complicated - too much at risk of malfunctioning) When there is a (backup) program update "you" can't just change the executable and that's it, from version 5.3 to 5.4. You will need to keep at least two different program versions (the old and the new) and two different backup archives (the old and the new) It is not uncommon to have even 4 different versions, when there are major changes Within, say, 6 months, when you have done numerous data restores with the new program, then you will replace "the" /usr/local/bin executable. In the meantime, you may erase (depends on the priority in the PATH of your scripts) /usr/local/bin/zpaqfranz While the "command" dir does not require these special precautions: if, for some reason, it's buggy, never mind, you'll use ls and df On the other hand, the occupation of space is irrelevant (~2MB) Short version: this category of programs (archivers, backups, etc.) requires special precautions and the modalities I adopt reflect what happens when using them in practice It may sound like paranoia. But it is only experience Thanks anyway for your interest
(In reply to Franco Corbelli from comment #20) > If static linking is not allowed on FreeBSD, then let it be dynamic It's not forbidden, but it's typically removed if there is no unavoidable reason. Remember, we're talking about managed packages here. No way required libs will just disappear (except for a user going rogue as root, but that's not really a concern). In your case, you're only linking libraries from base. Those are guaranteed to never receive incompatible changes (but they *might* receive security updates). Linking them statically will have two undesirable effects: - you can't control whether security fixes are really applied - you lose reproducible builds when these libs are ever changed So, in this case here, I see no reason for static linking. It's a different story if you want to provide a "portable" binary outside of FreeBSD ports as some kind of rescue tool, there, static linking makes a lot of sense. > I will make another one, almost identical, where the executable will be called > just... dir Almost certainly, this wouldn't be accepted, and people would tell you "add a symlink to your original port". > Because it happens to remove the executable of zpaqfranz, keeping the one of > dir. No, this certainly doesn't happen when both are in the same package. Again, a root user "going rogue" is not a relevant scenario. > When there is a (backup) program update "you" can't just change the executable > and that's it, from version 5.3 to 5.4. > > You will need to keep at least two different program versions (the old and the > new) and two different backup archives (the old and the new) > […] This sounds like a port/package would be unsuitable for your software. There's no sane way to keep multiple versions installed. You would have to create a new port for each and every version if that is *really* needed :o Multiple ports can be a strategy (there are ports doing that), but not if every single version drops backwards compatibility ... it can't grow forever.
(In reply to Felix Palmen from comment #21) >It's not forbidden, but it's typically removed if there is no unavoidable >reason. >Remember, we're talking about managed packages here. No way required libs will >just disappear (except for a user going rogue as root, but that's not really a >concern). In reality, they always "disappear" (better: some dependency of some program automatically uninstalls another program, which in turn had a specific version etc) Classic example: sphinx, mysqlclient and mariadb But I wouldn't want to go too far, let's go back to zpaqfranz: I do not think it is not the best choice, but if dynamic linking is good for you, is good for me too >- you can't control whether security fixes are really applied Security fix for an archiver? And what kind can they be? In reality, any malicious injection into a shared library affects "everyone", even zpaqfranz, which does not need it at all Similarly for bugs introduced (it often happens) My philosophy is: as long as it works, leave it alone. Do not even touch. Maybe it's different on "newer" FreeBSD As always, there are costs and benefits in every choice. >you lose reproducible builds when these libs are ever changed It doesn't seem like a problem to me, but it's not important >So, in this case here, I see no reason for static linking. OK, dynamic linking is definitely good (...) >Almost certainly, this wouldn't be accepted, and people would tell you "add a >symlink to your original port". I will make (if I have to) a dedicated Delphi program which, from the zpaqfranz source, will make the source dir.cpp, without all the useless portions A couple of hours of work, a different software, a different port. >Again, a root user "going rogue" is not a relevant scenario. It happens all the time :) So much so that there is a specific trick in zpaqfranz also for this case >This sounds like a port/package would be unsuitable for your software (...) Don't confuse the "normal" user, the one using port or package, with the "advanced" user. The latter will download the latest version of the source with wget or even git-something and compile it by hand (no make needed), linking statically :) Or, maybe, it will write its very own zpaq++ > but not if every single version drops backwards compatibility ... > it can't grow forever. Backward compatibility is ensured: it is a key element of my project. I worked hard to "hack" the storage format of zpaq to save the CRC-32 and the hash of each file, for SHA-1 collision detection, without the zpaq version already in the port tree even noticing it. Statically compiled executables can (and usually are) be launched directly with their version name: they are independent Remember: each is about 2MB in size, no configuration file, no dependencies --- But let's not exaggerate, the "normal" user doesn't want much more than "make install clean" or "pkg install something" My port proposal is for that "type" of user Short version: dynamic linking OK
root@bsd131:/tmp/caricasito # ldd zpaqfranz zpaqfranz: libc++.so.1 => /usr/lib/libc++.so.1 (0x800369000) libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x800440000) libm.so.5 => /lib/libm.so.5 (0x800462000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80049d000) libthr.so.3 => /lib/libthr.so.3 (0x8004b7000) libc.so.7 => /lib/libc.so.7 (0x8004e5000)
Hello Franco, if you submit a version that links dynamically and just packages that one binary, that would be ready for commit IMHO. The normal way would be to patch the `-static` in the upstream Makefile away in the port. But as you are also the upstream author, you might prefer to remove it upstream. That's entirely your choice, do whatever you think is more appropriate! Just two tiny things to fix: * the COMMENT is too long -- just remove the ", on FreeBSD amd64 only" part, it's redundant information anyways. * the pkg-descr should be reworded in whole sentences, and references to other ports, if you want to keep them, should leave out `/usr/ports`. If you want, I can provide a suggestion. --- But reading this: > Backward compatibility is ensured: it is a key element of my project. I now completely fail to see any reason why anyone should want to keep multiple versions installed (which is a good thing!). As a consequence, your decision not to include that `dir` symlink remains a mystery. Mind you again, people using packages don't just randomly remove files that were installed by a package. And if they do, they know they're about to shoot their own foot. Anyways, the port would be fine. I personally think you could improve the usefulness by adding the symlink. And you could improve it even further, once you trust your alternative code again, by enabling other architectures again. But as the port maintainer, you can always improve later, just open a new PR for an update.
I just ordered cables for the PowerPC Mac. If I can get it to work, to run the necessary tests (I don't even know if it turns on) I will decide what to do about supporting non-amd64 platforms. --- Why have different versions? Because of bugs and different behaviour on different compiler on different platforms This "subtle" change, for example, will have no "visible" effects (compression is very delicate from this point of view), but it does, and sometimes (not always) the archived files are no longer restorable in Ubuntu (!) dt[fn]=dtr; => dt.insert(std::pair<string, DT>(fn,dtr)); Inside the source there is, for this very reason, an entire separate program (complete with... a reworked main (!)), the reference decompressor unzpaq 2.06, with all its support functions. The unz() functions, with unzSHA1 class and so on (if you wonder why there are several different classes that do the same thing), to get the p (paranoid) command, which uses a DIFFERENT program at all to test restorability However, it has a performance problem, which I do not go into, and therefore cannot be used in all cases, and here the w command etc. --- Turning back: I will update the port-proposal with your suggestions (no -static, yes symlink etc) Work in progress.
(In reply to Franco Corbelli from comment #25) > Why have different versions? > Because of bugs and [...] > sometimes (not always) the archived files are no longer restorable in Ubuntu (!) Sounds horribly weird. :o But doesn't sound like a reason to have multiple ports for multiple versions (which would be the only way to support having multiple versions installed from ports/packages). After all, that's not a problem with backwards compatibility, but indeed, as you say, with bugs. But if things like this happen, providing a "portable" binary (statically linked) for "rescue" purposes (probably outside of ports) sounds like a good idea! > I will update the port-proposal with your suggestions (no -static, > yes symlink etc) > Work in progress. Thanks, looking forward to commit it!
Created attachment 236044 [details] Current shar (no architecture restriction) The "internal" Makefile now create a symlink (-s) to /usr/local/bin/dir "external" Makefile add a -DNOJIT if not amd64 pkg/plist with two entries (bin/zpaqfranz, bin/dir) for make deinstall pkg-descr changed (the same into the executable's help) source updated to the latest general version (not specific to FreeBSD)
(In reply to Franco Corbelli from comment #27) Thanks, testing it right now! There's a suggestion to integrate the PLIST in the Makefile (instead of a separate pkg-plist file) if it contains less than 6 items. Would you be fine if I changed that? Or do you expect the list of files to grow?
>Would you be fine if I changed that? You can change anything you want >Or do you expect the list of files to grow? It won't grow, the files are encapsulated inside the executable, a bit like Windows resources (but compressed with LZ4)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=0207109c3a96041960879532ef5e91d518781776 commit 0207109c3a96041960879532ef5e91d518781776 Author: Franco Corbelli <franco@francocorbelli.com> AuthorDate: 2022-08-21 13:18:00 +0000 Commit: Felix Palmen <zirias@FreeBSD.org> CommitDate: 2022-08-21 19:19:01 +0000 archivers/zpaqfranz: Add new port Archiver with deduplication and snapshot freezing (aka: N zfs' snapshots in one .zpaq file), just like 7z "merged" with Time Machine PR: 265651 Approved by: tcberner (mentor) Differential Revision: https://reviews.freebsd.org/D36278 archivers/Makefile | 1 + archivers/zpaqfranz/Makefile (new) | 23 +++++++++++++++++++++++ archivers/zpaqfranz/distinfo (new) | 3 +++ archivers/zpaqfranz/pkg-descr (new) | 16 ++++++++++++++++ 4 files changed, 43 insertions(+)
Committed (with a few minor cosmetic changes), thanks!
I just got a "collision" report from a user: "with misc/gnuls: file bin/dir I had to deinstall gnuls to upgrade zpaq when it appeared in the ports. I am wondering if zpaq it will continue to work as I then force installed gnuls overwriting the dir which is a symbolic link. [I have many aliases w/gnuls not possible with plain ls]" It really looks like the dir function is cursed :) Possible solutions 1) remove the symlink to dir 2) test if the conflict package is already installed, and if so DO NOT symlink 3) use the OPTIONS to let the user choose
4) add some kind of information to the user of the type "If you override the bin/dir symlink nothing bad happens, zpaqfranz continues to work fine" I think the more "doable" version is something like this - if the symlink already exists, do nothing - if it doesn't exist, do it, and in any case warns the user that he can safely delete or overwrite: it's just an abbreviation to type "dir something" instead of "zpaqfranz dir something" Thank you all for the various contributions
(In reply to Franco Corbelli from comment #32) > I just got a "collision" report from a user: Please see also https://forums.freebsd.org/threads/makefile-s-for-a-non-intel-only-port.86073/#post-579107 The immediate solution is to add an installation conflict, this will be committed soon. > 1) remove the symlink to dir Not the best option, as it will lose functionality. As suggested over on forums, you might just want to call it differently (less generic) in the future to avoid the conflict. > 2) test if the conflict package is already installed, and if so DO NOT symlink Definitely no. That's again the same problem as with SSE2 etc, the build machine isn't necessarily the same as the machine the package will be installed on. > 3) use the OPTIONS to let the user choose Possible, but maybe unnecessarily complicated...
(In reply to Franco Corbelli from comment #33) > it's just an abbreviation to type "dir something" instead of > "zpaqfranz dir something" Ok, in *that* case, an (off by default) option to install the symlink could be an alternative.
Created attachment 236056 [details] 0001-archivers-zpaqfranz-Add-option-for-installing-dir Here's my suggestion to just add an option for bin/dir
(In reply to Felix Palmen from comment #36) This seems good, but maybe I would operate in a negated way (opposite) That is, I would remove it (the symlink) from the "internal" Makefile, and put the option as in the "external" Makefile aka: by default, no symlink; if you really want => check the option => done (in the "external" Makefile) Removing the symlink after it has been created (for zpaqfranz) would delete the one created by the conflicting package, or at least I believe so (I haven't studied it yet) ======== What happens BTS is essentially this #ifdef unix /// If I pretend to be... dir... work like dir! string myexename=extractfilename(zpaqfranzexename); if (myexename=="dir") { files.clear(); int i=0; while (++i<argc && argv[i][0]!='-') // read filename args files.push_back(argv[i]); return dir(); } #endif And you get this root@aserver:~ # dir /root/ ==== Scanning dir <</root/>> 22/08/2022 09:21 <DIR> /root/ 11/08/2017 18:04 <DIR> /root/.cache/ 17/10/2018 15:05 <DIR> /root/.config/ 20/10/2018 17:00 957 /root/.cshrc 09/09/2020 17:24 96 /root/.detoxrc (...) instead of root@aserver:~ # zpaqfranz dir /root/ zpaqfranz v55.2c-experimental archiver, compiled Jul 14 2022 ==== Scanning dir <</root/>> 22/08/2022 09:21 <DIR> /root/ 11/08/2017 18:04 <DIR> /root/.cache/ 17/10/2018 15:05 <DIR> /root/.config/ 20/10/2018 17:00 957 /root/.cshrc 09/09/2020 17:24 96 /root/.detoxrc (...)
(In reply to Franco Corbelli from comment #37) > Removing the symlink after it has been created (for zpaqfranz) would delete > the one created by the conflicting package, or at least I believe so No it wouldn't, that's why we have a STAGEDIR for all packaging. But I forgot something, updated patch coming.
Created attachment 236057 [details] 0001-archivers-zpaqfranz-Add-option-for-installing-dir Updated to include the CONFLICTS_INSTALL when bin/dir is installed. If you're fine with that change, please go to the patch details and set the maintainer-approval flag to '+'.
Comment on attachment 236057 [details] 0001-archivers-zpaqfranz-Add-option-for-installing-dir I think this is OK
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=85b9e25367ffc7e74cff4e9800fb02232679220e commit 85b9e25367ffc7e74cff4e9800fb02232679220e Author: Felix Palmen <zirias@FreeBSD.org> AuthorDate: 2022-08-22 11:27:18 +0000 Commit: Felix Palmen <zirias@FreeBSD.org> CommitDate: 2022-08-22 16:20:22 +0000 archivers/zpaqfranz: Add option for installing dir bin/dir is just a shorthand for a subcommand of zpaqfranz. Installing it would conflict with misc/gnuls, so make it optional and disabled by default. PR: 265651 Reported by: diizzy (on IRC) Approved by: Franco Corbelli <franco@francocorbelli.com> (maintainer) Approved by: tcberner (mentor) Differential Revision: https://reviews.freebsd.org/D36292 archivers/zpaqfranz/Makefile | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)