Testing v84 on my stable/10 desktop. make test reports in the final summary: 14 failed, 257 passed, 85 skipped, 1 warnings in 118.02 seconds tests/test_presenters.py:126: AssertionError tests/comparators/test_device.py:50: AssertionError tests/comparators/test_device.py:55: AssertionError tests/comparators/test_elf.py:54: AssertionError tests/comparators/test_elf.py:59: AssertionError tests/comparators/test_elf.py:88: AssertionError tests/comparators/test_elf.py:102: AssertionError tests/comparators/test_iso9660.py:47: IndexError tests/comparators/test_iso9660.py:52: IndexError tests/comparators/test_iso9660.py:56: IndexError tests/comparators/test_iso9660.py:62: IndexError tests/comparators/test_iso9660.py:72: IndexError tests/comparators/test_pcap.py:49: AssertionError tests/comparators/test_rlib.py:69: IndexError This is really an upstream issue, but submitting this PR for tracking and reference as we import newer diffoscope versions (while still working to incorporate fixes upstream).
Are we creating/tracking issues upstream as well or doing it here instead due to upstream not being easy to submit issues for?
> Are we creating/tracking issues upstream as well or doing it here instead > due to upstream not being easy to submit issues for? Charlie or I will submit upstream bugs if appropriate as we triage the individual issues, although most likely we'll just be submitting patches. For example, the first issue we've triaged is due to an unfixed bug in our cdrtools port/package, described in PR220868.
Use sysutils/cdrtools-devel instead of the default sysutil/cdrtools can fix the following cases: tests/comparators/test_iso9660.py::test_symlink PASSED tests/comparators/test_iso9660.py::test_compressed_files PASSED tests/comparators/test_iso9660.py::test_compare_non_existing PASSED And can receive more info on these cases: tests/comparators/test_iso9660.py::test_iso9660_content FAILED tests/comparators/test_iso9660.py::test_iso9660_rockridge FAILED See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220868 for more info.
koobs, do you have any advice on handling cdrtools vs. cdrtools-devel? we'd rather have the latter installed when using diffoscope.
The following test failures have been fixed: tests/comparators/test_device.py:50 tests/comparators/test_device.py:55 Related commit: https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=1390b76
(In reply to Ed Maste from comment #4) Preference for using released versions of any dependent software (with backports of fixes if necessary) and encouraging upstream to release new bugfix versions, over -devel versions unless unavoidable.
CC cdrtools/cdrtools-devel maintainer (marius)
(In reply to Kubilay Kocak from comment #6) I note that the debian diffoscope package doesn't depend on cdrtools )removed from repository a while ago). Is there something else we could/should be using or is there more to this story (say for example: utility compatibility with freebsd) ?
(In reply to Kubilay Kocak from comment #8) From my knowledge, on Debian they use genisoimage. Need to confirm its compatibility on FreeBSD.
I did a bit more investigation to find out that other than the problems mentioned above, the rest failed cases are all caused by the different handlings between bsddiff and gnudiff (using gdiff instead will fix all the rest cases).
(In reply to Guangyuan (Charlie) Yang from comment #10) bapt@, what do you think we should do about this wrt BSD diff? We have a few possible paths for this specific port: - work on making BSD diff make the same choices about diff hunk selection as GNU diff - have diffoscope check for and prefer gdiff if it exists - have diffoscope require GNU diffutils be installed and use it unconditionally on FreeBSD
(In reply to Ed Maste from comment #11) (3) first, with (1) (gnudiff compat) as a longer term target
I will add code to upstream diffoscope to force using gdiff (GNU Diff) tool on FreeBSD then. Dependency list of sysutils/py-diffoscope should also be updated along with the upstream diffoscope update. Since PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220868 is Closed/FIXED, I don't see any other failed cases on my machine except for the ones described here: https://github.com/yzgyyang/diffoscope/issues/2.
(In reply to Guangyuan (Charlie) Yang from comment #13) The ability to set the diff (and other) tools via some setting, environment variable or similar would be more desirable and make for a more portable/usable diffoscope. Then the consumer (in this case, the port) can use/switch/test utilities without needing to patch code or create more upstream work
(In reply to Kubilay Kocak from comment #14) I think that might be more effort than it's worth. diffoscope was written to expect GNU diff, so we ought to just use GNU diff if possible.
A quick update regarding this issue: Tests should all pass except for the ones that expect GNU diff results instead of BSD diff. A bug report with patch is submitted and is awaiting upstream's response. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871029 > The ability to set the diff (and other) tools via some setting, environment variable or similar would be more desirable and make for a more portable/usable diffoscope. Had discussions with the developers of diffoscope, they agreed that an external config dictionary or similar should be added. More investigations are to be done, and no plan is available for now. See a similar issue here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=869868
Multiple commits have been made on upstream to "prefer tools that start with 'g' on a non-GNU system". So all the failed cases should be gone (at least on my computer, can't say about skipped tests though) upon the ports update of the next diffoscope release. One last thing: add textproc/diffutils to the dependency list.
Thanks Charlie - I assume that now we just need to wait for the next diffoscope release and then update the port.
A commit references this bug: Author: ygy Date: Wed Sep 27 21:19:00 UTC 2017 New revision: 450791 URL: https://svnweb.freebsd.org/changeset/ports/450791 Log: sysutils/py-diffoscope: Update to 86 PR: 220870 Approved by: koobs (maintainer) Differential Revision: https://reviews.freebsd.org/D12328 Changes: head/sysutils/py-diffoscope/Makefile head/sysutils/py-diffoscope/distinfo
Reported and fixed readelf version parsing errors in the upstream Version 90. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886963 As the codebase iterates, I may close this one and open another PR for latest errors and fixes.
On my laptop I currently see two test failures: % pytest-3.6 ============================= test session starts ============================== platform freebsd12 -- Python 3.6.5, pytest-3.6.4, py-1.6.0, pluggy-0.6.0 rootdir: /usr/home/emaste/src/diffoscope, inifile: collected 382 items / 2 errors ==================================== ERRORS ==================================== ________________ ERROR collecting tests/comparators/test_elf.py ________________ tests/comparators/test_elf.py:77: in <module> @skip_if_binutils_does_not_support_x86() tests/utils/tools.py:101: in skip_if_binutils_does_not_support_x86 'elf64-x86-64' not in get_supported_elf_formats(), tests/utils/tools.py:109: in get_supported_elf_formats ('objdump', '--info'), /usr/local/lib/python3.6/subprocess.py:336: in check_output **kwargs).stdout /usr/local/lib/python3.6/subprocess.py:418: in run output=stdout, stderr=stderr) E subprocess.CalledProcessError: Command '('objdump', '--info')' returned non-zero exit status 1. ------------------------------- Captured stderr -------------------------------- objdump: Unknown command line argument '--info'. Try: 'objdump -help' objdump: Did you mean '-df'? _______________ ERROR collecting tests/comparators/test_rlib.py ________________ tests/comparators/test_rlib.py:78: in <module> @skip_if_binutils_does_not_support_x86() tests/utils/tools.py:101: in skip_if_binutils_does_not_support_x86 'elf64-x86-64' not in get_supported_elf_formats(), tests/utils/tools.py:109: in get_supported_elf_formats ('objdump', '--info'), /usr/local/lib/python3.6/subprocess.py:336: in check_output **kwargs).stdout /usr/local/lib/python3.6/subprocess.py:418: in run output=stdout, stderr=stderr) E subprocess.CalledProcessError: Command '('objdump', '--info')' returned non-zero exit status 1. ------------------------------- Captured stderr -------------------------------- objdump: Unknown command line argument '--info'. Try: 'objdump -help' objdump: Did you mean '-df'? !!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!! =========================== 2 error in 3.08 seconds ============================
Oops, tests are not being run when those errors are encountered. PATH=/usr/local/bin:$PATH /usr/bin/time pytest reports: ============= 8 failed, 272 passed, 120 skipped in 482.29 seconds ============== 484.13 real 218.76 user 654.70 sys tests/test_main.py:161: AssertionError tests/comparators/test_berkeley_db.py:33: AssertionError tests/comparators/test_cpio.py:50: AssertionError tests/comparators/test_device.py:59: AssertionError tests/comparators/test_device.py:68: AssertionError tests/comparators/test_ppu.py:50: AssertionError tests/comparators/test_ppu.py:66: IndexError tests/comparators/test_ppu.py:71: tests/utils/nonexisting.py:32: AssertionError
Tried building in Cirrus-CI, https://cirrus-ci.com/task/5110258826477568 ============= 9 failed, 254 passed, 153 skipped in 188.06 seconds ============== 199.92 real 99.76 user 166.00 sys
^Triage: OBE. This port has been updated multipled times since 2017. sunpoet@ currently maintains it.