Bug 247035 - multimedia/x265: Update to 3.4
Summary: multimedia/x265: Update to 3.4
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Mikhail Teterin
URL:
Keywords: buildisok
Depends on:
Blocks:
 
Reported: 2020-06-06 19:14 UTC by daniel.engberg.lists
Modified: 2020-09-21 21:52 UTC (History)
8 users (show)

See Also:
pi: maintainer-feedback+


Attachments
Patch for x265 (WIP) (12.08 KB, patch)
2020-06-06 19:14 UTC, daniel.engberg.lists
no flags Details | Diff
Updated diff, with VMAF-build fixed (13.90 KB, patch)
2020-07-19 21:59 UTC, Mikhail Teterin
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description daniel.engberg.lists 2020-06-06 19:14:00 UTC
Created attachment 215300 [details]
Patch for x265 (WIP)

Remove outdated mirrors
Make assembly optimization optional (default on AMD64)
Fix version info (OS on all targets and 64-bit on aarch64)

Tested on FreeBSD 13.0-CURRENT r361421 amd64
Options: ASM, no ASM, SVT (compile) (8+10+12bit)
"make test" with ASM (8+10+12bit) and ASM+SVT

Tested on FreeBSD 13.0-CURRENT r361660 (aarch64)
Options: ASM, no ASM (compile) (8bit), (8+10+12bit)
"make test" with ASM (8bit)
Comment 1 daniel.engberg.lists 2020-06-06 19:15:20 UTC
I can't seem to get assembly to work using binutils and clang on aarch64, I tried passing -no-integrated-as (using CFLAGS) but it barfs on "error: unknown directive" for .func . For now using GCC is the only solution.

Only 8-bit works with assembly on aarch64 (addressed in port)
Ref: https://bitbucket.org/multicoreware/x265/issues/549/fail-to-build-for-aarch64-and-armhf

vamf is broken I can't figure out how to fix it, error on amd64:

FAILED: CMakeFiles/cli.dir/abrEncApp.cpp.o
/usr/bin/c++  -DENABLE_HDR10_PLUS -DENABLE_LIBVMAF -DEXPORT_C_API=1 -DHAVE_INT_TYPES_H=1 -DHIGH_BIT_DEPTH=0 -DX265_ARCH_X86=1 -DX265_DEPTH=8 -DX265_NS=x265 -DX86_64=1 -D__STDC_LIMIT_MACROS=1 -I/usr/ports/multimedia/x265/work/x265_3.4/source/. -I/usr/ports/multimedia/x265/work/x265_3.4/source/dynamicHDR10 -I. -I/usr/ports/multimedia/x265/work/x265_3.4/source/common -I/usr/ports/multimedia/x265/work/x265_3.4/source/encoder -O2 -pipe -march=ivybridge -DNDEBUG -O3 -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -isystem /usr/local/include -O2 -pipe -march=ivybridge -DNDEBUG -O3 -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -isystem /usr/local/include   -Wall -Wextra -Wshadow -std=gnu++11 -fPIC -Wno-array-bounds -ffast-math -mstackrealign -fno-exceptions -MD -MT CMakeFiles/cli.dir/abrEncApp.cpp.o -MF CMakeFiles/cli.dir/abrEncApp.cpp.o.d -o CMakeFiles/cli.dir/abrEncApp.cpp.o -c /usr/ports/multimedia/x265/work/x265_3.4/source/abrEncApp.cpp
/usr/ports/multimedia/x265/work/x265_3.4/source/abrEncApp.cpp:817:59: error: no member named 'argCount' in 'x265::CLIOptions'; did you mean 'argCnt'?
                api->vmaf_encoder_log(m_encoder, m_cliopt.argCount, m_cliopt.argString, m_cliopt.param, vmafdata);
                                                          ^~~~~~~~
                                                          argCnt
/usr/ports/multimedia/x265/work/x265_3.4/source/./x265cli.h:404:13: note: 'argCnt' declared here
        int argCnt;
            ^
1 error generated.
Comment 2 Steve Wills freebsd_committer 2020-06-06 19:47:01 UTC
Build info is available at https://gitlab.com/swills/freebsd-ports/pipelines/153601729
Comment 3 Kurt Jaeger freebsd_committer 2020-06-27 15:44:50 UTC
testbuilds are fine on cur-amd64, cur-i386, 12.1-amd64, 11.4-amd64.
Comment 4 Mikhail Teterin freebsd_committer 2020-07-19 21:29:52 UTC
Thanks, Daniel. I'm running my own build right now.

Would you like to become the port's maintainer too?
Comment 5 Mikhail Teterin freebsd_committer 2020-07-19 21:59:18 UTC
Created attachment 216585 [details]
Updated diff, with VMAF-build fixed

I may have fixed the VMAF build-problems. Could someone, please, test this updated diff?
Comment 6 daniel.engberg.lists 2020-08-02 16:35:47 UTC
Seems to work fine on my end, at least doing some encodes using ffmpeg. Sorry for the late reply.
Comment 7 commit-hook freebsd_committer 2020-09-20 22:18:02 UTC
A commit references this bug:

Author: mi
Date: Sun Sep 20 22:17:54 UTC 2020
New revision: 549401
URL: https://svnweb.freebsd.org/changeset/ports/549401

Log:
  Upgrade from 3.2 to 3.4. Resolve incompatibilities with newer
  SVTHEVC.

  PR:		247035, 248479
  Sponsored by:	United Marsupials

Changes:
  head/multimedia/x265/Makefile
  head/multimedia/x265/distinfo
  head/multimedia/x265/files/patch-source_CMakeLists.txt
  head/multimedia/x265/files/patch-source_abrEncApp.cpp
  head/multimedia/x265/files/patch-source_cmake_Findsvthevc.cmake
  head/multimedia/x265/files/patch-source_common_version.cpp
  head/multimedia/x265/files/patch-source_dynamicHDR10_CMakeLists.txt
  head/multimedia/x265/files/patch-source_encoder_api.cpp
  head/multimedia/x265/files/patch-source_encoder_svt.h
  head/multimedia/x265/pkg-plist
Comment 8 Tomoaki AOKI 2020-09-21 01:35:45 UTC
(In reply to commit-hook from comment #7)

Cannot fetch.
Upstream seems to move to [1], but there is no 3.4 tarball.
The latest tarball is currently 3.3 there.

The site seems to move to git interface, so we would need cloning (pulling?) the repo and use the proper branch, maybe [2] this time unless 3.4 tarball is generated later. (Possibly need fixing distinfo.)

Unfortunately, it is NEITHER github NOR gitlab. So helper in bsd.sites.mk would be unusable.

CAUTION: I have NOT tested, as I'm not enough familiar to git.

[1] https://bitbucket.org/multicoreware/x265_git/downloads/

[2] https://bitbucket.org/multicoreware/x265_git/src/Release_3.4/
Comment 9 John Kennedy 2020-09-21 04:07:33 UTC
I'll 2nd the "cannot fetch":

=> Attempting to fetch https://bitbucket.org/multicoreware/x265/downloads/x265_3.4.tar.gz
fetch: https://bitbucket.org/multicoreware/x265/downloads/x265_3.4.tar.gz: Not Found
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/x265_3.4.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/x265_3.4.tar.gz: Not Found

From pkg-descr, if you go to http://www.x265.org, downloads sends you to username/password prompt.  There are links from there to bitbucket.org, but some of them (wiki?) are dead, like https://bitbucket.org/multicoreware/x265/wiki/Contribute (via http://www.x265.org/developers).

https://bitbucket.org/multicoreware/x265 is dead, but links like https://bitbucket.org/multicoreware/x265_git/wiki/Home exist (NOTE:  x265_git, vs just x265).  But like other people have noted, x265_3.3.tar.gz is the most recent thing on there right now.

(https://bitbucket.org/multicoreware/x265_git/downloads/)
Comment 10 commit-hook freebsd_committer 2020-09-21 04:48:07 UTC
A commit references this bug:

Author: mi
Date: Mon Sep 21 04:47:34 UTC 2020
New revision: 549419
URL: https://svnweb.freebsd.org/changeset/ports/549419

Log:
  While we were testing the new code, upstream have removed the 3.4
  release from download area. Change our port to download using
  Bitbucket's method.

  Although the upstream branch Release_3.4 is very similar to what
  used to be available as 3.4, there are a few improvements -- in
  particular, one of our patches is no longer necessary, which is
  nice.

  PR:		247035
  Reported by:	John Kennedy, Tomoaki AOKI
  Sponsored by:	United Marsupials

Changes:
  head/multimedia/x265/Makefile
  head/multimedia/x265/distinfo
  head/multimedia/x265/files/patch-source_abrEncApp.cpp
Comment 11 VVD 2020-09-21 11:18:26 UTC
(In reply to commit-hook from comment #10)
/tmp/work/usr/ports/multimedia/x265/work/source/encoder/api.cpp:453:55: error: array type 'unsigned char [1024]' is not assignable
                    inputData->dolbyVisionRpu.payload = X265_MALLOC(uint8_t, 1024);
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/tmp/work/usr/ports/multimedia/x265/work/source/encoder/api.cpp:460:55: error: array type 'unsigned char [1024]' is not assignable
                    inputData->dolbyVisionRpu.payload = NULL;
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/tmp/work/usr/ports/multimedia/x265/work/source/encoder/api.cpp:698:43: warning: address of array 'inputData->dolbyVisionRpu.payload' will always evaluate to 'true' [-Wpointer-bool-conversion]
            if (inputData->dolbyVisionRpu.payload) X265_FREE(inputData->dolbyVisionRpu.payload);
            ~~  ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
/tmp/work/usr/ports/multimedia/x265/work/source/encoder/api.cpp:2054:39: error: array type 'unsigned char [1024]' is not assignable
    inputData->dolbyVisionRpu.payload = NULL;
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
1 warning and 3 errors generated.

12.1-p10 amd64.
        DEBUG          : off
        HI10P          : on
        HI12P          : on
        HI8P           : on
        OPTIMIZED_FLAGS: on
        SVTHEVC        : on
        VMAF           : off
Comment 12 John Kennedy 2020-09-21 14:40:24 UTC
(In reply to commit-hook from comment #10)

Following up after the #549419 commit, it pulled the file down just fine today:

=> x265-3.4.tar.gz doesn't seem to exist in /portdistfiles/.
=> Attempting to fetch https://bitbucket.org/multicoreware/x265_git/get/25b2c07035ff.tar.gz?meow=/x265-3.4.tar.gz
x265-3.4.tar.gz                                       1496 kB 1441 kBps    01s

It still doesn't exist at distcache.freebsd.org (assuming that it would show up there as the previous filename).
Comment 13 commit-hook freebsd_committer 2020-09-21 17:16:32 UTC
A commit references this bug:

Author: mi
Date: Mon Sep 21 17:15:50 UTC 2020
New revision: 549462
URL: https://svnweb.freebsd.org/changeset/ports/549462

Log:
  Actually commit the SVT-HEVC patch carelessly ommitted previously.

  PR:		247035, 248479
  Reported by:	VVD
  Sponsored by:	United Marsupials

Changes:
  head/multimedia/x265/files/patch-source_encoder_api.cpp
Comment 14 VVD 2020-09-21 21:52:10 UTC
Build fine for me.