Bug 200818

Summary: [maintainer update] for graphics/openimageio
Product: Ports & Packages Reporter: Shane <FreeBSD>
Component: Individual Port(s)Assignee: Marcus von Appen <mva>
Status: Closed FIXED    
Severity: Affects Only Me CC: mva
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
update openimageio to 1.5.16
none
update openimageio to 1.5.16
none
Fix simd includes none

Description Shane 2015-06-12 13:53:05 UTC
Created attachment 157677 [details]
update openimageio to 1.5.16

This updates graphics/openimageio to v1.5.16

Also includes a small update to the slave port graphics/py-openimageio
Comment 1 Shane 2015-06-12 14:33:07 UTC
bug #198330 can also be closed with this, the openssl dependency has been turned off based on feedback from the main oiio developer.
Comment 2 Marcus von Appen freebsd_committer freebsd_triage 2015-06-14 11:01:48 UTC
Could you make GIF_DESC and RAW_DESC more elaborate about what they are for (probably enabling GIF image and RAW image support?)
Comment 3 Shane 2015-06-15 02:14:24 UTC
Created attachment 157744 [details]
update openimageio to 1.5.16

make option descriptions a little more informative
Comment 4 Marcus von Appen freebsd_committer freebsd_triage 2015-06-19 07:07:14 UTC
LIB_DEPENDS within the SLAVE_PORT conditionals needs a +=, otherwise the OPTION lib depends are not picked up correctly.

I'm getting a SSSE3 error on RELENG_10 with clang 3.4.1:

[  6%] Building CXX object src/libutil/CMakeFiles/OpenImageIO_Util.dir/plugin.cpp.o
In file included from /usr/ports/graphics/openimageio/work/oiio-Release-1.5.16/src/libOpenImageIO/imagebuf.cpp:52:
In file included from /usr/ports/graphics/openimageio/work/oiio-Release-1.5.16/src/include/OpenImageIO/simd.h:57:
/usr/include/clang/3.4.1/tmmintrin.h:28:2: error: "SSSE3 instruction set not enabled"
#error "SSSE3 instruction set not enabled"
[...]

My arch should support it:

FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Core(TM)2 Quad CPU    Q8300  @ 2.50GHz (2493.80-MHz K8-class CPU)
[...]
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xc08e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,OSXSAVE>
  AMD Features=0x20100800<SYSCALL,NX,LM>
[...]
Comment 5 Marcus von Appen freebsd_committer freebsd_triage 2015-06-20 07:21:16 UTC
(In reply to Marcus von Appen from comment #4)
> 
> I'm getting a SSSE3 error on RELENG_10 with clang 3.4.1:
> 

OpenImageIO seems to require USE_SIMD to be set by the user or build infrastructure and does not do any auto-detection. This could be achieved with something like

.if ${MACHINE_CPU}
.if ${MACHINE_CPU:Msse2}
_SIMD= sse2
.endif
.if ${MACHINE_CPU:Msse3}
_SIMD= sse3
.endif
...
.endif
.if ${_SIMD} == ""
_SIMD= 0
.endif
CMAKE_ARGS+= -DUSE_SIMD:STRING="${_SIMD}"

The problem however is that clang will support features based on tests, which bsd.cpu.mk defines a set of fixed features (e.g. my CPU supports SSSE3 and SSE41, but bsd.cpu.mk only reports SSE2 and SSE3, since it's set to "nocona").

I'd assume that the most safe approach is to disable SIMD (USE_SIMD="0") until upstream implements a proper feature detection (blender's cmake build does a good job here).
Comment 6 Shane 2015-06-22 07:15:35 UTC
I can't find how you get the failures you have.

I have a corei5 running 10-stable, host and poudriere using 10.0 and 10.1, amd64 and x86 build without error.

I also have a core2 duo running current, host as well as poudriere 10.1 amd64 and x86 also build without error.

poudriere 9.3 using gcc and clang tests also build fine.

Looking over the cmake files, simd compiler flags are added if specifically enabled by setting USE_SIMD, while setting USE_SIMD to 0 would force disable all simd usage even if the compiler says simd is available.

Leaving USE_SIMD empty means simd usage is based on compiler settings configured outside of the port file. This should allow the user to set CFLAGS/CXXFLAGS to enable simd features matching their cpu or to work with features based on compiler detected options.

Are you sure you don't have some old testing CFLAGS/CXXFLAGS set that are breaking your build?

Some ports have a SIMD/SSE/SSE3 option, so another possibility is to add SSE SSE3 SSE4 options to allow the user to manually enable simd configuration. With this approach I would go with a simd group that the user could choose which simd level to use.
Comment 7 Marcus von Appen freebsd_committer freebsd_triage 2015-06-22 09:34:43 UTC
(In reply to FreeBSD from comment #6)
[...]
>
> Looking over the cmake files, simd compiler flags are added if specifically
> enabled by setting USE_SIMD, while setting USE_SIMD to 0 would force disable
> all simd usage even if the compiler says simd is available.
> 
> Leaving USE_SIMD empty means simd usage is based on compiler settings
> configured outside of the port file. This should allow the user to set
> CFLAGS/CXXFLAGS to enable simd features matching their cpu or to work with
> features based on compiler detected options.
>
>
> Are you sure you don't have some old testing CFLAGS/CXXFLAGS set that are
> breaking your build?

Yes. I think the problem relates to the following code in OpenImageIO/simd.h:

#  if defined(__SSE3__) || defined(__SSSE3__)
#    include <pmmintrin.h>
#    include <tmmintrin.h>
#  endif

tmmintrin.h contains the relevant SSSE3 functions. If SSSE3 is not available (but SSE3 is), the compiler bails out. I'll test a small change to the header later on and also will try without CPUTYPE=nocona (but CPUTYPE=native) instead

> Some ports have a SIMD/SSE/SSE3 option, so another possibility is to add SSE
> SSE3 SSE4 options to allow the user to manually enable simd configuration.
>
> With this approach I would go with a simd group that the user could choose
> which simd level to use.

This is risky, if the CPU, libc, etc. do not support the instructions, since the resulting library may break.
Comment 8 Shane 2015-06-23 12:03:31 UTC
(In reply to Marcus von Appen from comment #7)

> will try without CPUTYPE=nocona (but CPUTYPE=native) instead

Setting to nocona is what is breaking your build, nocona has SSE3 but not SSSE3.

So that section should be -

#  if defined(__SSE3__)
#    include <pmmintrin.h>
#  endif
#  if defined(__SSSE3__)
#    include <tmmintrin.h>
#  endif

>> Some ports have a SIMD/SSE/SSE3 option, so another possibility is to add SSE
>> SSE3 SSE4 options to allow the user to manually enable simd configuration.
>>
>> With this approach I would go with a simd group that the user could choose
>> which simd level to use.

>This is risky, if the CPU, libc, etc. do not support the instructions, since the >resulting library may break.

Official binary pkgs would not have simd enabled, the user would turn on an option and build for their computer, if they break it they choose a different option. Automating the choice of simd could break the binary pkgs.
Comment 9 Shane 2015-06-23 12:26:02 UTC
Created attachment 158012 [details]
Fix simd includes

An additional patch file to fix the simd inclusions when cpu is nocona.
Comment 10 Shane 2015-06-23 12:38:39 UTC
Thinking more about adding simd options to the port, I think it would be better to leave as is - letting the user adjust it in their make.conf. A user that wants to compile binaries optimized for their machine can add CPUTYPE=corei7 or CFLAGS+=-march=corei7 or similar to /etc/make.conf and add the benefit to every binary they compile.

The USE_SIMD in openimageio's CMakeLists.txt only adds -msse3 type flags for the compiler, so it's no different than the users global options being carried into the build. With the exception of USE_SIMD=0 which disables any simd includes.
Comment 11 commit-hook freebsd_committer freebsd_triage 2015-07-08 16:45:28 UTC
A commit references this bug:

Author: mva
Date: Wed Jul  8 16:45:22 UTC 2015
New revision: 391571
URL: https://svnweb.freebsd.org/changeset/ports/391571

Log:
  - Update to version 1.5.16

  OpenImageIO 1.5 comes with many new features and bug fixes:
  * lots of new functions for image handling (API and command line tools),
    such as rotations, median filters, matrix transformations
  * new filters (cubic, rifman, simon, ...)
  * image metadata support (EXIF)
  * SIMD (SSE) instruction support to speed up processing images
  * basic movie file support via ffmpeg

  Changelog: https://github.com/OpenImageIO/oiio/blob/RB-1.5/CHANGES

  PR:		200818
  Submitted by:	Shane Ambler <FreeBSD@ShaneWare.Biz>

Changes:
  head/graphics/openimageio/Makefile
  head/graphics/openimageio/distinfo
  head/graphics/openimageio/files/
  head/graphics/openimageio/files/patch-src_include_OpenImageIO_simd.h
  head/graphics/openimageio/pkg-plist
Comment 12 commit-hook freebsd_committer freebsd_triage 2015-07-08 16:47:30 UTC
A commit references this bug:

Author: mva
Date: Wed Jul  8 16:46:58 UTC 2015
New revision: 391572
URL: https://svnweb.freebsd.org/changeset/ports/391572

Log:
  - Forgot py-openimageio on the previous openimageio commit

  PR:		200818
  Submitted by:	Shane Ambler <FreeBSD@ShaneWare.Biz>

Changes:
  head/graphics/py-openimageio/Makefile
Comment 13 Marcus von Appen freebsd_committer freebsd_triage 2015-07-08 17:01:57 UTC
Committed with minor modifications (sync'd DESCs with the offered defaults), thanks!