Bug 192744 - [stage] math/atlas: STAGEify, fix tmpfile leak during build
Summary: [stage] math/atlas: STAGEify, fix tmpfile leak during build
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: John Marino
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-17 16:08 UTC by Don Lewis
Modified: 2014-08-18 00:01 UTC (History)
1 user (show)

See Also:


Attachments
add stage support to math/atlas and misc other fixes (10.11 KB, patch)
2014-08-17 16:08 UTC, Don Lewis
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Don Lewis freebsd_committer freebsd_triage 2014-08-17 16:08:26 UTC
Created attachment 145927 [details]
add stage support to math/atlas and misc other fixes

Add stage support to math/atlas.

Add a patch that prevents leaking files in /var/tmp during build.

Add all the include files to the static pkg-plist and remove the Makefile code that did this on the fly.

When installing the shared libraries, use INSTALL_LIB instead of INSTALL_DATA.

Fix a portlint warning about IGNORE.

With these change, it is possible to get a clean build (other than a few portlint warnings) in poudriere if MANUAL_PACKAGE_BUILD is commented out.
Comment 1 John Marino freebsd_committer freebsd_triage 2014-08-17 18:51:02 UTC
I'm curious, why is this port set for MANUAL_PACKAGE_BUILD ?

<researches>

Optimizes for local machine? 
That can't be fixed?


I'd like to reset the maintainer too.  bf@ hasn't committed to this in two years and it would have been reset in the sweep anyway.  A non-staging PR must have blocked the reset.

If you want to be maintainer, great.  Otherwise we just "save" it but leave it unmaintained in case somebody wants to give it a home.
Comment 2 John Marino freebsd_committer freebsd_triage 2014-08-17 21:24:35 UTC
By the way, the original port masked a bunch of installation commands with "@".  You changed those lines but didn't remove the mask.

It's easy to overlook, I've done it myself.  We've asked to get that check into portlint to help.
Comment 3 John Marino freebsd_committer freebsd_triage 2014-08-17 21:26:05 UTC
oh, and I'm having to wrap the same lines to fit 80 columns too.
Comment 4 John Marino freebsd_committer freebsd_triage 2014-08-17 22:13:13 UTC
Is atlas supposed to take forever to build?  37 minutes already, but load only at 1.0.  I'm not sure if that's normal or not.
Comment 5 Don Lewis freebsd_committer freebsd_triage 2014-08-17 22:19:27 UTC
(In reply to John Marino from comment #1)
> I'm curious, why is this port set for MANUAL_PACKAGE_BUILD ?
> 
> <researches>
> 
> Optimizes for local machine? 
> That can't be fixed?

That's sort of the whole point with ATLAS (Automatically Tuned Linear Algebra Software).  It figures out a bunch of stuff about the machine, and then tries
a bunch of different implementations of the same operations and chooses the
fastest for inclusion in the library.

I'm not sure how many ports can use atlas, but none do by default.  For instance, math/octave defaults to math/blas + math/lapack.  That's fine unless you want the extra performance.

> I'd like to reset the maintainer too.  bf@ hasn't committed to this in two
> years and it would have been reset in the sweep anyway.  A non-staging PR
> must have blocked the reset.
>
> If you want to be maintainer, great.  Otherwise we just "save" it but leave
> it unmaintained in case somebody wants to give it a home.

I'm not sure I'm sufficiently motivated to be the maintainer.  I just jumped in because I didn't want the port to go away.  Other than that, I've only touched this port when it has broken for me.
Comment 6 Don Lewis freebsd_committer freebsd_triage 2014-08-17 22:28:21 UTC
(In reply to John Marino from comment #4)
> Is atlas supposed to take forever to build?  37 minutes already, but load
> only at 1.0.  I'm not sure if that's normal or not.

Yeah.

===>  Applying FreeBSD patches for atlas-3.8.4_4,1

        This port engages in a long, computationally-demanding tuning process.
        Builds can take a day or more on older hardware, and can temporarily
        occupy several hundred megabytes of disk space.  You are advised to
        disable all forms of power management or CPU throttling during the
        build, and to set operating parameters to static values at which you
        ultimately intend to use the software.  You should also avoid placing
        heavy loads on the system during the build, and temporarily disable
        any automatons that may do so.  If you fail to follow these guidelines,
        you may prolong the build and obtain libraries with sub-optimal
        performance.

My somewhat aged Athlon 64 X2 running in i386 mode takes about 1/2 hour.  It used to take a lot longer before the Athlon CPU detection code was fixed.

The INSTALL.txt file says this:
Install times vary widely, depending on whether ATLAS has architectural
defaults for your platform, and the speed of your compiler and computer.
Under gcc/linux, an install may take as little as 15 minutes for all four
types/precisions.  Under IRIX using SGI's rather slow (but high performance)
compilers, the install might take as long as four hours.  1-2 hours is probably
fairly typical.  However, the user is not required to enter any input during
the install phase, and all operations are logged, so it is safe to start the
install and ignore it until completion.
Comment 7 John Marino freebsd_committer freebsd_triage 2014-08-17 22:31:42 UTC
This is running on a core i5 which smokes an athlon.  strange...
Comment 8 Don Lewis freebsd_committer freebsd_triage 2014-08-17 22:35:25 UTC
(In reply to John Marino from comment #2)
> By the way, the original port masked a bunch of installation commands with
> "@".  You changed those lines but didn't remove the mask.
> 
> It's easy to overlook, I've done it myself.  We've asked to get that check
> into portlint to help.

I didn't know there was a policy, so I left it as is.

Of the remaining portlint changes, one was an obvious spoof.  The other two had to do with port options and by that point I was tired enough of messing with this port that I didn't feel like digging into that.  I spent way too much time dealing with plist issues because I didn't notice PORT_DOCS and PORT_DATA were both set to "*" because it was done in such an obscure way.  I couldn't figure
why I was getting dups in .PLIST.mktmp ...
Comment 9 Don Lewis freebsd_committer freebsd_triage 2014-08-17 22:43:10 UTC
(In reply to John Marino from comment #7)
> This is running on a core i5 which smokes an athlon.  strange...

This code is probably too old to know about an i5, so it ends up trying a bunch more options to optimize the algorithms.

Hmn, this is 3.8.4 and the latest release is 3.10.2.  Lots of changes in the meantime;
<http://sourceforge.net/projects/math-atlas/files/Stable/3.10.2/ChangeLog/download>
Comment 10 John Marino freebsd_committer freebsd_triage 2014-08-17 22:51:19 UTC
75 minutes and counting.  I so hope I didn't mess anything up to cause the staging to fail... :(
Comment 11 John Marino freebsd_committer freebsd_triage 2014-08-17 23:17:07 UTC
The good news: it built and passed stage checks.

The bad news: It has NOPORTDOCS in it and the svn hooks are blocking the commit.
Comment 12 John Marino freebsd_committer freebsd_triage 2014-08-17 23:18:32 UTC
NOPORTDATA rather.
Comment 13 commit-hook freebsd_committer freebsd_triage 2014-08-17 23:20:28 UTC
A commit references this bug:

Author: marino
Date: Sun Aug 17 23:19:28 UTC 2014
New revision: 365261
URL: http://svnweb.freebsd.org/changeset/ports/365261

Log:
  Stage math/atlas and reset maintainer

  PR:		192744
  Submitted by:	Don Lewis

Changes:
  head/math/atlas/Makefile
  head/math/atlas/files/patch-tmpnamleak
  head/math/atlas/pkg-plist
Comment 14 John Marino freebsd_committer freebsd_triage 2014-08-17 23:22:55 UTC
I just deleted the .if/.endif lines that contained NOPORTDATA so the commands are run unconditionally.

Thanks for staging this, Don!
Comment 15 Don Lewis freebsd_committer freebsd_triage 2014-08-18 00:01:31 UTC
(In reply to John Marino from comment #14)
> I just deleted the .if/.endif lines that contained NOPORTDATA so the
> commands are run unconditionally.

This should probably be changed as well (though I don't know if
unconditionally setting PORTDOCS would be ok).

Index: Makefile
===================================================================
--- Makefile	(revision 365261)
+++ Makefile	(working copy)
@@ -57,12 +57,12 @@
 LIB_DEPENDS+=	libblas.so:${PORTSDIR}/math/blas
 .endif
 
-.for i in DATA DOCS
-.  if !defined(NOPORT${i})
-PORT${i}=	*
-.  endif
-.endfor
+.if ${PORT_OPTIONS:MDOCS}
+PORTDOCS=	*
+.endif
 
+PORTDATA=	*
+
 .if ${PORT_OPTIONS:MARCHDEF}
 .  if defined(ARCHDEF)
 .    if ${ARCHDEF} == "NONE"


> Thanks for staging this, Don!

So should I tackle atlas-devel?