Bug 192949 - [exp-run] packaged ruby gems contain source and intermediate build output
Summary: [exp-run] packaged ruby gems contain source and intermediate build output
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Steve Wills
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-23 18:37 UTC by Steve Wills
Modified: 2016-02-02 15:34 UTC (History)
3 users (show)

See Also:


Attachments
partial patch from antoine (1001 bytes, text/plain)
2014-08-27 20:48 UTC, Steve Wills
no flags Details
gem cleanup patch (1.14 KB, patch)
2015-07-24 14:26 UTC, Steve Wills
no flags Details | Diff
Updated cleanup patch that fixes fallout (13.97 KB, patch)
2015-07-26 21:42 UTC, Steve Wills
no flags Details | Diff
regenerated patch (12.93 KB, patch)
2016-01-17 07:37 UTC, Steve Wills
no flags Details | Diff
updated patch (18.58 KB, patch)
2016-01-20 04:10 UTC, Steve Wills
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Steve Wills freebsd_committer freebsd_triage 2014-08-23 18:37:49 UTC
Packaged gems contain source for the gem and intermediate build output. This stuff isn't necessary, so we need to make changes to bsd.ruby.mk to clean this stuff up before packaging. Creating this bug for myself as a reminder.
Comment 1 Steve Wills freebsd_committer freebsd_triage 2014-08-27 20:48:24 UTC
Created attachment 146406 [details]
partial patch from antoine

something like this is a start
Comment 2 Steve Wills freebsd_committer freebsd_triage 2015-07-24 14:26:46 UTC
Created attachment 159165 [details]
gem cleanup patch

Here's an updated patch that does a number of cleanups:

- strip .so files in or under GEM_LIB_DIR
- delete logs
- delete the gem cache dir
- cleans up the "ext" subdir of GEM_LIB_DIR

There are still some more things that could be cleaned up, lots of gems leave lots of stuff we don't really want to package in the GEM_LIB_DIR. Rather than build an ever growing blacklist of things that want to delete, I am going to try to come up with the right white list of things we probably want to package and delete the rest, but that's for future work.
Comment 3 Steve Wills freebsd_committer freebsd_triage 2015-07-26 21:42:12 UTC
Created attachment 159278 [details]
Updated cleanup patch that fixes fallout

Here's an updated version.

Deleting gem.build_complete is no good because gem checks for it at run time.

There were a few other minor issues, mainly around gems that use .h files from other gems (ruby-gnome stuff and nokogiri/nokogumbo). This updated patch fixes those. I think this is the correct way to do it, but I'm not 100% sure yet.

Would like some folks to test this, think about it, and perhaps exp-run it before committing. Or a larger sampling of gems that use other gems .h files might help.
Comment 4 Steve Wills freebsd_committer freebsd_triage 2016-01-09 21:46:53 UTC
Please exp-run this to see if it causes any issues. If the patch is too out of date to apply, let me know and I'll update it (or just apply the bsd.ruby.mk part if that works).
Comment 5 Steve Wills freebsd_committer freebsd_triage 2016-01-17 07:37:05 UTC
Created attachment 165696 [details]
regenerated patch

Here's an updated patch that was regenerated against a current ports tree
Comment 6 Antoine Brodin freebsd_committer freebsd_triage 2016-01-17 13:50:52 UTC
Exp-run results:

http://package23.nyi.freebsd.org/build.html?mastername=102amd64-default-PR192949&build=2016-01-17_13h39m39s

6 new failures, and 20 new ports skipped:

+ {"origin"=>"devel/rubygem-cairo-gobject", "pkgname"=>"rubygem-cairo-gobject-2.2.5", "phase"=>"stage", "errortype"=>"makefile"}
+ {"origin"=>"devel/rubygem-rbtrace", "pkgname"=>"rubygem-rbtrace-0.4.7", "phase"=>"package", "errortype"=>"???"}
+ {"origin"=>"graphics/rubygem-rsvg2", "pkgname"=>"rubygem-rsvg2-2.2.5", "phase"=>"stage", "errortype"=>"makefile"}
+ {"origin"=>"mail/milter-manager", "pkgname"=>"milter-manager-2.0.7", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"x11-toolkits/rubygem-pango", "pkgname"=>"rubygem-pango-2.2.5", "phase"=>"stage", "errortype"=>"makefile"}
+ {"origin"=>"x11-toolkits/rubygem-poppler", "pkgname"=>"rubygem-poppler-2.2.5_1", "phase"=>"stage", "errortype"=>"makefile"}

Failure logs:

http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/rubygem-cairo-gobject-2.2.5.log
http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/rubygem-rbtrace-0.4.7.log
http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/rubygem-rsvg2-2.2.5.log
http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/milter-manager-2.0.7.log
http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/rubygem-pango-2.2.5.log
http://package23.nyi.freebsd.org/data/102amd64-default-PR192949/2016-01-17_13h39m39s/logs/errors/rubygem-poppler-2.2.5_1.log
Comment 7 Antoine Brodin freebsd_committer freebsd_triage 2016-01-17 20:28:06 UTC
wouldn't it be better to patch mkmf-gnome2.rb from devel/rubygem-glib2 so that it picks header from the good place instead of manually adding CFLAGS?
Comment 8 Steve Wills freebsd_committer freebsd_triage 2016-01-20 04:10:19 UTC
Created attachment 165846 [details]
updated patch

You're right. I went back and did it that way but still had to patch a few things. Still, this seems like the "right way". So here's an updated patch that does that and also fixes the build issues the exp-run found. Please test again.
Comment 9 Antoine Brodin freebsd_committer freebsd_triage 2016-01-22 19:37:05 UTC
With the updated patch:

x11-toolkits/rubygem-gtk3 fails to patch

Those 6 ports are skipped:

graphics/rubygem-clutter-gtk graphics/rubygem-goocanvas www/rubygem-webkit-gtk x11-toolkits/rubygem-gtksourceview3 x11-toolkits/rubygem-vte3 x11/rubygem-gnome2
Comment 10 Antoine Brodin freebsd_committer freebsd_triage 2016-01-25 06:32:52 UTC
Exp-run results with the last version of the patch:

http://package18.nyi.freebsd.org/build.html?mastername=102amd64-default-PR192949&build=2016-01-25_06h26m04s

Everything looks fine.

I think the patched ports (devel/rubygem-glib2..) should be bumped
Comment 11 commit-hook freebsd_committer freebsd_triage 2016-02-02 01:22:25 UTC
A commit references this bug:

Author: swills
Date: Tue Feb  2 01:21:26 UTC 2016
New revision: 407788
URL: https://svnweb.freebsd.org/changeset/ports/407788

Log:
  Remove source and intermediate build output from intalled rubygem- ports

  PR:		192949

Changes:
  head/Mk/bsd.ruby.mk
  head/accessibility/rubygem-atk/Makefile
  head/devel/rubygem-cairo-gobject/Makefile
  head/devel/rubygem-cairo-gobject/files/
  head/devel/rubygem-cairo-gobject/files/patch-ext_cairo-gobject_extconf.rb
  head/devel/rubygem-gio2/Makefile
  head/devel/rubygem-glib2/Makefile
  head/devel/rubygem-glib2/files/
  head/devel/rubygem-glib2/files/patch-lib_mkmf-gnome2.rb
  head/devel/rubygem-gobject-introspection/Makefile
  head/devel/rubygem-pkg-config/Makefile
  head/devel/rubygem-rbtrace/Makefile
  head/graphics/rubygem-cairo/Makefile
  head/graphics/rubygem-clutter/Makefile
  head/graphics/rubygem-clutter-gtk/Makefile
  head/graphics/rubygem-gdk_pixbuf2/Makefile
  head/graphics/rubygem-goocanvas/files/patch-ext_goocanvas_extconf.rb
  head/graphics/rubygem-rsvg2/Makefile
  head/graphics/rubygem-rsvg2/files/
  head/graphics/rubygem-rsvg2/files/patch-ext_rsvg2_extconf.rb
  head/mail/milter-manager/Makefile
  head/mail/milter-manager/files/patch-configure
  head/multimedia/rubygem-clutter-gstreamer/Makefile
  head/multimedia/rubygem-gstreamer/Makefile
  head/textproc/rubygem-nokogiri/files/patch-ext_nokogiri_extconf.rb
  head/textproc/rubygem-nokogumbo/files/
  head/textproc/rubygem-nokogumbo/files/patch-ext_nokogumboc_extconf.rb
  head/www/rubygem-webkit-gtk/Makefile
  head/www/rubygem-webkit-gtk2/Makefile
  head/x11/rubygem-gnome2/Makefile
  head/x11-toolkits/rubygem-gdk3/Makefile
  head/x11-toolkits/rubygem-gtk2/Makefile
  head/x11-toolkits/rubygem-gtk2/files/
  head/x11-toolkits/rubygem-gtk2/files/patch-ext_gtk2_extconf.rb
  head/x11-toolkits/rubygem-gtk3/Makefile
  head/x11-toolkits/rubygem-gtk3/files/
  head/x11-toolkits/rubygem-gtk3/files/patch-ext_gtk3_extconf.rb
  head/x11-toolkits/rubygem-gtksourceview2/Makefile
  head/x11-toolkits/rubygem-gtksourceview2/files/
  head/x11-toolkits/rubygem-gtksourceview2/files/patch-ext_gtksourceview2_extconf.rb
  head/x11-toolkits/rubygem-gtksourceview3/Makefile
  head/x11-toolkits/rubygem-pango/Makefile
  head/x11-toolkits/rubygem-pango/files/
  head/x11-toolkits/rubygem-pango/files/patch-ext_pango_extconf.rb
  head/x11-toolkits/rubygem-poppler/Makefile
  head/x11-toolkits/rubygem-poppler/files/patch-ext_poppler_extconf.rb
  head/x11-toolkits/rubygem-vte/Makefile
Comment 12 Steve Wills freebsd_committer freebsd_triage 2016-02-02 01:22:49 UTC
Patch committed, closing.
Comment 13 Steve Wills freebsd_committer freebsd_triage 2016-02-02 15:34:03 UTC
Actually close