Bug 258108 - [exp-run] devel/ruby-gems: Update to 3.3.7 (Fixes for Ruby 3.0)
Summary: [exp-run] devel/ruby-gems: Update to 3.3.7 (Fixes for Ruby 3.0)
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Dan Langille
URL:
Keywords: needs-qa, patch, regression
: 259491 260780 (view as bug list)
Depends on: 261637
Blocks: 260780
  Show dependency treegraph
 
Reported: 2021-08-27 23:38 UTC by mikeg
Modified: 2022-02-21 20:51 UTC (History)
11 users (show)

See Also:
bugzilla: maintainer-feedback? (ruby)
antoine: exp-run+


Attachments
Patch ruby-gems 3.0.2 -> 3.2.26 (8.09 KB, patch)
2021-08-27 23:38 UTC, mikeg
no flags Details | Diff
Patch ruby-gems 3.0.2 -> 3.2.26 (8.19 KB, patch)
2021-10-05 22:56 UTC, mikeg
no flags Details | Diff
list of orphans detected via poudriere testport (21.95 KB, text/plain)
2021-10-06 13:47 UTC, Dan Langille
no flags Details
update to 3.2.29 (10.84 KB, patch)
2021-10-13 13:51 UTC, Thibault Jouan
no flags Details | Diff
Workaround broken --destdir with bundler (856 bytes, patch)
2021-10-13 14:02 UTC, Thibault Jouan
no flags Details | Diff
update to 3.2.29 - with new plugins system handling (12.75 KB, patch)
2021-10-14 12:53 UTC, Thibault Jouan
no flags Details | Diff
ports failing to build with rubygems 3.2 (1.58 KB, text/plain)
2021-10-25 14:59 UTC, Thibault Jouan
no flags Details
update to 3.2.29 - with updates to gemspec patches in rubygems ports (251.92 KB, patch)
2021-10-25 15:18 UTC, Thibault Jouan
no flags Details | Diff
update to 3.2.30 (171.75 KB, patch)
2021-10-27 15:30 UTC, Thibault Jouan
no flags Details | Diff
update to 3.2.30 (167.00 KB, patch)
2021-10-29 09:12 UTC, Thibault Jouan
no flags Details | Diff
update to 3.2.30 (349.40 KB, patch)
2021-11-01 22:54 UTC, Thibault Jouan
no flags Details | Diff
update to 3.2.30 (347.98 KB, patch)
2021-12-07 16:58 UTC, Thibault Jouan
no flags Details | Diff
potential fix for patching issue (4.07 KB, patch)
2021-12-08 13:03 UTC, Thibault Jouan
no flags Details | Diff
Updated patch file (337.84 KB, patch)
2021-12-22 08:43 UTC, Yasuhiro Kimura
no flags Details | Diff
update to 3.3.6 (322.50 KB, patch)
2022-01-26 13:58 UTC, Thibault Jouan
no flags Details | Diff
update to 3.3.6 (324.49 KB, patch)
2022-01-27 11:30 UTC, Thibault Jouan
no flags Details | Diff
update to 3.3.7 (326.12 KB, patch)
2022-02-17 15:52 UTC, Thibault Jouan
no flags Details | Diff
update to 3.3.7 (330.20 KB, patch)
2022-02-19 13:02 UTC, Thibault Jouan
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description mikeg 2021-08-27 23:38:47 UTC
Created attachment 227498 [details]
Patch ruby-gems 3.0.2 -> 3.2.26

The attached patch updates devel/ruby-gems to 3.2.26 (Latest available as of 2021-08-27). This obviates 256080 (the port will no longer be broken).

I'm running gem installed from this port at work with Ruby 3.0, and on my personal box with Ruby 3.7 - seems to be working.

Needs QA - in particular:

The staging installed under /usr/local/bin/gem instead of /usr/local/bin/gem/${RUBY_VER_SHORT}
I hacked the Makefile to move and symlink it so it behaves "properly" (the way it previously did) but I feel like I'm missing something stupidly obvious that's causing it to misbehave in the first place.

The 3.2.26 install includes Bundler and bunch of Bundler-related "stuff" - I ripped that out of the packing list since we have sysutils/rubygem-bundler for this.
It doesn't appear to break anything with gem's functionality and sysutils/rubygem-bundler installs cleanly with this updated port so I think my packing list is clean, but please sanity check me here.
Comment 1 mikeg 2021-08-27 23:52:57 UTC
...and re: my first QA note - I uploaded an old patch with "mv" instead of "${MV}" because I forgot to re-stage the Makefile :/

If my hack isn't necessary that line should be removed from the patch anyway.
Comment 2 Dan Langille freebsd_committer freebsd_triage 2021-10-05 20:23:04 UTC
This patch does not apply cleanly:

[dvl@ava-pkg-02prd:/usr/dvl/ports-head/devel/ruby-gems] $ patch < ~/tmp/patch 
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- b/devel/ruby-gems/Makefile	
|+++ b/devel/ruby-gems/Makefile	
--------------------------
Patching file Makefile using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 failed at 19.
1 out of 2 hunks failed--saving rejects to Makefile.rej
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- b/devel/ruby-gems/distinfo	
|+++ b/devel/ruby-gems/distinfo	
--------------------------
Patching file distinfo using Plan A...
Hunk #1 succeeded at 1.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- b/devel/ruby-gems/pkg-plist	
|+++ b/devel/ruby-gems/pkg-plist	
--------------------------
Patching file pkg-plist using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 46.
Hunk #3 succeeded at 69.
Hunk #4 succeeded at 83.
Hunk #5 succeeded at 105.
Hunk #6 succeeded at 167.
Hunk #7 failed at 186.
1 out of 7 hunks failed--saving rejects to pkg-plist.rej
done
Comment 3 mikeg 2021-10-05 22:55:10 UTC
Hmm.... not sure if I cut a bad patch or head moved, but probably the former because I'm usually the one who screwed up when the computer yells.

Updated patch (2021100501) incoming (also fixes my mv -> ${MV} as noted previously) - looks clean on my end with a pull from git about 5 minutes ago:


$ patch < ~/rubygems-patch-2021100501
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/devel/ruby-gems/Makefile b/devel/ruby-gems/Makefile
|index 8f6eceea3a04..278c7421e29d 100644
|--- a/devel/ruby-gems/Makefile
|+++ b/devel/ruby-gems/Makefile
--------------------------
Patching file devel/ruby-gems/Makefile using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 19.
Hunk #3 succeeded at 58.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/devel/ruby-gems/distinfo b/devel/ruby-gems/distinfo
|index b39eec5b0356..6d133abd58c2 100644
|--- a/devel/ruby-gems/distinfo
|+++ b/devel/ruby-gems/distinfo
--------------------------
Patching file devel/ruby-gems/distinfo using Plan A...
Hunk #1 succeeded at 1.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git a/devel/ruby-gems/pkg-plist b/devel/ruby-gems/pkg-plist
|index 3e04be24e353..fcae759bfae0 100644
|--- a/devel/ruby-gems/pkg-plist
|+++ b/devel/ruby-gems/pkg-plist
--------------------------
Patching file devel/ruby-gems/pkg-plist using Plan A...
Hunk #1 succeeded at 1.
Hunk #2 succeeded at 46.
Hunk #3 succeeded at 69.
Hunk #4 succeeded at 83.
Hunk #5 succeeded at 105.
Hunk #6 succeeded at 167.
Hunk #7 succeeded at 186.
done
Comment 4 mikeg 2021-10-05 22:56:41 UTC
Created attachment 228469 [details]
Patch ruby-gems 3.0.2 -> 3.2.26
Comment 5 Dan Langille freebsd_committer freebsd_triage 2021-10-06 13:47:29 UTC
Created attachment 228480 [details]
list of orphans detected via poudriere testport

(In reply to mikeg from comment #4)

The new patch applies cleanly. There are a large amount of orphans detected during poudriere testport.
Comment 6 mikeg 2021-10-06 19:25:34 UTC
The crazy long list of bundler orphans are expected(ish) per my second QA note - I pulled these from the package in favor of the current status quo of "If you want bundler install sysutils/rubygem-bundler." since that was more current that what's packaged with gem.
(The stuff it's staging under /wrkdir is also an artifact of building Bundler.)

If you think it makes sense to install the Bundler that comes with gems instead to quiet the poudriere I can re-patch to pack Bundler and mark the two ports as conflicting - that might need a dependencies audit though.

I could also have the Makefile clean all of this up in a post-install if you think that's sensible, seemed dirty to me though. If I could get it to stop building Bundler entirely it should be a clean packing list.
Comment 7 Dan Langille freebsd_committer freebsd_triage 2021-10-06 20:30:18 UTC
I agree with you, since we have rubygem-bundle, let's not use that. At persent, the port does not do that.  Happy to test again.
Comment 8 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-10-10 15:17:54 UTC
If we try to update this port to 3.1 or later, we face two problems.

The first one is that since 3.1 setup.rb doesn't check if DONT_USE_BUNDLER_FOR_GEMDEPS environment variable is set. It means we can't install RubyGems without installing bundler any more. But fortunately RubyGems and bundler are released in sync since RubyGems 3.2.0 and bundler 2.2.0. So we can avoid this problem by unifying devel/ruby-gems and sysutils/rubygem-bundler.

The second one is that since 3.1 --destdir option of setup.rb has been broken. There are two report about it on upstream repository in GitHub.

https://github.com/rubygems/rubygems/pull/2768
https://github.com/rubygems/rubygems/issues/3604

To make --destdir option work correctly both need to be fixed.

Since the former is pull request, there is proposed patch to fix the problem. On the other hand, the latter is just issue report, and currently no patch is provided to fix it.
Comment 9 Thibault Jouan 2021-10-13 13:51:43 UTC
Created attachment 228666 [details]
update to 3.2.29

Hello,

  I hope I'm not overstepping, but I'm currently looking into RubyGems
3.2 update too. I tried with version 3.2.29, with the original
approach of removing bundler, but with as few manual changes as I
could (since as noted, we cannot rely on `USE_BUNDLER_FOR_GEMDEPS'
anymore).

  My detailed notes are in the patch, at the top. I usually test
patches (such as this one) for weeks/months before submitting or
publishing, but given that current RubyGems port is old, and that Ruby
3.1 will be out soon, I hoped that sharing this work could help.

  I have only tested a few dozens gems (half written by me), with
`lang/ruby27' and `lang/ruby30' for now, but will continue testing
over the next days. The most annoying issue from my point of view, is
the new `%%GEMS_BASE_DIR%%/plugins' directory created when installing
gem via the port system (during FreeBSD package build), I don't know
how to handle this (gems ported with RubyGems 3.0.8 will generate a
different package once `devel/ruby-gems' is updated to 3.2.29).
Comment 10 Thibault Jouan 2021-10-13 14:02:24 UTC
Created attachment 228667 [details]
Workaround broken --destdir with bundler

(In reply to Yasuhiro Kimura from comment #8)                          
> The second one is that since 3.1 --destdir option of setup.rb has been broken.  \ There are two report about it on upstream repository in GitHub.
>·                                                                     
> https://github.com/rubygems/rubygems/pull/2768                       
> https://github.com/rubygems/rubygems/issues/3604                     
>·                                                                     
> To make --destdir option work correctly both need to be fixed.       
                                                                       
  I looked into it, but only for a very short time. If I recall        
correctly, the new attached patch fix the "duplicated" dest dir in one 
of the paths, but there was a third dest dir issue in addition to      
those two already reported.                                            
                                                                       
  Also, I'm worried that even if it get fixed, it will break shortly   
after (it happened multiple times in the past already, still for       
`--destdir' option. Some people (myself included) trying to fix it     
upstream in the past seems to have given up and now prefer to patch    
their port directly (especially because they don't have to support     
Windows too in this case).
Comment 11 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-10-14 05:39:00 UTC
(In reply to Thibault Jouan from comment #9)

I tried your patch but it causes plist error as following if I build
rubygem ports.

----------------------------------------------------------------------
===========================================================================
====> Running Q/A tests (stage-qa)
====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: @dir %%GEMS_BASE_DIR%%/plugins
===> Checking for items in pkg-plist which are not in STAGEDIR
===> Error: Plist issues found.
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/rubygem-bundler
=>> Error: check-plist failures detected
----------------------------------------------------------------------

Full build log of some rubygem ports.

devel/rubygem-racc:
https://www.utahime.org/FreeBSD/poudriere/data/logs/bulk/130amd64-default/2021-10-14_14h31m09s/logs/rubygem-racc-1.5.2.log
devel/rubygem-rdoc:
https://www.utahime.org/FreeBSD/poudriere/data/logs/bulk/130amd64-default/2021-10-14_14h28m43s/logs/rubygem-rdoc-6.3.2.log
sysutils/rubygem-bundler:
https://www.utahime.org/FreeBSD/poudriere/data/logs/bulk/130amd64-default/2021-10-14_14h16m27s/logs/rubygem-bundler-2.2.28,1.log
Comment 12 Thibault Jouan 2021-10-14 12:38:50 UTC
(In reply to mikeg from comment #0)
> The staging installed under /usr/local/bin/gem instead of /usr/local/bin/gem/${RUBY_VER_SHORT}
> I hacked the Makefile to move and symlink it so it behaves "properly" (the way it previously did) but I feel like I'm missing something stupidly obvious that's causing it to misbehave in the first place.

  It's because of the new `--format-executable' option flag. This
behavior was previously hard-coded, now it is configurable but it has
the opposite behavior as before. The same thing happened in the past
when `--regenerate-binstubs' was introduced, and now with
`--regenerate-plugins' too.

  I didn't understood this part of your comment during my first read,
and later I forgot about it when posting my patch, sorry about that.
Comment 13 Thibault Jouan 2021-10-14 12:53:42 UTC
Created attachment 228695 [details]
update to 3.2.29 - with new plugins system handling

(In reply to Yasuhiro Kimura from comment #11)
> I tried your patch but it causes plist error as following if I build
> rubygem ports.

  Thank you for testing the patch and your feedback. With that version
of the patch, this error is expected. I didn't fully understand the
issue at the time, but I worked on it and made progress. Here are my
notes (full notes in the patch) about this specific issue:

    RubyGems reworked the plugins system (see upstream 27d786e8). There
    isn't a lot of info, but by looking at upstream change f5bd2fe0 we can
    guess that the idea is to generate wrappers in the new "global"
    plugins directory (${GEMS_BASE_DIR}/plugins) with code to require the
    actual plugin in the gem own directory. For example with the `yard'
    gem (`textproc/rubygem-yard' port):

      % cat /usr/local/lib/ruby/gems/2.7/plugins/yard_plugin.rb
      require_relative '../gems/yard-0.9.26/lib/rubygems_plugin.rb'

    We handle plugins like extensions: if the gem has no plugin, we remove
    the empty directory at the end of `do-install' target; if the gem has
    a plugin, we add it to the plist at the end of `gem-autoplist' target.

  Regarding the last part, in `gem-autoplist' target of
`Mk/Uses/gem.mk' file, I kept the usage of find but there is only
*one* file to find, maybe the path can be derived from the port name:
with port `yard' path is `plugins/yard_plugin.rb'. But I'd need to
check for special cases.

  With the new attached patch, the "new" plugin wrapper is
automatically included in the plist, gets in the package built from
the port, and RubyGems loads it successfully (I only tested with yard
however for now).
Comment 14 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-10-16 08:44:22 UTC
(In reply to Thibault Jouan from comment #13)

Thanks for quick fix. I confirmed build of rubygem ports succeeds with latest patch. Since updating this ports affects all rubygem ports in ports tree, It should be tested exp-run before committing.
Comment 15 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-10-16 09:07:14 UTC
I would like to ask for exp-run with attachment #228695 [details].
Comment 16 Antoine Brodin freebsd_committer freebsd_triage 2021-10-19 09:16:20 UTC
All ruby gems that have patches are failing.
Comment 17 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-10-19 09:22:43 UTC
(In reply to Antoine Brodin from comment #16)

Thanks for exp-run. Would you please provide some build failure logs?
Comment 18 Antoine Brodin freebsd_committer freebsd_triage 2021-10-19 09:25:55 UTC
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-minitar-cli-0.8_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-dm-types-1.2.2_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-shellplay-0.1.9_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-activesupport4-4.2.11.3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-astrolabe-1.3.1_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-cf-uaa-lib-3.13.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-chef-provisioning-2.7.6_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-chronic_duration-0.10.6.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-cucumber-messages4-4.0.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-dynflow-1.6.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-faraday_middleware-aws-signers-v4-0.2.0.b_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-flipper-ui017-0.17.2_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-gettext-setup-0.34_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-gitlab-pygments.rb-0.5.4_3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-kafo-6.4.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-mongo-2.15.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-ms_rest-0.7.6.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-opt_parse_validator-1.9.3_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-pleaserun-0.0.32.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-pry-byebug-3.9.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-rash-0.4.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-rubygems-mirror-1.3.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-ruport-1.7.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-sidekiq5-5.2.9.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-sidetiq-0.7.2_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-spring-watcher-listen-2.0.1_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-terrapin-0.6.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-tilt1-1.4.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-travis-1.10.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-tty-0.10.0_10.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-turn-0.9.7_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-fission-0.5.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-lolcat-100.0.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-gitlab_emoji-0.3.1_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-goocanvas-2.2.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-net-imap-0.2.2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-net-pop-0.1.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-net-smtp-0.2.2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-vmail-2.9.8_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-lita-4.8.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-oxidized-0.28.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-riemann-client-0.2.6_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-riemann-dash-0.2.14.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-amqp-utils-0.5.1_5.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-grpc130-1.30.2_3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-twitter-7.0.0_3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-twitter-stream-0.1.16_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-chef-16.14.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-chef15-15.15.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-choria-mcorpc-support-2.25.2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-fluentd-1.14.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-specinfra-2.82.16.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-gitlab-grit-2.8.3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-nokogiri-1.11.7.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-nokogiri18-1.8.5.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-addressable25-2.5.2_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-em-websocket-0.5.2_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-gollum-rugged_adapter-0.4.4_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-http3-3.3.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-platform-api-3.3.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-redis-rack1-1.6.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-rest-client-2.1.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-16_18h20m53s/logs/rubygem-tumblr_client-0.8.5_4.log
Comment 19 Thibault Jouan 2021-10-19 11:36:25 UTC
  Thanks for the exp-run and logs.

  I will check all of them later, but the few I could check now fail
because of a small output change in gem specification generation by
RubyGem (the same which affected `gemdeps()' function in
`Mk/Scripts/qa.sh'). That's "expected" but I missed it.

  As an example, mail/rubygem-net-smtp has 3 runtimes dependencies. To
make example short, I kept only 1 on 3 (digest). With RubyGems 3.0.8:

      if s.respond_to? :specification_version then
        s.specification_version = 4

        if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
          s.add_runtime_dependency(%q<digest>.freeze, [">= 0"])
        else
          s.add_dependency(%q<digest>.freeze, [">= 0"])
        end
      else
        s.add_dependency(%q<digest>.freeze, [">= 0"])
      end

  And now with RubyGems 3.2.29:

      if s.respond_to? :specification_version then
        s.specification_version = 4
      end

      if s.respond_to? :add_runtime_dependency then
        s.add_runtime_dependency(%q<digest>.freeze, [">= 0"])
      else
        s.add_dependency(%q<digest>.freeze, [">= 0"])
      end

  Most of our patches want to adapt, or sometimes remove the first
`s.add_runtime_dependency' call, and as you can see both the context
and indentation changed. I'm not sure why we patch the "generated" gem
specification at install and not the one provided in the gem, but
there must be good reasons (maybe it's considered more "stable", in a
way).

  I see two options:

1. Patch the gem specification generation output to restore a format
suitable for patches, so that they apply cleanly again;
2. Redo all the patches in failing gem ports.

  In my view, 2 is better because the upstream change is beneficial
and also because it avoids maintaining a new patch in RubyGems port.
However I don't know if it acceptable to submit new patch with both
the RubyGems update and also the changes to all the gems ports (it
would be same "logical" change, but affects many ports at once).
Comment 20 Sir l33tname 2021-10-23 18:39:08 UTC
Nice to see that there is finally progress to get ruby3 with gems on FreeBSD.
A bit unfortunate that i see this after finally getting my patch to work :D 


Basically found more or less the same stuff as in this patch minus the plugins stuff but with a different (same as fedora) patch to disable bundler.
(see https://gist.github.com/fliiiix/be69eb8e56ef6aac273167f469aecb26#file-0003-devel-ruby-gems-update-3-0-8-3-2-29-patch)

And because of that i found the same problem with the patches and update at least the ports i need see here https://gist.github.com/fliiiix/9f92644e1eb54c0db64a034541f0d8ae

I also think here option 2 to update the patches is the better approach. 
And can help with that if needed. Question for me is if it makes sense to make a commit/patch per port or one big commit/patch for all affected ports.
Comment 21 Thibault Jouan 2021-10-25 14:59:23 UTC
Created attachment 229025 [details]
ports failing to build with rubygems 3.2

  I looked into the Fedora solution (setup command monkey-patch), but
it was written for an old RubyGems version (3.1.3 I guess). It doesn't
work anymore with RubyGems 3.2 because in addition to not calling
`#install_default_bundler_gem', we'd have to monkey-patch
`#install_lib' which now also does something related to bundler (but
we'd still need to execute it without the bundler thing), and maybe
rewrite `#remove_old_lib_files' too.

  I started working on the update of the patches that were causing
build failures with RubyGems 3.2.29. I think since it's a single
"logical" change, it should be one patch/commit, in my opinion easier
to maintain or revert. But I'm not yet familiar with FreeBSD ports
specifics. I can split the change in multiple ones later if needed.

  I'm attaching a list of those failing ports (it helped me).
Comment 22 Thibault Jouan 2021-10-25 15:18:34 UTC
Created attachment 229026 [details]
update to 3.2.29 - with updates to gemspec patches in rubygems ports

  This new patch should fix the build errors reported in the exp-run.
I only checked the patching aspect for now, as I don't know how to use
and check runtime behavior for most of those gems.

  I updated notes in the patch itself, but here are two important
ones:

devel/rubygem-mongo: this patch isn't about changing gem dependencies
(as most do), but about removing files from the gem if I'm correct, I
should investigate more to check maybe.

net-mgmt/rubygem-riemann-dash: this one fails stage-qa step with
poudriere testport, I'm not sure if it's related to the patch yet.
Comment 23 Antoine Brodin freebsd_committer freebsd_triage 2021-10-27 06:41:39 UTC
Some new failure logs:


http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-fpm-1.11.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-azure-storage-common-2.0.2_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-azure_sdk-0.73.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-errand-0.8.1_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-gitlab-labkit-0.21.1_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-gitlab_git-10.7.0_3,1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-jeweler-2.3.9.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-pdk-2.3.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-cogbot-0.1.13_8.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-av-0.9.0_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-termtter-2.2.9_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-tinder-1.10.1_3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-oxidized-web-0.13.1_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-riemann-tools-0.2.14.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-activestorage613-6.1.3.2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-fog-google19-1.9.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-fog-vsphere-3.5.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-t-3.1.0_3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-train-3.8.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-tweetstream-2.6.1_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-devise-two-factor30-rails52-3.0.3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-devise-two-factor30-rails60-3.0.0.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-doorkeeper-openid_connect-1.7.5_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-rhc-1.32.2_2.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-chef-dk-4.13.3_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-r10k-3.12.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-html2text-0.3.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-actioncable5-5.1.7.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-actionpack4-4.2.11.3.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-actionpack50-5.0.7.2_1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-gitlab-gollum-lib-4.2.7.10.gitlab.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-gollum-lib-4.2.10.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-jekyll-4.2.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-jekyll3-3.9.1.log
http://package18.nyi.freebsd.org/data/122amd64-default-foo/2021-10-25_15h37m24s/logs/rubygem-kensa-3.1.0.log
Comment 24 Thibault Jouan 2021-10-27 09:57:47 UTC
  Thanks for the new build errors logs.

  I will look into fixing them as soon as I can. RubyGems 3.2.30 was
released yesterday, so I will update this part of the patch for 3.2.30
instead of 3.2.29 too.
Comment 25 Thibault Jouan 2021-10-27 15:30:44 UTC
Created attachment 229073 [details]
update to 3.2.30

RubyGems 3.2.30
---------------

  Patch now updates devel/ruby-gems to 3.2.30 instead of 3.2.29. New
file added to pkg-plist:
%%RUBY_SITELIBDIR%%/rubygems/resolver/molinillo/LICENSE. There are
other small changes, but not impacting any patch for now.


Obsolete gem ports failing to build
-----------------------------------

  We can now ignore fixes for 5 gem ports:

build failures from comment #18:
  sysutils/rubygem-chef15
    renamed to sysutils/rubygem-chef
  devel/rubygem-chef-provisioning
    2021-10-25 Chef Provisioning is EoL'd on August 31, 2019
  graphics/rubygem-goocanvas
    2021-10-25 Deprecated by upstream. Unmaintained since October 12, 2014

build failure from comment #23:
  sysutils/rubygem-chef-dk
    2021-10-25 ChefDK is EoL'd on December 31, 2020
  www/rubygem-kensa
    2021-10-25 Kensa is deprecated in favor of the addons-admin Heroku
    CLI plugin since December 14, 2018


New/updated gem ports failing to build
--------------------------------------

devel/rubygem-mongo

  I unfortunately removed the change that was in previous patch:
* what I did to update the patch in version 2.15.1 of the gem does not
  seem to work anymore with 2.16.0 (gem port was updated yesterday);
* I still don't know how to check if the patch is behaving how it
  should anyway;
* This change alone was bigger than all others and made the patch
  really huge.

net-mgmt/rubygem-riemann-tools

  I fixed the gem specification patching issue, but there is a new
one:

    =======================<phase: package        >==================
    ===>  Building package for rubygem-riemann-tools-0.2.14
    pkg-static: Unable to access file /wrkdirs/usr/ports/net-mgmt/rubygem-riemann-tools/work/stage/usr/local/share/doc/riemann-tools/README.markdown:No such file or directory
    *** Error code 1


All other gem ports reported in comment #23 should build now.
Comment 26 Antoine Brodin freebsd_committer freebsd_triage 2021-10-29 08:32:33 UTC
I have 2 errors applying the patch:

error: devel/rubygem-gitlab-labkit/files/patch-gitlab-labkit.gemspec: No such file or directory
error: net/rubygem-activestorage613/files/patch-activestorage.gemspec: No such file or directory
Comment 27 Thibault Jouan 2021-10-29 09:12:18 UTC
Created attachment 229114 [details]
update to 3.2.30

  I updated the patch. Both errors are about removed gemspec patches
in ports.

devel/rubygem-gitlab-labkit:
  updated from 0.21.1 to 0.21.2, gemspec patch removed
net/rubygem-activestorage613:
  port removed, looks like it was for gitlab, which now use 6.1.4
  instead of 6.1.3

  So both errors can be ignored or new attached patch can be used.
Comment 29 Thibault Jouan 2021-11-01 22:54:57 UTC
Created attachment 229210 [details]
update to 3.2.30

  Thank you for the logs, I updated the patch which should fix those
11 new build failures.

  In this list was devel/rubygem-mongo, and the change for its gemspec
patch is back (removed in comment #25 following an update).

  Should maintainers of gems affected by this patch be contacted or
review those changes in any way? I'm asking, since I realize there was
at least one error in the patch I wrote. Maybe reviews happens after
at least one successful exp-run?

  I also added three new changes for ports I would have expected to
fail to patch in next exp-run, as they were skipped in last one:

    devel/rubygem-omnibus
    sysutils/rubygem-bosh-gen
    sysutils/rubygem-backup


  I looked into issues I reported earlier:

net-mgmt/rubygem-riemann-dash port

  I had introduced an error in the gemspec file when updating its
patch since the beginning:

    ====> Running Q/A tests (stage-qa)
    Error: RubyGem dependency webrick ~> 1.3.1 is not satisfied.

  I missed this dependency removal, which I fixed in attached patch. I
also slightly changed the original patch which removed "dead code" in
addition to the important change (at the difference of most other gem
ports gemspec patch). In my opinion this will make it easier to change
the patch now, but since this is opinionated I can avoid doing that in
a next patch if I shouldn't have.

net-mgmt/rubygem-riemann-tools port

  It fails to build with `DOCS' option:

    ====> Running Q/A tests (stage-qa)
    ====> Checking for pkg-plist issues (check-plist)
    […]
    ===> Checking for items in pkg-plist which are not in STAGEDIR
    Error: Missing: %%DOCSDIR%%/README.markdown

  I have the same issue with RubyGems 3.0.8 (current version in ports)
and the update to 3.2.30.
Comment 30 deivid.rodriguez 2021-11-05 11:56:33 UTC
Hello!

I want to make things easier for you in the future, so:

* I merged https://github.com/rubygems/rubygems/pull/2768.
* I created https://github.com/rubygems/rubygems/pull/5051 and https://github.com/rubygems/rubygems/pull/5052 to fix more `--destdir` issues.
* I will try to create a CI check that make the default gem home unwritable, and then runs `ruby setup --destdir <folder>` to a writable folder, to make sure nothing tries to write to the default gem home.

Feel free to try my patches and leave feedback <3
Comment 31 deivid.rodriguez 2021-11-05 14:26:40 UTC
Quick update, I merged https://github.com/rubygems/rubygems/pull/5051 and created yet one more patch with another fix: https://github.com/rubygems/rubygems/pull/5053.
Comment 32 deivid.rodriguez 2021-11-05 20:32:22 UTC
Upstream patches have now either landed or been included with https://github.com/rubygems/rubygems/pull/5053, and Yasuhiro Kimura has tried them and they seem to be working!
Comment 33 Antoine Brodin freebsd_committer freebsd_triage 2021-11-06 08:55:01 UTC
Exp-run looks fine
Comment 34 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-11-08 18:35:11 UTC
@ruby Now that latest patch passed exp-run, would you please approve to commit it?
Comment 35 Thibault Jouan 2021-11-14 16:50:43 UTC
  Thanks for the last exp-run and changes to RubyGems upstream, that's
great news.

  I noticed the patch fails to apply again following two ports updates:

    sysutils/rubygem-bolt/files/patch-bolt.gemspec
    sysutils/rubygem-r10k/files/patch-r10k.gemspec

  Should I update the patch now, or wait for approval of the patch
currently attached to the PR?
Comment 36 deivid.rodriguez 2021-11-15 08:14:18 UTC
Thank you, I understand that upstream fixes are only useful for future updates, not for this one which is mainly done. Correct?

Anyways, I'm still waiting to merge the last PR since I found some more issues which I want to fix there, but I hope it will make it into a release soon.
Comment 37 Dan Langille freebsd_committer freebsd_triage 2021-12-02 16:28:22 UTC
When I send fixes upstream, I also implement them in the port until upstream issues a new release which contains those fixes.

Is there anything holding us up now? I'd love to get this into the tree. I have developers at $WORK who would love to test it.

I tried the current patch and it stops patching here. Is there anything I can do to help please?

....
Patching file www/rubygem-addressable25/files/patch-gemspec using Plan A...
Hunk #1 succeeded at 1.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- www/rubygem-em-websocket/files/patch-gemspec	
|+++ www/rubygem-em-websocket/files/patch-gemspec	
--------------------------
File to patch:
Comment 38 deivid.rodriguez 2021-12-04 19:39:49 UTC
For what it's worth, I decided to go with the current version of the fix, since it fixes most of the issues with `--destdir` and I don't want to block it on the little issue that I found because fixing it requires a bigger refactoring that I'm not sure when I'll attempt.

I hope to release it in the next few days with rubygems 3.2.33.

For completeness, the little issue with the current patch is that binstub regeneration inside the DESTDIR only works properly for gems that are also installed globally.

So if I have `rake` installed globally, this works:

```
$ gem list rake

*** LOCAL GEMS ***

rake (13.0.6)

$ rm -rf /tmp/foo && gem install rake --install-dir /tmp/foo && (echo "foo" > /tmp/foo/bin/rake) && ruby -Ilib setup.rb --destdir /tmp/foo --prefix / && head -1 /tmp/foo/bin/rake
Fetching rake-13.0.6.gem
Successfully installed rake-13.0.6
1 gem installed
  Successfully built RubyGem
  Name: bundler
  Version: 2.3.0.dev
  File: bundler-2.3.0.dev.gem
Bundler 2.3.0.dev installed
RubyGems 3.3.0.dev installed
Regenerating binstubs
Regenerating plugins



------------------------------------------------------------------------------

RubyGems installed the following executables:
	/tmp/foo/bin/gem
	/tmp/foo/bin/bundle
	/tmp/foo/bin/bundler

#!/Users/deivid/.rbenv/versions/3.0.3/bin/ruby
```

But if I remove the globally installed rake, the `rake` binstub in the DESTDIR is no longer regenerated properly:

```
$ gem uninstall rake --executables --force
Removing rake
Successfully uninstalled rake-13.0.6

$ rm -rf /tmp/foo && gem install rake --install-dir /tmp/foo && (echo "foo" > /tmp/foo/bin/rake) && ruby -Ilib setup.rb --destdir /tmp/foo --prefix / && head -1 /tmp/foo/bin/rake
Fetching rake-13.0.6.gem
Successfully installed rake-13.0.6
1 gem installed
  Successfully built RubyGem
  Name: bundler
  Version: 2.3.0.dev
  File: bundler-2.3.0.dev.gem
Bundler 2.3.0.dev installed
RubyGems 3.3.0.dev installed
Regenerating binstubs
Regenerating plugins



------------------------------------------------------------------------------

RubyGems installed the following executables:
	/tmp/foo/bin/gem
	/tmp/foo/bin/bundle
	/tmp/foo/bin/bundler

foo

```
Comment 39 Dan Langille freebsd_committer freebsd_triage 2021-12-05 14:16:29 UTC
Happy to try an updated patch.
Comment 40 Thibault Jouan 2021-12-05 17:16:29 UTC
About updating/committing a patch:

  The issue here as I understand, is that once a patch is attached we
have to check it with exp-run, then wait for review, approval and
commit it (Comment #34). But understandably, the exp-run and
adjustments on the patch take some time, reviewing it too, and by this
time some gem ports affected by the patch have been updated.

  I'm happy to update the patch as many time as needed but:
* I don't want to give unnecessary work to people (Antoine Brodin for
  exp-run for example);
* I don't have a workstation with the availability and build power to
  rebuild *all* ports depending on RubyGems fast enough before the gem
  ports get updated (I mean trying to do exp-run on my machine is not
  really possible event if I limit ports to the ones depending on
  RubyGems).

  Anyway I will make a new update, but not before one or two days at
least. In the meantime, if you don't use any of the gems where the
FreeBSD port patched the gem specification file, then you can ignore
the failure and keep the version currently in the port tree. It won't
affect devel/ruby-gems itself, nor any gem installed with `gem install
...`.

  This is slightly unrelated, but a thing that could help is to get
rid of those gem specification patches. Just an idea but ideally,
maintainers of the FreeBSD port could convince gem author to fix
version specifications in dependencies, so that we wouldn't need
patches in the port (I'm pretty sure it was done, but don't know to
which extent). I also know that a dedicated feature for this was
discussed upstream (to avoid patching), but IIRC wasn't very popular
(because ideally gems should be fixed directly).


About RubyGems setup command `--destdir' and
`--no-regenerate-binstubs' options (Comment #36 and Comment #38):

  The original issue regarding executable wrappers in combination with
`--destdir' is from RubyGems 2.7.0 if I'm correct (909b5fb8). I
reported it by mail with a possible fix to the author, but got a rude
answer at the time. It was later possible to workaround this with
4619f13a, we have to pass `--no-regenerate-binstubs' explicitly. This
is similar to the more recent `--no-regenerate-plugins' and
`--format-executable' which we must now pass explicitly to the setup
command for success. One of the places where this was discussed later
is here: https://github.com/rubygems/rubygems/issues/2370

  With a normal installation of RubyGems, there cannot be any
executable wrapper to rewrite (or any plugin wrapper) since we need
RubyGems to install gems, so it happens before we could have a gem
installed. The kind of manual update that was explained in the GitHub
issue, or shown with rake gem in Comment #38 only affects some
RubyGems developers and maintainers if I understand correctly, and
only when "experimentally" installing RubyGems manually over an
existing installation. End users should never encounter this scenario.

  I only noticed it (years ago) because of the behavior being the
default, while attempting to update the port, because the name of the
environment variable to enable bundler changed, and because the build
system was warning about things not installed where they should be.
The conditions to trigger it are very specific. IMO all people
maintaining RubyGems ports introduced `--no-regenerate-binstubs' in
their build at the time, and wont use the code path where that last
remaining bug may be. What would help is if following similar features
were "opt-in" instead of "opt-out" (I guess we cannot change actual
ones in 3.x again, to avoid breakage).
Comment 41 deivid.rodriguez 2021-12-05 18:13:47 UTC
Thanks for your comment. I'll reply to what concerns me.

>   The original issue regarding executable wrappers in combination with
> `--destdir' is from RubyGems 2.7.0 if I'm correct (909b5fb8). I
> reported it by mail with a possible fix to the author, but got a rude
> answer at the time. It was later possible to workaround this with
> 4619f13a, we have to pass `--no-regenerate-binstubs' explicitly. This
> is similar to the more recent `--no-regenerate-plugins' and
> `--format-executable' which we must now pass explicitly to the setup
> command for success. One of the places where this was discussed later
> is here: https://github.com/rubygems/rubygems/issues/2370

I'm sorry someone replied to an email to you rudely at the time. Anyways, I'm trying to help out now so let's move on, yeah? The issue you linked to and a few others have been fixed by the different patches merged recently, and rubygems should no longer try to write outside of `--destdir` if it's passed (both during normal operation or binstub/plugin regeneration).

> With a normal installation of RubyGems, there cannot be any
> executable wrapper to rewrite (or any plugin wrapper) since we need
> RubyGems to install gems, so it happens before we could have a gem
> installed. The kind of manual update that was explained in the GitHub
> issue, or shown with rake gem in Comment #38 only affects some
> RubyGems developers and maintainers if I understand correctly, and
> only when "experimentally" installing RubyGems manually over an
> existing installation. End users should never encounter this scenario.

They absolutely do! In fact, binstub regeneration is an important feature of rubygems upgrader. We sometimes need to tweak the binstubs that rubygems generates. Having rubygems upgrader automatically upgrade all binstubs to the new format a new rubygems version needs and generates is really nice. Even when first installing rubygems, I could imagine situations where upgrading binstubs in an exisiting gem path could be handy. In any case, it's great that you don't need that yourselves, I'm happy I merged the main fix and reticketed that little issue for later.

> What would help is if following similar features
> were "opt-in" instead of "opt-out" (I guess we cannot change actual
> ones in 3.x again, to avoid breakage).

I'm sorry the introduction of those flags broke things for you. That doesn't mean the flags themselves are backwards incompatible, actually quite the opposite, they were introduced to fix backwards compatibility issues with new rubygems versions. Unfortunately in this case, their introduction was buggy since they didn't play well with `--destdir`. So I think a more reasonable request would be: don't introduce new bugs with new features or bug fixes xD. That's our goal but you know, shit happens. Anyways, when I make changes that I think could possibly affect/break packagers I usually ping at least Debian & Fedora packagers to gather opinions. I'm happy to add someone from FreeBSD to my "rubygems packagers ping list" if you wish. I'm aware you're not happy about past upstream maintenance, and that upstream has never paid too much attention to packagers (they are a minority of our users, and upstream could barely deal with the rest of our community), but I'm trying to help more now, so let's move on and not talk about the past, specially since I wasn't really involved back then. Ok?
Comment 42 Thibault Jouan 2021-12-07 16:55:41 UTC
(In reply to deivid.rodriguez from comment #41)

  I thought it would help to know about the difficulties encountered
when updating the port, and also to help anyone reading the PR know
what would be relevant to the attached patch or not (also to avoid
delay like a potential reviewer thinking that the issue discussed was
related to the patch). This was my intention at least, but I probably
gave too much context. I wrote earlier that I was thankful for your
work and this is still true :-)
Comment 43 Thibault Jouan 2021-12-07 16:58:09 UTC
Created attachment 229958 [details]
update to 3.2.30

(In reply to Dan Langille from comment #39)

  I fixed the patch and it should now apply cleanly.

  I made changes for 4 gem ports:
net/rubygem-grpc130: gemspec patch changed since last patch
sysutils/rubygem-bolt: gemspec patch changed since last patch
sysutils/rubygem-r10k: gemspec patch changed since last patch
www/rubygem-em-websocket: gemspec patch was removed

  I also tried to look for other modified or added gem ports since the
previous patch I made, ports that would have a change in `files'
subdirectory since then:

  devel/rubygem-gitlab-labkit
  devel/rubygem-rugged
  net/rubygem-gitaly
  net/rubygem-grpc142
  net/rubygem-kas-grpc
  net/rubygem-spamcheck

  Unfortunately some have many dependencies and I cannot wait for the
build right now. In one or two days I will identify those which may
include a gemspec patch that require fixing. I used an ugly command to
find them, so maybe there are other, with zsh:

    for f ($(git grep -l ^USES=.\*gem \*/\*/Makefile | cut -d / -f 1-2)) \
      git diff bccba76b... --quiet -- $f/files || echo $f


  Note: I didn't update to RubyGems 3.2.32 yet, because it looks like
3.2.31 vendors more libraries, for example `optparse' from Ruby but
with some modifications like renaming the class from `OptionParser' to
`Gem::OptionParser'. I'll have to reread porter handbook to know if
this should be "unvendored".
Comment 44 Dan Langille freebsd_committer freebsd_triage 2021-12-07 17:23:15 UTC
The patch applies cleanly here.  Building against FreeBSD 13 and head on the ports tree, I get this build error.

sudo poudriere testport -i -p dvl -j 130R -z REDACTED devel/ruby-gems

...

===>   Returning to build of ruby27-gems-3.2.30
===>   ruby27-gems-3.2.30 depends on file: /usr/local/bin/ruby27 - found
===========================================================================
=>> Recording filesystem state for prestage... done
=======================<phase: stage          >============================
===>  Staging for ruby27-gems-3.2.30
===>   ruby27-gems-3.2.30 depends on package: ca_root_nss>=0 - found
===>   ruby27-gems-3.2.30 depends on file: /usr/local/bin/ruby27 - found
===>   Generating temporary packing list
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/build_info
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/plugins
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/cache
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/doc
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/extensions
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/gems
/bin/mkdir -p /wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7/specifications
cd /wrkdirs/usr/ports/devel/ruby-gems/work/rubygems-3.2.30/ && /usr/bin/env GEM_HOME=/wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7 /usr/local/bin/ruby27 setup.rb --destdir=/wrkdirs/usr/ports/devel/ruby-gems/work/stage  --no-regenerate-binstubs  --no-regenerate-plugins  --format-executable
ERROR:  While executing gem ... (Errno::ENOENT)
    No such file or directory @ dir_chdir - bundler/lib
*** Error code 1
Comment 45 deivid.rodriguez 2021-12-07 17:31:50 UTC
> I thought it would help to know about the difficulties encountered
> when updating the port, and also to help anyone reading the PR know
> what would be relevant to the attached patch or not (also to avoid
> delay like a potential reviewer thinking that the issue discussed was
> related to the patch). This was my intention at least, but I probably
> gave too much context. I wrote earlier that I was thankful for your
> work and this is still true :-)

It's alright, the context is indeed useful, sorry if I sounded harsh and thanks for your kind words :) I released rubygems 3.2.33 with the last patch.

> Note: I didn't update to RubyGems 3.2.32 yet, because it looks like
> 3.2.31 vendors more libraries, for example `optparse' from Ruby but
> with some modifications like renaming the class from `OptionParser' to
> `Gem::OptionParser'. I'll have to reread porter handbook to know if
> this should be "unvendored".

Yeah, unfortunately, as more standard libraries that we use internally get converted into gems, we need to vendor them, because we can't have any dependencies ourselves because otherwise we would restrict user's choice of the dependency to the version that we use internally.
Comment 46 Thibault Jouan 2021-12-07 18:17:30 UTC
(In reply to Dan Langille from comment #44)
> sudo poudriere testport -i -p dvl -j 130R -z REDACTED devel/ruby-gems
[…]
> cd /wrkdirs/usr/ports/devel/ruby-gems/work/rubygems-3.2.30/ && /usr/bin/env GEM_HOME=/wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7 /usr/local/bin/ruby27 setup.rb --destdir=/wrkdirs/usr/ports/devel/ruby-gems/work/stage  --no-regenerate-binstubs  --no-regenerate-plugins  --format-executable
> ERROR:  While executing gem ... (Errno::ENOENT)
>     No such file or directory @ dir_chdir - bundler/lib
> *** Error code 1

  This is new to me. I'm surprised because I didn't change anything in
the patch for `devel/ruby-gems', and multiple previous patch passed
the exp-run for this port.

  I did my patch on d624c6c0fba3880074ca8fd7455bef3aa7a8298a in port
tree (not so old). I use FreeBSD 12.2 RELEASE, and have
`DEFAULT_VERSIONS= ruby=2.7' too.

  "No such file or directory @ dir_chdir - bundler/lib": that this
directory does not exist is expected (`post-extract' target in the
Makefile), the code triggering this is not.

  Thank you for the very fast test and report, unfortunately I will
need more time to look into it.
Comment 47 Thibault Jouan 2021-12-08 13:03:43 UTC
Created attachment 229977 [details]
potential fix for patching issue

(In reply to Dan Langille from comment #44)

> ERROR:  While executing gem ... (Errno::ENOENT)
>     No such file or directory @ dir_chdir - bundler/lib
> *** Error code 1

  I think that the patch for devel/ruby-gems (which is included in the
patch for the update) did not apply. The only issue I see is that I
made this patch with git, not with `make makepatch`.

  If I `rm devel/ruby-gems/files/patch-bundler-remove.patch`, I get
the same error output. My guess is that the error is from following
chdir:

    class Gem::Commands::SetupCommand < Gem::Command
      […]
      def install_lib(lib_dir)
        libs = { 'RubyGems' => 'lib' }
        libs['Bundler'] = 'bundler/lib' # patch should remove this line
        libs.each do |tool, path|
          […]
          Dir.chdir path do
            […]
          end
        end
      end
    end

  It can be checked by modifying `RUBY_SETUP_OPTIONS' in
`devel/ruby-gems/Makefile' by adding two arguments: --backtrace --verbose

    ERROR:  While executing gem ... (Errno::ENOENT)
        No such file or directory @ dir_chdir - bundler/lib
      /wrkdirs/usr/ports/devel/ruby-gems/work/rubygems-3.2.30/lib/rubygems/commands/setup_command.rb:490:in `chdir'
      […]
      /wrkdirs/usr/ports/devel/ruby-gems/work/rubygems-3.2.30/lib/rubygems/commands/setup_command.rb:321:in `install_lib'


  While I can obtain the same error, I don't know how to fully
reproduce it (if the patch embedded in the patch does not apply, I
don't know why). I'd be interested to see the `patch' target in
poudriere testport output. In the meantime, I'm attaching a patch that
convert the "git" patch to a `make makepatch` one. It need to be
applied after the "main" patch (to avoid being huge).
Comment 48 Dan Langille freebsd_committer freebsd_triage 2021-12-08 14:37:19 UTC
I suspect I'm using the diff, not the attachment. Let me try again and provide full details.

[dvl@ava-pkg-02prd:/usr/dvl/main] $ git reset --hard 
HEAD is now at d624c6c0fba3 japanese/font-plemoljp{,-hs,-nf}: Update to 1.2.2.

[dvl@ava-pkg-02prd:/usr/dvl/main] $ rm patch-bundler-remove.patch

[dvl@ava-pkg-02prd:/usr/dvl/main] $ fetch -o ~/tmp/patch.gem "https://bz-attachments.freebsd.org/attachment.cgi?id=229958"
/home/dvl/tmp/patch.gem                                347 kB  377 kBps    01s

[dvl@ava-pkg-02prd:/usr/dvl/main] $ patch < ~/tmp/patch.gem 

output not shown - see https://gist.github.com/dlangille/3cdbab365865c736ed3372b3bc224f69 instead


For me: patch-bundler-remove.patch gets created in the top level directory (see `git status` output in the first comment of the above mentioned gist.

[dvl@ava-pkg-02prd:/usr/dvl/main] $ mkdir devel/ruby-gems/files
[dvl@ava-pkg-02prd:/usr/dvl/main] $ mv patch-bundler-remove.patch devel/ruby-gems/files/patch-bundler-remove.patch


Now it builds.
Comment 49 Thibault Jouan 2021-12-08 15:10:05 UTC
Comment on attachment 229977 [details]
potential fix for patching issue

  Interesting, I didn't notice the issue because I do already have
`devel/ruby-gems/files' directory existing, because of other custom
patches.

  Creating the directory before running patch with no arguments solves
the issue. Not creating the directory and running patch with `-p0'
works too.

    mkdir devel/ruby-gems/files && patch < ~/tmp/patch.gem

    patch -p0 < ~/tmp/patch.gem
Comment 50 Dan Langille freebsd_committer freebsd_triage 2021-12-13 14:31:12 UTC
(In reply to Thibault Jouan from comment #49)

The new patch applies and builds cleanly for me.
Comment 51 Yasuhiro Kimura freebsd_committer freebsd_triage 2021-12-22 08:43:38 UTC
Created attachment 230305 [details]
Updated patch file

Chase update of ports tree. Some rubygem ports are removed.
Comment 52 Dan Langille freebsd_committer freebsd_triage 2021-12-22 14:33:12 UTC
(In reply to Yasuhiro Kimura from comment #51)
ruby27-gems: 3.2.30 builds for me here.

What is the next step?
Comment 53 Charlie Li freebsd_committer freebsd_triage 2021-12-26 02:05:23 UTC
3.2.33 and 3.3.3 are now the latest versions.
Comment 54 Tomoyuki Sakurai 2022-01-08 00:02:39 UTC
happy new year to all ruby users in the bug report.
Comment 55 Dan Langille freebsd_committer freebsd_triage 2022-01-19 20:11:39 UTC
Anything I can so to help test?
Comment 56 Thibault Jouan 2022-01-26 13:58:11 UTC
Created attachment 231355 [details]
update to 3.3.6

Patch updated:
* Update to 3.3.6 instead of 3.2.30
* Remove changes because of gemspec patch removal in some ports:
  - devel/rubygem-azure-storage-common
  - devel/rubygem-dynflow
  - mail/rubygem-net-imap
  - mail/rubygem-net-pop
  - mail/rubygem-net-smtp
* Remove changes temporarily because of pending gemspec changes:
  - www/rubygem-gollum-lib
  - www/rubygem-gollum-rugged_adapter
  - www/rubygem-gollum
  (see bug #260207)
* Remove unnecessary info from commit messages
* Add notes for notable changes between RubyGems 3.2.30 and 3.3.6:
  * Remove `.document' files in vendored libraries (See upstream
    2d7cd652).

    RubyGems now bundles tsort library (d3268809) and optparse one
  (47c4d12e). We chose not to unbundle those, because they were moved into
  `Gem' namespace (renamed as `Gem::TSort' and `Gem::OptionParser') which
  would require an invasive and complex patch.
  * The server command has been moved to the rubygems-server gem.


  I choose to remove those `.document' files because it seemed strange
to have them in the release tarball, and previously vendored library
didn't have them. Also, patch in bug #260780 remove them too.


  Please let me know if you have issues with this new patch.

  Now if someone has extra free time and build power to test building
ports that depends on devel/ruby-gems *and* have a gemspec patch, I'd
be happy to know about any build error related to the patch. Since the
last exp-run, I've been doing manual, incomplete, inefficient checks
with this ugly command:

for f ($(git grep -l ^USES=.\*gem \*/\*/Makefile | cut -d / -f 1-2)) \
  git diff ${COMMIT}... --quiet -- $f/files || echo $f >> ~/gem_ports_list
  (zsh)

  Where ${COMMIT} is the original commit my last version of the patch
was based on. Then I poudriere testport the suspicious ones, but
probably missed a lot. Test building *all* ports that depend on
devel/ruby-gems is better and simpler, but takes lot of time.
Comment 57 Dan Langille freebsd_committer freebsd_triage 2022-01-26 14:29:22 UTC
(In reply to Thibault Jouan from comment #56)

That patch applies cleanly and builds here.  Thank you.
Comment 58 Dan Langille freebsd_committer freebsd_triage 2022-01-26 14:31:16 UTC
If someone gives me a list of ports for test builds, I can do that on a powerful system.
Comment 59 Thibault Jouan 2022-01-26 14:59:14 UTC
(In reply to Dan Langille from comment #58)
Thank you for the very fast feedback.

Here is a list of ports:
http://vault.a13.fr/media/rg-test_build-ports_list.txt

I don't even need the build logs. Only the list of port "origin" (like in my list) that fail to build. Then I investigate the failing ones, if there are, 99% should be about gem spec patches conflicts and I will redo them.

This way IMO we should catch most errors as in an exp-run, but with less ports to build (2600 gem ports… and probably substantially more for dependencies). If this is still too much, we could decrease the list by keeping only ports which have a gemspec patch.
Comment 60 Dan Langille freebsd_committer freebsd_triage 2022-01-26 15:23:28 UTC
(In reply to Thibault Jouan from comment #59)
That list hightlighted that I was testing on an older ports tree.  After a git pull, I have patch issues:

Patching file textproc/rubygem-html2text/files/patch-gemspec using Plan A...
Hunk #1 succeeded at 1.
Hmm...  The next patch looks like a unified diff to me...
The text leading up to this was:
--------------------------
|diff --git textproc/rubygem-nokogiri/files/patch-gemspec textproc/rubygem-nokogiri/files/patch-gemspec
|index 43f56f9a0877..f4885e236892 100644
|--- textproc/rubygem-nokogiri/files/patch-gemspec
|+++ textproc/rubygem-nokogiri/files/patch-gemspec
--------------------------
File to patch: 

That is the only issue.
Comment 61 Thibault Jouan 2022-01-26 17:06:03 UTC
(In reply to Dan Langille from comment #60)
  Yes sorry about that, my own port tree always lag one or two days
behind because I do those updates on a "desktop" computer, and when
some ports are updated (say rust for example), I need a whole night or
more to rebuild all. And I want to work on the RubyGems update with up
to date ports to be sure it's correct.

  My last patch is based on 2e1fdc173831debc50f2c25b86e202c0a4967334
(Mon Jan 24 17:41:06 2022 -0600). textproc/rubygem-nokogiri was
updated few hours ago (Wed Jan 26 11:43:00 2022 +0100).

  Thankfully, this last update is the gemspec patch removal, which is
great news: you can safely ignore failure to apply for this specific
file (but only this one), and it will also simplify next patch update.
Hopefully I submit an updated patch tomorrow (if I can rebuild my
ports next night).
Comment 62 Dan Langille freebsd_committer freebsd_triage 2022-01-26 17:27:11 UTC
(In reply to Thibault Jouan from comment #61)

These results are based on 807e6fc0e4c03c8cbe25f91a0e15be3e3a8a727b

2748 ports queued, 2361 built, 12 failed (see below), 375 skipped

failures:

textproc/rubygem-nokogiri
databases/rubygem-mysql
devel/rubygem-mongo
science/cdcl
www/rubygem-gollum-rugged_adapter
textproc/rubygem-nokogiri111
devel/rubygem-xdg5
net/rubygem-spamcheck
devel/rubygem-xdg
net/rubygem-kas-grpc
mail/rubygem-tmail
graphics/rubygem-objectdetect
Comment 63 Dan Langille freebsd_committer freebsd_triage 2022-01-26 17:39:49 UTC
I am now trying with the same version of the tree as your patch.
Comment 64 Dan Langille freebsd_committer freebsd_triage 2022-01-26 20:20:14 UTC
When testing against your ports tree release, these fail:

science/cdcl
mail/rubygem-tmail
devel/rubygem-xdg
devel/rubygem-mongo
devel/rubygem-xdg5
www/rubygem-gollum-rugged_adapter
net/rubygem-kas-grpc
net/rubygem-spamcheck
www/rubygem-gollum-lib
devel/rubygem-gitlab-labki
databases/rubygem-mysq
graphics/rubygem-objectdetect
Comment 65 Thibault Jouan 2022-01-27 11:30:58 UTC
Created attachment 231387 [details]
update to 3.3.6

  Thank you for the builds and list of failing ports.

  I was wrong in comment #60, change on textproc/rubygem-nokogiri is a
recent patch addition, not removal. Anyway I adapted my patch for it,
and other ports:

  devel/rubygem-gitlab-labkit fixed with nokogiri gemspec patch change
  devel/rubygem-mongo RubyGems version was in gemspec patch, fixed manually
  net/rubygem-kas-grpc gemspec patch fix
  net/rubygem-spamcheck gemspec patch fix
  textproc/rubygem-nokogiri new gemspec patch
  textproc/rubygem-nokogiri111 new port, needed gemspec patch

  Those ports seem to fail for reasons unrelated to RubyGems port
patch:

  devel/rubygem-xdg port requires Ruby ~> 3.1 (marked broken otherwise)
  graphics/rubygem-objectdetect port maked broken
  mail/rubygem-tmail port marked broken
  science/cdcl port marked broken, unfetchable

  Those ports I ignore on purpose for now (comment #56 and
bug #260207):

  www/rubygem-gollum-lib
  www/rubygem-gollum-rugged_adapter

  And now about databases/rubygem-mysql, it builds OK in my poudriere
with Ruby 2.7 and default port options for the port. If you have time,
take a look at the build log and guess if it's related to RubyGems. If
something is worrying related to RubyGems, I'm interested to see the
log.

  Last, can someone with the permission update the PR title with 3.3.6
instead of 3.2.30?
Comment 66 Dan Langille freebsd_committer freebsd_triage 2022-01-27 14:25:55 UTC
(In reply to Thibault Jouan from comment #65)

I can't analyze the databases/rubygem-mysql  failure just not, but I can copy/paste it to: https://gist.github.com/dlangille/3a22bc234bd90d637a752dbd4030d97e
Comment 67 Thibault Jouan 2022-01-27 15:18:26 UTC
(In reply to Dan Langille from comment #66)
  I took a quick look at it, I don't think it's related to the patch,
thank you for the log.

  I could reproduce the build error with MySQL 8, in make.conf:

    DEFAULT_VERSIONS= mysql=8.0

  With default MySQL (5.7) there are no issue. It seems that the gem
itself is not compatible with MySQL 8:

    https://github.com/luislavena/mysql-gem/issues/35#issuecomment-399764286
    (build error is similar)

    https://github.com/luislavena/mysql-gem
    "As January 1st, 2015, this project has become unmaintained"


  So in my opinion, we can ignore that build error in the context of
this PR.
Comment 68 Dan Langille freebsd_committer freebsd_triage 2022-01-27 18:06:58 UTC
(In reply to Thibault Jouan from comment #67)
Testing with the recently added patch (2022-01-27 11:30 UTC), only 9 failures against the ports tree as of 2bde8a83e68d522b09ad38199bab5419dc19f200 (Wed Jan 26 15:59:20 2022 +0100).


* devel/rubygem-xdg
* devel/rubygem-xdg5
* graphics/rubygem-objectdetect
* mail/rubygem-tmail	
* science/cdcl
* www/rubygem-gitlab-gollum-lib
* www/rubygem-gollum-lib
* www/rubygem-gollum-rugged_adapter	


Next I will run against 2564716ea1ef7424132779e3b74261c101955c6d
Comment 69 Dan Langille freebsd_committer freebsd_triage 2022-01-27 19:02:06 UTC
(In reply to Dan Langille from comment #68)
That additional run has the same failures. Nothing new to add here.
Comment 70 Thibault Jouan 2022-02-01 13:50:18 UTC
  I'm now tracking bug #261637 instead of bug #260207 for
`www/rubygem-gollum-lib' port. There is a patch attached in the PR.
If/when it's commited, I'll update `devel/ruby-gems' one.

  No new version of Ruby Gems was released since 3.3.6.

  I went to #rubyports IRC channel (EFnet) a few days ago, asking for
help and/or advice in the process of commiting this patch, but no
activity there so far.
Comment 71 Dan Langille freebsd_committer freebsd_triage 2022-02-01 15:46:28 UTC
(In reply to Thibault Jouan from comment #70)

I'm there on IRC now too.
Comment 72 Dan Langille freebsd_committer freebsd_triage 2022-02-02 14:10:21 UTC
Worst case: we wait two weeks for maintainer timeout? Then I commit.
Comment 73 Sir l33tname 2022-02-02 18:15:22 UTC
So sorry for the additional work, i try to get bug #261637 merged.
Because i would love to get this one as well as it unblocks ruby 3.0.
Let me know if i can do anything else.
Comment 74 Thibault Jouan 2022-02-03 15:01:32 UTC
(In reply to Dan Langille from comment #72)

  I'm happy to learn that it's possible. Of course I would wait for
the last gemspec patch to be patched, verify again the patch one last
time, and probably make a new list of ports that we could test build
(in case there was new gemspec patches added to the tree since then).

  Then my last concern would probably be about reviewing it, my hope is
that some people in this PR reviewed it, maybe some are using it, and
they had no issue so didn't report anything.
Comment 75 Thibault Jouan 2022-02-03 15:01:59 UTC
(In reply to Sir l33tname from comment #73)

  No worry here, it's only one in many other ports having gemspec
patches, for reasons outside our control anyway. By noticing your
original PR and reading the comments I saw an opportunity to be lazy
and save some work, after guessing that the gemspec patch would change
soon. I mentioned it for information, I hope it didn't lead to some
pressure :-)
Comment 76 deivid.rodriguez 2022-02-07 15:55:22 UTC
Hello!

Forgive me if this is not the right place, but I wanted to ask whether you faced an issue similar to https://github.com/rubygems/rubygems/issues/5326 while packaging the latest RubyGems.

One of the upstream maintainers uses FreeBSD and run into that issue while running our tests on here system.

Thanks!
Comment 77 Dan Langille freebsd_committer freebsd_triage 2022-02-14 20:02:59 UTC
(In reply to deivid.rodriguez from comment #76)
I don't know enough ruby to answer.
Comment 78 deivid.rodriguez 2022-02-14 20:43:09 UTC
Well, thanks for your reply nevertheless!
Comment 79 Dan Langille freebsd_committer freebsd_triage 2022-02-14 20:50:45 UTC
With 261637 closed, is there anything preventing this PR from progressing?
Comment 80 Dan Langille freebsd_committer freebsd_triage 2022-02-14 20:51:10 UTC
I'm taking this PR after conversation with zi@
Comment 81 Dan Langille freebsd_committer freebsd_triage 2022-02-14 21:11:38 UTC
Patching against head, I get failures on these files:

* devel/rubygem-azure-storage-common/files/patch-gemspec
* devel/rubygem-dynflow/files/patch-gemspec
* mail/rubygem-net-imap/files/patch-gemspec
* mail/rubygem-net-pop/files/patch-gemspec
* mail/rubygem-net-smtp/files/patch-gemspec
* sysutils/rubygem-fluentd/files/patch-gemspec
* www/rubygem-gollum-lib/files/patch-gemspec - I removed that via 261637
* www/rubygem-gollum-rugged_adapter/files/patch-gollum-rugged__adapter.gemspec

I suspect the patches for these bits have been overtaken by events.
Comment 82 Dan Langille freebsd_committer freebsd_triage 2022-02-14 22:30:36 UTC
(In reply to Dan Langille from comment #81)

I used the wrong patch above.

Using the latest patch, the list is:

* sysutils/rubygem-fluentd/files/patch-gemspec
* textproc/rubygem-nokogiri/files/patch-nokogiri.gemspec
Comment 83 Dan Langille freebsd_committer freebsd_triage 2022-02-14 22:31:55 UTC
Error on building against head:

cd /wrkdirs/usr/ports/devel/ruby-gems/work/rubygems-3.3.6/ && /usr/bin/env GEM_HOME=/wrkdirs/usr/ports/devel/ruby-gems/work/stage/usr/local/lib/ruby/gems/2.7 /usr/local/bin/ruby27 setup.rb --destdir=/wrkdirs/usr/ports/devel/ruby-gems/work/stage  --no-regenerate-binstubs  --no-regenerate-plugins  --format-executable
ERROR:  While executing gem ... (Errno::ENOENT)
    No such file or directory @ dir_chdir - bundler/lib
*** Error code 1
Comment 84 Thibault Jouan 2022-02-17 15:52:47 UTC
Created attachment 231896 [details]
update to 3.3.7

  Patch updated for RubyGems 3.3.7 (instead of 3.3.6), and to update
changes in gem ports having gemspec patches:

devel/rubygem-pdk gemspec patch updated (deeaa184 changed line endings)
sysutils/rubygem-fluentd gemspec patch removed (35a8361d)
sysutils/rubygem-r10k gemspec patch updated (fb2191e8 updated the port)
www/rubygem-gollum-lib patch reintroduced (see comment #65)
www/rubygem-gollum-rugged_adapter patch reintroduced (see comment #65)

  I did the update with port tree at change 57815a47 (few hours ago),
then rebased my change on 3fcb3729 (few minutes ago).

  I updated the list of gem ports from comment #59:
http://vault.a13.fr/media/rg-test_build-ports_list.txt


(In reply to Dan Langille from comment #83)
> Error on building against head:
[…]
>     No such file or directory @ dir_chdir - bundler/lib

  I suspect the patch did not apply cleanly like in comment #44.
Comments #46 to #49 might have the solution, which I guess is creating
`devel/ruby-gems/files' directory or using `patch -p0'. I guess
applying with git would work too.


(In reply to deivid.rodriguez from comment #76)
[…]
> an issue similar to https://github.com/rubygems/rubygems/issues/5326 while packaging the latest RubyGems.

  It looks like the issue is running the test suite with `rake test`
with version "3.3.0.dev", and it would not pass. I can't reproduce
right now but observe a few things:
* 3.3.0.dev has no corresponding released archive:
  https://rubygems.org/rubygems/rubygems-3.3.0.dev.tgz is 403
* The Rakefile is not included in the releases, so they don't use the
  FreeBSD port, but probably directly the sources from the repository
* They don't use the lang/ruby{27,30,31} FreeBSD ports but a custom
  user install, I hope the issue still happens with a normal Ruby
  install.

  At first view it does not seem related to the FreeBSD port (since
they don't use it, and probably same for Ruby). I'll try to look into
it, but the test suite now depends on bundler if I'm correct
(test/rubygems/helper.rb for example). In the mean time I can send
various patches by private mail, patches I'm using to workaround some
issues. Maybe they'll provide a hint, I see one of them refer to
change 91a96d01 in upstream repository, I'm not sure if it could be
related.
  Last thing I can write is that the FreeBSD port does not use yet the
"operating system defaults" feature (I'll look into it after we commit
current update).
Comment 85 Dan Langille freebsd_committer freebsd_triage 2022-02-18 20:18:19 UTC
(In reply to Thibault Jouan from comment #84)

Patch applies cleanly here against head via: patch -p0

Builds cleanly against 13.0 and 12.3

My next task: build againt the list of ports which depend upon devel/ruby-gems

If that succeeds, do we agree to commit?

I have approval from ruby@ (via zi) to proceed after clean compiles.
Comment 86 Dan Langille freebsd_committer freebsd_triage 2022-02-18 21:09:40 UTC
(In reply to Dan Langille from comment #85)
failed ports are:

www/rubygem-gitlab-gollum-lib:patch
science/cdcl:fetch devel/rubygem-xdg5:stage 
devel/rubygem-xdg:stage mail/rubygem-tmail:stage
www/rubygem-gollum:patch
graphics/rubygem-objectdetect:stage

I am looking at the gollum stuff first.
Comment 87 Dan Langille freebsd_committer freebsd_triage 2022-02-18 21:59:27 UTC
Noticed this:

[dvl@ava-pkg-02prd:/usr/dvl/main] $ find . -name '*.rej'
./Oops.rej
./textproc/rubygem-nokogiri/files/patch-gemspec.rej
./sysutils/rubygem-r10k/files/patch-r10k.gemspec.rej
./sysutils/rubygem-chef/files/patch-gemspec.rej
./www/rubygem-gollum/files/patch-gemspec.rej
./devel/rubygem-pdk/files/patch-pdk.gemspec.rej
Comment 88 Dan Langille freebsd_committer freebsd_triage 2022-02-18 22:25:03 UTC
I have given up for today.  I don't know.
Comment 89 Thibault Jouan 2022-02-19 13:02:34 UTC
Created attachment 231942 [details]
update to 3.3.7

  I have updated the patch.

(In reply to Dan Langille from comment #86)

  Those are expected failure, not related to the patch:

devel/rubygem-xdg5: requires Ruby 3.0
devel/rubygem-xdg: requires Ruby 3.1
graphics/rubygem-objectdetect: port explicitly marked broken
mail/rubygem-tmail: port explicitly marked broken
science/cdcl: marked broken, unfetchable

  Those I updated their gemspec patches, they should build now:

www/rubygem-gitlab-gollum-lib: gemspec patch updated
www/rubygem-gollum: gemspec patch updated


(In reply to Dan Langille from comment #87)

  *.rej files I suspect they are leftovers after the application of
an outdated patch, and that they were still there after application of
a more recent patch.
Comment 90 Dan Langille freebsd_committer freebsd_triage 2022-02-19 14:35:54 UTC
Patch applied. Now waiting on build of 2600+ ports
 
I confirmed the .rej issue by applying the patch into a different working copy of the ports treet.
Comment 91 Dan Langille freebsd_committer freebsd_triage 2022-02-19 15:09:55 UTC
My first build attempt results in the expected 5 failures.

sudo poudriere bulk -p dvl-FreeBSD -j 130R -f /tmp/ruby-gems-buildlist.sorted

Good. Now I'm building them all from scratch:

sudo poudriere bulk -p dvl-FreeBSD -j 130R -C -f /tmp/ruby-gems-buildlist.sorted
Comment 92 Dan Langille freebsd_committer freebsd_triage 2022-02-19 15:10:24 UTC
The buildlist: https://gist.github.com/dlangille/a7f05fa277727ba0604355bb41e6df09
Comment 93 Dan Langille freebsd_committer freebsd_triage 2022-02-19 15:56:43 UTC
[00:48:08] Failed ports: 
science/cdcl:fetch
devel/rubygem-xdg5:stage
devel/rubygem-xdg:stage
mail/rubygem-tmail:stage 
graphics/rubygem-objectdetect:stage

[00:48:08] Skipped ports: science/rubygem-ruby-dcl (because of cdcl-5.4.8_1)

[130R-dvl-FreeBSD] [2022-02-19_15h07m14s] [committing:] Queued: 2644 Built: 2638 Failed: 5    Skipped: 1    Ignored: 0    Tobuild: 0     Time: 00:48:08

I think we are good to commit. Do you agree?
Comment 94 Thibault Jouan 2022-02-20 11:09:38 UTC
(In reply to Dan Langille from comment #93)

I agree to commit.

If possible, can we keep most of the notes from my original commit (in the patch too) as context, for the git history? So that if there are any issue, and I'm not available to look into it, at least people will know choices I made and why.
Comment 95 Sir l33tname 2022-02-20 11:28:02 UTC
I made a build today on 10120efedc79c6caf872a32f65e241631af3919e with this patch
and everything i build for my local setup is building successfully
and a quick sanity check looks good to me.

So let's get this done!
Comment 96 Dan Langille freebsd_committer freebsd_triage 2022-02-20 12:12:16 UTC
(In reply to Sir l33tname from comment #95)
Working on this now.
Comment 97 commit-hook freebsd_committer freebsd_triage 2022-02-20 13:07:24 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=7b279c9e09e1d27a385b7400842622b03a60c5bb

commit 7b279c9e09e1d27a385b7400842622b03a60c5bb
Author:     Dan Langille <dvl@FreeBSD.org>
AuthorDate: 2022-02-20 12:13:41 +0000
Commit:     Dan Langille <dvl@FreeBSD.org>
CommitDate: 2022-02-20 13:06:12 +0000

    devel/ruby-gems: Update to 3.3.7

    These are fixes Ruby 3.0

    Changes were compiled against all dependencies of devel/ruby-gems and
    they all built cleanly.

    There are five known build failures, all prexisting:

    * devel/rubygem-xdg5: requires Ruby 3.0
    * devel/rubygem-xdg: requires Ruby 3.1
    * graphics/rubygem-objectdetect: port explicitly marked broken
    * mail/rubygem-tmail: port explicitly marked broken
    * science/cdcl: marked broken, unfetchable

    Plus, science/rubygem-ruby-dcl gets skipped because of science/cdcl

    PR:             258108
    Approved by:    ruby (zi@)

 Mk/Scripts/qa.sh                                   |   2 +-
 Mk/Uses/gem.mk                                     |   7 +
 archivers/rubygem-fpm/files/patch-gemspec          |  36 +--
 archivers/rubygem-minitar-cli/files/patch-gemspec  |  18 +-
 databases/rubygem-dm-types/files/patch-gemspec     |  20 +-
 deskutils/rubygem-shellplay/files/patch-gemspec    |  18 +-
 devel/ruby-gems/Makefile                           |  17 +-
 devel/ruby-gems/distinfo                           |   6 +-
 .../files/patch-bundler-remove.patch (new)         |  42 ++++
 devel/ruby-gems/pkg-plist                          |  35 ++-
 devel/rubygem-activesupport4/files/patch-gemspec   |  18 +-
 devel/rubygem-astrolabe/files/patch-gemspec        |  18 +-
 devel/rubygem-azure_sdk/files/patch-gemspec        |  20 +-
 devel/rubygem-cf-uaa-lib/files/patch-gemspec       |  16 +-
 .../files/patch-chronic__duration.gemspec          |  18 +-
 .../rubygem-cucumber-messages4/files/patch-gemspec |  18 +-
 devel/rubygem-errand/files/patch-gemspec           |  16 +-
 devel/rubygem-fog/files/patch-gemspec              |  32 +--
 devel/rubygem-gettext-setup/files/patch-gemspec    |  22 +-
 devel/rubygem-github_api/files/patch-gemspec       |  18 +-
 .../rubygem-gitlab-pygments.rb/files/patch-gemspec |  18 +-
 devel/rubygem-gitlab_git/files/patch-gemspec       |  24 +-
 devel/rubygem-jeweler/files/patch-gemspec          |  20 +-
 devel/rubygem-kafo/files/patch-gemspec             |  20 +-
 devel/rubygem-mongo/files/patch-gemspec            |   6 +-
 devel/rubygem-ms_rest/files/patch-gemspec          |  20 +-
 devel/rubygem-omnibus/files/patch-gemspec          |  20 +-
 .../files/patch-gemspec                            |  18 +-
 devel/rubygem-pdk/files/patch-pdk.gemspec          |  88 ++-----
 devel/rubygem-pleaserun/files/patch-gemspec        |  20 +-
 devel/rubygem-pry-byebug/files/patch-gemspec       |  18 +-
 devel/rubygem-rash/files/patch-gemspec             |  18 +-
 devel/rubygem-rubygems-mirror/files/patch-gemspec  |  18 +-
 devel/rubygem-ruport/files/patch-gemspec           |  22 +-
 devel/rubygem-sidetiq/files/patch-gemspec          |  20 +-
 .../files/patch-gemspec                            |  20 +-
 devel/rubygem-terrapin/files/patch-gemspec         |  18 +-
 devel/rubygem-tilt1/files/patch-gemspec            |  28 +--
 devel/rubygem-travis/files/patch-gemspec           |  20 +-
 devel/rubygem-tty/files/patch-gemspec              |  84 +++----
 devel/rubygem-turn/files/patch-gemspec             |  18 +-
 emulators/rubygem-fission/files/patch-gemspec      |  16 +-
 graphics/rubygem-gitlab_emoji/files/patch-gemspec  |  18 +-
 irc/rubygem-cogbot/files/patch-gemspec             |  46 ++--
 mail/rubygem-vmail/files/patch-gemspec             |  20 +-
 multimedia/rubygem-av/files/patch-av.gemspec       |  20 +-
 net-im/rubygem-lita/files/patch-gemspec            |  18 +-
 net-im/rubygem-termtter/files/patch-gemspec        |  26 +-
 net-im/rubygem-tinder/files/patch-gemspec          |  18 +-
 net-mgmt/rubygem-oxidized-web/files/patch-gemspec  |  51 ++--
 .../rubygem-oxidized/files/patch-oxidized.gemspec  |  20 +-
 .../files/patch-riemann-client.gemspec             |  33 +--
 .../files/patch-riemann-dash.gemspec               |  39 +--
 .../files/patch-riemann-tools.gemspec              |  25 +-
 net-mgmt/rubygem-visage-app/files/patch-gemspec    |  32 +--
 net/rubygem-amqp-utils/files/patch-gemspec         |  34 +--
 net/rubygem-fog-vsphere/files/patch-gemspec        |  18 +-
 net/rubygem-kas-grpc/files/patch-kas-grpc.gemspec  |  21 +-
 .../files/patch-spamcheck.gemspec                  |  21 +-
 net/rubygem-t/files/patch-gemspec                  |  23 +-
 net/rubygem-train/files/patch-gemspec              |  24 +-
 net/rubygem-tweetstream/files/patch-gemspec        |  20 +-
 net/rubygem-twitter-stream/files/patch-gemspec     |  24 +-
 net/rubygem-twitter/files/patch-gemspec            |  28 +--
 .../files/patch-doorkeeper-openid__connect.gemspec |  18 +-
 sysutils/rhc/files/patch-gemspec                   |  20 +-
 sysutils/rubygem-backup/files/patch-gemspec        | 268 ++++++++++-----------
 sysutils/rubygem-bolt/files/patch-bolt.gemspec     |  74 ++----
 sysutils/rubygem-bosh-gen/files/patch-gemspec      |  20 +-
 sysutils/rubygem-chef/files/patch-gemspec          |  20 +-
 .../files/patch-choria-mcorpc-support.gemspec      |  23 +-
 sysutils/rubygem-license_scout/files/patch-gemspec |  28 +--
 sysutils/rubygem-r10k/files/patch-r10k.gemspec     |  75 ++----
 sysutils/rubygem-specinfra/files/patch-gemspec     |  41 +---
 textproc/rubygem-gitlab-grit/files/patch-gemspec   |  20 +-
 textproc/rubygem-html2text/files/patch-gemspec     |  18 +-
 textproc/rubygem-nokogiri/files/patch-gemspec      |  16 +-
 textproc/rubygem-nokogiri111/files/patch-gemspec   |  16 +-
 www/rubygem-actioncable5/files/patch-gemspec       |  20 +-
 www/rubygem-actioncable50/files/patch-gemspec      |  20 +-
 www/rubygem-actionpack4/files/patch-gemspec        |  20 +-
 www/rubygem-actionpack50/files/patch-gemspec       |  20 +-
 .../files/patch-gitlab-gollum-lib.gemspec          |  20 +-
 .../files/patch-gollum-lib.gemspec                 |  20 +-
 .../files/patch-gemspec                            |  24 +-
 www/rubygem-gollum/files/patch-gemspec             |  34 +--
 www/rubygem-http3/files/patch-gemspec              |  18 +-
 www/rubygem-jekyll/files/patch-gemspec             |  20 +-
 www/rubygem-jekyll3/files/patch-gemspec            |  30 +--
 www/rubygem-octopress/files/patch-gemspec          |  18 +-
 www/rubygem-platform-api/files/patch-gemspec       |  20 +-
 www/rubygem-rails4/files/patch-gemspec             |  20 +-
 www/rubygem-rest-client/files/patch-gemspec        |  20 +-
 www/rubygem-tumblr_client/files/patch-gemspec      |  22 +-
 94 files changed, 1191 insertions(+), 1342 deletions(-)
Comment 98 Dan Langille freebsd_committer freebsd_triage 2022-02-20 13:09:29 UTC
Thank you all for your help. Much appreciated.
Comment 99 Dan Langille freebsd_committer freebsd_triage 2022-02-20 15:43:04 UTC
*** Bug 260780 has been marked as a duplicate of this bug. ***
Comment 100 Dan Langille freebsd_committer freebsd_triage 2022-02-20 15:44:23 UTC
*** Bug 259491 has been marked as a duplicate of this bug. ***