Bug 131775

Summary: lang/ruby18: portupgrade fails to fetch files from ftp
Product: Ports & Packages Reporter: Helmut Schneider <jumper99>
Component: Individual Port(s)Assignee: Sergey Matveychuk <sem>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
patch-pkg_fetch none

Description Helmut Schneider 2009-02-17 11:50:00 UTC
[root@proxy ~]# portupgrade -PP mysql-client-5.0.75
--->  Checking for the latest package of 'databases/mysql50-client'
--->  Found a package of 'databases/mysql50-client': /usr/ports/packages/All/mysql-client-5.0.75.tbz (mysql-client-5.0.75)
--->  Fetching the package(s) for 'mysql-client-5.0.77' (databases/mysql50-client)
--->  Fetching mysql-client-5.0.77
fetch: ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/mysql-client-5.0.77.tbz: Bad Gateway
[...]

See the missing "/" between "proxy.domain.tld" and "pub".

pkg_add itself is fine:

[root@proxy ~]# pkg_add ftp://proxy.domain.tld/pub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/mysql-client-5.0.77.tbz
Fetching ftp://proxy.domain.tld/pub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/mysql-client-5.0.77.tbz... Done.
[root@proxy ~]#
Comment 1 Edwin Groothuis freebsd_committer freebsd_triage 2009-02-17 11:50:11 UTC
Responsible Changed
From-To: freebsd-ports-bugs->sem

Over to maintainer (via the GNATS Auto Assign Tool)
Comment 2 Helmut Schneider 2009-02-18 09:37:07 UTC
Appendix: Rolling back to ruby-1.8.6.287,1 solves the problem.
Comment 3 Jille 2009-02-22 19:29:19 UTC
A simpeler workaround is:
$ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg


-- Jille
Comment 4 Helmut Schneider 2009-02-25 12:28:49 UTC
From: "Jille Timmermans" <jille@quis.cx>

> A simpeler workaround is:
> $ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg

No, that's no workaround at all:

[root@teg ~]# PACKAGEROOT="ftp://proxy.domain.tld/" portupgrade -aPP
** Custom MAKE_ARGS or -m option is specified 
(-DWITH_BDB,-DWITH_POF,-DWITH_RAR,-DWITH_UNRAR,-DWITH_UNARJ,-DWITH_NOMARCH,-DWITH_UNZOO,-DWITH_TNEF)
** Trying package anyway, since -PP/--use-packages-only is specified
--->  Checking for the latest package of 'security/amavisd-new'
--->  Fetching the package(s) for 'amavisd-new-2.6.2_2,1' 
(security/amavisd-new)
--->  Fetching amavisd-new-2.6.2_2,1
fetch: 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tbz: 
Bad Gateway
** The command returned a non-zero exit status: 1
** Failed to fetch 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tbz
fetch: 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tgz: 
Bad Gateway
** The command returned a non-zero exit status: 1
** Failed to fetch 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tgz
** Failed to fetch amavisd-new-2.6.2_2,1
** Listing the failed packages (-:ignored / *:skipped / !:failed)
        ! amavisd-new-2.6.2_2,1 (fetch error)
--->  Fetching the latest package(s) for 'amavisd-new' 
(security/amavisd-new)
--->  Fetching amavisd-new
^C
Interrupted.
** Could not find the latest version (2.6.2_2,1)
** No package available: security/amavisd-new
** Listing the failed packages (-:ignored / *:skipped / !:failed)
        ! security/amavisd-new (amavisd-new-2.6.2,1)    (package not found)
[root@teg ~]#
Comment 5 Jille 2009-02-25 21:03:21 UTC
Helmut Schneider schreef:
> From: "Jille Timmermans" <jille@quis.cx>
> 
>> A simpeler workaround is:
>> $ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg
> 
> No, that's no workaround at all:
I am running 8-CURRENT of a few weeks old, all ports up to date (or
succeeding index, still need to figure out why that is).

I ran the same command (with another package), and it also worked...
("Can't" paste it due to not having X, a mail client, etc)

Would you mind trying:
# PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP <someport> ?

-- Jille

> 
> [root@teg ~]# PACKAGEROOT="ftp://proxy.domain.tld/" portupgrade -aPP
> ** Custom MAKE_ARGS or -m option is specified
> (-DWITH_BDB,-DWITH_POF,-DWITH_RAR,-DWITH_UNRAR,-DWITH_UNARJ,-DWITH_NOMARCH,-DWITH_UNZOO,-DWITH_TNEF)
> 
> ** Trying package anyway, since -PP/--use-packages-only is specified
> --->  Checking for the latest package of 'security/amavisd-new'
> --->  Fetching the package(s) for 'amavisd-new-2.6.2_2,1'
> (security/amavisd-new)
> --->  Fetching amavisd-new-2.6.2_2,1
> fetch:
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tbz:
> Bad Gateway
> ** The command returned a non-zero exit status: 1
> ** Failed to fetch
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tbz
> 
> fetch:
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tgz:
> Bad Gateway
> ** The command returned a non-zero exit status: 1
> ** Failed to fetch
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/amavisd-new-2.6.2_2,1.tgz
> 
> ** Failed to fetch amavisd-new-2.6.2_2,1
> ** Listing the failed packages (-:ignored / *:skipped / !:failed)
>        ! amavisd-new-2.6.2_2,1 (fetch error)
> --->  Fetching the latest package(s) for 'amavisd-new'
> (security/amavisd-new)
> --->  Fetching amavisd-new
> ^C
> Interrupted.
> ** Could not find the latest version (2.6.2_2,1)
> ** No package available: security/amavisd-new
> ** Listing the failed packages (-:ignored / *:skipped / !:failed)
>        ! security/amavisd-new (amavisd-new-2.6.2,1)    (package not found)
> [root@teg ~]#
Comment 6 Helmut Schneider 2009-02-26 09:23:57 UTC
From: "Jille Timmermans" <jille@quis.cx>
> Helmut Schneider schreef:
>> From: "Jille Timmermans" <jille@quis.cx>
>>
>>> A simpeler workaround is:
>>> $ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg
>>
>> No, that's no workaround at all:
> I am running 8-CURRENT of a few weeks old, all ports up to date (or
> succeeding index, still need to figure out why that is).
>
> I ran the same command (with another package), and it also worked...
> ("Can't" paste it due to not having X, a mail client, etc)
>
> Would you mind trying:
> # PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP <someport> ?

$PACKAGEROOT seems ignored:

[root@teg ~]# PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP cairo
--->  Checking for the latest package of 'graphics/cairo'
--->  Fetching the package(s) for 'cairo-1.8.6_1,1' (graphics/cairo)
--->  Fetching cairo-1.8.6_1,1
fetch: 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tbz: 
Bad Gateway
** The command returned a non-zero exit status: 1
** Failed to fetch 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tbz
fetch: 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tgz: 
Bad Gateway
** The command returned a non-zero exit status: 1
** Failed to fetch 
ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tgz
** Failed to fetch cairo-1.8.6_1,1
** Listing the failed packages (-:ignored / *:skipped / !:failed)
        ! cairo-1.8.6_1,1       (fetch error)
--->  Fetching the latest package(s) for 'cairo' (graphics/cairo)
^C
Interrupted.
** Could not find the latest version (1.8.6_1,1)
** No package available: graphics/cairo
** Listing the failed packages (-:ignored / *:skipped / !:failed)
        ! graphics/cairo (cairo-1.8.6,1)        (package not found)
[root@teg ~]# uname -rs
FreeBSD 7.1-RELEASE-p3
[root@teg ~]#
Comment 7 Jille 2009-02-26 20:22:07 UTC
Helmut Schneider schreef:
> From: "Jille Timmermans" <jille@quis.cx>
>> Helmut Schneider schreef:
>>> From: "Jille Timmermans" <jille@quis.cx>
>>>
>>>> A simpeler workaround is:
>>>> $ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg
>>>
>>> No, that's no workaround at all:
>> I am running 8-CURRENT of a few weeks old, all ports up to date (or
>> succeeding index, still need to figure out why that is).
>>
>> I ran the same command (with another package), and it also worked...
>> ("Can't" paste it due to not having X, a mail client, etc)
>>
>> Would you mind trying:
>> # PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP <someport> ?
> 
> $PACKAGEROOT seems ignored:
The only thing I can think of now is that you have some configfile
(portupgrade, pkg_add) that overwrites the environment setting.

-- Jille
> 
> [root@teg ~]# PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP cairo
> --->  Checking for the latest package of 'graphics/cairo'
> --->  Fetching the package(s) for 'cairo-1.8.6_1,1' (graphics/cairo)
> --->  Fetching cairo-1.8.6_1,1
> fetch:
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tbz:
> Bad Gateway
> ** The command returned a non-zero exit status: 1
> ** Failed to fetch
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tbz
> 
> fetch:
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tgz:
> Bad Gateway
> ** The command returned a non-zero exit status: 1
> ** Failed to fetch
> ftp://proxy.domain.tldpub/FreeBSD/ports/i386/packages-7.1-RELEASE/All/cairo-1.8.6_1,1.tgz
> 
> ** Failed to fetch cairo-1.8.6_1,1
> ** Listing the failed packages (-:ignored / *:skipped / !:failed)
>        ! cairo-1.8.6_1,1       (fetch error)
> --->  Fetching the latest package(s) for 'cairo' (graphics/cairo)
> ^C
> Interrupted.
> ** Could not find the latest version (1.8.6_1,1)
> ** No package available: graphics/cairo
> ** Listing the failed packages (-:ignored / *:skipped / !:failed)
>        ! graphics/cairo (cairo-1.8.6,1)        (package not found)
> [root@teg ~]# uname -rs
> FreeBSD 7.1-RELEASE-p3
> [root@teg ~]#
>
Comment 8 Helmut Schneider 2009-02-27 09:04:56 UTC
From: "Jille Timmermans" <jille@quis.cx>
> Helmut Schneider schreef:
>> From: "Jille Timmermans" <jille@quis.cx>
>>> Helmut Schneider schreef:
>>>> From: "Jille Timmermans" <jille@quis.cx>
>>>> 
>>>>> A simpeler workaround is:
>>>>> $ PACKAGEROOT="ftp://ftp.freebsd.org/" portinstall -PP x11/xorg
>>>> 
>>>> No, that's no workaround at all:
>>> I am running 8-CURRENT of a few weeks old, all ports up to date (or
>>> succeeding index, still need to figure out why that is).
>>> 
>>> I ran the same command (with another package), and it also worked...
>>> ("Can't" paste it due to not having X, a mail client, etc)
>>> 
>>> Would you mind trying:
>>> # PACKAGEROOT="ftp://ftp.freebsd.org/" portupgrade -PP <someport> ?
>> 
>> $PACKAGEROOT seems ignored:
> The only thing I can think of now is that you have some configfile
> (portupgrade, pkg_add) that overwrites the environment setting.

No. For sure. :)
Comment 9 Mark Linimon freebsd_committer freebsd_triage 2009-03-25 22:14:30 UTC
State Changed
From-To: open->feedback

Are you sure you aren't running -current?  A problem with path handling, 
leading to e.g. "freebsd.orgpub" being tried as a URL, was fixed in 
http://svn.freebsd.org/changeset/base/190142 . 


Comment 10 Mark Linimon freebsd_committer freebsd_triage 2009-03-25 22:14:30 UTC
Responsible Changed
From-To: sem->linimon

track, since I have been trying to chase this problem.
Comment 11 Tim Kientzle freebsd_committer freebsd_triage 2009-04-03 02:43:04 UTC
The following patch fixed it for me:

--- /usr/local/sbin/pkg_fetch   2009-02-25 18:01:54.000000000 -0800
+++ pkg_fetch   2009-03-23 21:13:42.000000000 -0700
@@ -318,7 +318,7 @@

    $pkg_site_uris.each do |uri_base|
      PKG_SUFFIXES.each do |suffix|
-      uri = uri_base + (subdir + '/' + pkgname + suffix)
+      uri = "#{uri_base}#{subdir}/#{pkgname}#{suffix}"
        path = path_base + suffix

        fetch(uri, path) and return path
Comment 12 Jaakko Heinonen 2009-05-12 19:18:35 UTC
Hi,

There seems to be a regression in ruby uri module. The test program
below prints ftp://localhostabc/ with ruby-1.8.7.72_1,1 while it prints
ftp://localhost/abc/ when using the uri module from ruby 1.8.6
patchlevel 287.

%%%
#!/usr/bin/env ruby
require 'uri'
print URI.parse("ftp://localhost/abc/def/") + "../"
print "\n"
%%%

Tim's patch doesn't help if you use the PACKAGESITE environment
variable.

-- 
Jaakko
Comment 13 Jaakko Heinonen 2009-05-19 16:23:03 UTC
On 2009-05-12, Jaakko Heinonen wrote:
> There seems to be a regression in ruby uri module.

Here is a patch to revert the specific change causing the problem. The
change was introduced between ruby-1.8.6 patchlevel 286 and ruby-1.8.7
patchlevel 72.

--- patch begins here ---
--- ./lib/uri/ftp.rb.orig	2008-04-19 14:56:22.000000000 +0300
+++ ./lib/uri/ftp.rb	2009-05-18 22:06:25.000000000 +0300
@@ -176,9 +176,9 @@
     #
     # This method will then return "/pub/ruby"
     #
-    def path
-      return @path.sub(/^\//,'').sub(/^%2F/i,'/')
-    end
+    #def path
+    #  return @path.sub(/^\//,'').sub(/^%2F/i,'/')
+    #end
 
     def to_s
       save_path = nil
--- patch ends here ---

-- 
Jaakko
Comment 14 Peter Hofer 2009-05-21 01:13:19 UTC
This problem is indeed caused by Ruby: when parsing an FTP URI with  
URI.parse, an instance of the URI::FTP subclass is returned. This subclass  
overwrites the generic path method and omits the leading slash, apparently  
for compliance with some RFC. This works fine as long as you use the path  
method by itself, but apparently the developer did not take into account  
that it's also used for building the URI's string representation,  
resulting in the missing separator after the host/port component.

I have attached a patch to address the problem in pkg_fetch. It avoids the  
URI.parse method and instead calls URI.split manually and creates a  
URI::Generic instance which does not have the faulty path handling of  
URI::FTP. I consider this fix to be clean enough for a permanent solution,  
but it should at least make a decent workaround.

Does anyone know if the Ruby developers are aware of this issue?
Comment 15 Helmut Schneider 2009-06-04 16:15:12 UTC
From: "Peter Hofer" <ph@desktopbsd.net>
> This problem is indeed caused by Ruby: when parsing an FTP URI with  
> URI.parse, an instance of the URI::FTP subclass is returned. This
> subclass   overwrites the generic path method and omits the leading
> slash, apparently   for compliance with some RFC. This works fine as
> long as you use the path   method by itself, but apparently the
> developer did not take into account   that it's also used for building
> the URI's string representation,   resulting in the missing separator
> after the host/port component.
> 
> I have attached a patch to address the problem in pkg_fetch. It avoids
> the   URI.parse method and instead calls URI.split manually and creates
> a   URI::Generic instance which does not have the faulty path handling
> of   URI::FTP. I consider this fix to be clean enough for a permanent
> solution,   but it should at least make a decent workaround.

The patch works fine indeed.

Helmut
Comment 16 Mark Linimon freebsd_committer freebsd_triage 2009-06-05 13:30:22 UTC
State Changed
From-To: feedback->open

The most recent comments seem to imply that this is a regression in 
ruby18, and have supplied a patch.  Over to ruby maintainer for comments. 


Comment 17 Mark Linimon freebsd_committer freebsd_triage 2009-06-05 13:30:22 UTC
Responsible Changed
From-To: linimon->stas
Comment 18 Stanislav Sedov freebsd_committer freebsd_triage 2009-06-05 13:41:39 UTC
Responsible Changed
From-To: stas->sem

It looks like that it is a bug in pkg_fetch util of portupgrade.
Comment 19 dfilter service freebsd_committer freebsd_triage 2009-06-18 13:43:40 UTC
stas        2009-06-18 12:43:26 UTC

  FreeBSD ports repository

  Modified files:
    Mk                   bsd.ruby.mk 
  Added files:
    lang/ruby18/files    patch-class.c patch-gc.c 
                         patch-lib_uri_generic.rb 
  Log:
  - Fix the bug in ruby GC when object finalizer were never called. This has
    been triggered by portupgrade which uses finalizers to remove lockfiles.
  - Fix the bug in URI module that smashed uri paths in merge method for
    ftp uris. [1]
  - Fix segmentation fault when cloning objects with dynamically created
    methods (via instance_eval). [2]
  
  PR:             ports/131775 [1], ports/135533 [2]
  Submitted by:   Yoshisato YANAGISAWA <osho@pcc-software.org> [2],
                  Helmut Schneider <jumper99@gmx.de> [1],
                  Jaakko Heinonen <jh@saunalahti.fi> [1],
                  Peter Hofer <ph@desktopbsd.net> [1]
  Obtained from:  ruby_1_8 svn tree (rev 22679) [2]
  
  Revision  Changes    Path
  1.183     +2 -2      ports/Mk/bsd.ruby.mk
  1.1       +11 -0     ports/lang/ruby18/files/patch-class.c (new)
  1.3       +11 -0     ports/lang/ruby18/files/patch-gc.c (new)
  1.1       +19 -0     ports/lang/ruby18/files/patch-lib_uri_generic.rb (new)
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
Comment 20 dfilter service freebsd_committer freebsd_triage 2009-06-18 14:01:32 UTC
stas        2009-06-18 13:01:02 UTC

  FreeBSD ports repository

  Modified files:
    Mk                   bsd.ruby.mk 
  Added files:
    lang/ruby19/files    patch-lib_uri_generic.rb 
  Log:
  - Fix the bug in URI module that smashed uri paths in merge method for
      ftp uris.
  
  PR:             ports/131775
  Reported by:    Helmut Schneider <jumper99@gmx.de>
  
  Revision  Changes    Path
  1.184     +2 -2      ports/Mk/bsd.ruby.mk
  1.1       +19 -0     ports/lang/ruby19/files/patch-lib_uri_generic.rb (new)
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
Comment 21 Stanislav Sedov freebsd_committer freebsd_triage 2009-06-18 14:11:01 UTC
State Changed
From-To: open->closed

Committed. Thanks!