Bug 67445 - [PATCH] mirror don't fetch files with $ in names when update_local=yes
Summary: [PATCH] mirror don't fetch files with $ in names when update_local=yes
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Pav Lucistnik
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-06-01 11:10 UTC by Dmitry A Grigorovich
Modified: 2004-06-01 15:29 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dmitry A Grigorovich 2004-06-01 11:10:14 UTC
Use port ftp/mirror and include update_local=yes in config
This mean, that mirror must update only files, which present in local directory

To do this mirror scan local directory and build get_patt pattern.
Pattern get_patt have perl RE syntax, like '^file1$|^file2$'.
This pattern must match only local files and no more.

Next step, mirror list remote files and update only files,
which match get_patt.

But mirror incorrectly build get_patt pattern
if file name have special characters like '^', '$' and other.
Pattern don't match this files and accordingly mirror
don't update this files from remote site.

Fix: Put follow file into /usr/ports/ftp/mirror/files
Remove port ftp/mirror
Build and install port ftp/mirror



Try again test above

3) Create empty dir local
   Create empty files with names: normal_file, $bad_file, bad_file$

odip$ ls -l local
total 0
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 17:03 $bad_file
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 17:03 bad_file$
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 17:04 normal_file

4) Run mirror in debug mode

odip@hydra$ mirror -d -p test ./config
package=test ftp.bionet.nsc.ru:/test -> /home/odip/mirror/local
Scanning local directory /home/odip/mirror/local
get_patt = ^\$bad_file$|^bad_file\$$|^normal_file$
Scanning remote directory /test
compare directories (src 3, dest 3)
Got bad_file$ 9 0
Got $bad_file 9 0
Got normal_file 12 0

disconnecting from ftp.bionet.nsc.ru
All done, Exiting

5) Show local directory

odip@hydra$ ls -l local
total 6
-r--r--r--  1 odip  odip   9  1 ÉÀÎ 09:44 $bad_file
-r--r--r--  1 odip  odip   9  1 ÉÀÎ 09:44 bad_file$
-r--r--r--  1 odip  odip  12  1 ÉÀÎ 09:16 normal_file


All is ok - files mirrored--8FcoXvuPkhOnBo0pEEojXnqANatRU1NvcCBFNIrUFCElgG5C
Content-Type: text/plain; name="patch-ba"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="patch-ba"

--- mirror.pl.orig	Thu Aug  7 11:47:06 2003
+++ mirror.pl	Fri Apr 30 14:21:48 2004
@@ -1323,8 +1323,11 @@
 
 		# Create a get_patt from the contents of the local directory
 		if( $update_local && $#get_top >= 0 ){
-			$get_patt = '^' . join( '|^', @get_top );
-			$get_patt =~ s/$squished//g;
+			my ($path,$re,@re_patt);
+			#$get_patt = '^' . join( '|^', @get_top );
+			#$get_patt =~ s/$squished//g;
+			foreach $path ( @get_top ) { push @re_patt, "\^\Q$path\E\$"; }
+			$get_patt= join( '|', @re_patt );
 			&msg( "get_patt = $get_patt\n" ) if $debug;
 		}
How-To-Repeat: 
1) Create file config with content:

package=test
        comment=test
        site=ftp.bionet.nsc.ru
        passive_ftp=true
        # where to start pulling files back from
        remote_dir=/test
        # where to put the files on your machine
        local_dir=/home/odip/mirror/local
	update_local=yes

2) Put non empty files to ftp://ftp.bionet.nsc.ru/test/
    with names: normal_file, $bad_file, bad_file$

root# ls -l /home/ftp/test
total 6
-rw-r--r--  1 root  wheel   9 Jun  1 16:44 $bad_file
-rw-r--r--  1 root  wheel   9 Jun  1 16:44 bad_file$
-rw-r--r--  1 root  wheel  12 Jun  1 16:16 normal_file

3) Create empty dir local
   Create empty files with names: normal_file, $bad_file, bad_file$

odip$ ls -l local
total 0
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 16:58 $bad_file
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 16:58 bad_file$
-rw-r--r--  1 odip  odip  0  1 ÉÀÎ 16:58 normal_file

4) Run mirror in debug mode

odip@hydra$ mirror -d -p test ./config
package=test ftp.bionet.nsc.ru:/test -> /home/odip/mirror/local
Scanning local directory /home/odip/mirror/local
get_patt = ^$bad_file|^bad_file$|^normal_file
Scanning remote directory /test
compare directories (src 3, dest 3)
No files to transfer
unlink /home/odip/mirror/local/bad_file$
unlink /home/odip/mirror/local/$bad_file

disconnecting from ftp.bionet.nsc.ru
All done, Exiting


ERROR: get_patt in invalid and mirror remove two files !!!

5) Show local directory

odip@hydra$ ls -l local
total 2
-r--r--r--  1 odip  odip  12  1 ÉÀÎ 09:16 normal_file
Comment 1 Pav Lucistnik freebsd_committer freebsd_triage 2004-06-01 12:32:42 UTC
Dear maintainer of FreeBSD port ftp/mirror, please take a look at

http://www.freebsd.org/cgi/query-pr.cgi?q=67445

Do you approve this patch?

-- 
Pav Lucistnik <pav@oook.cz>
              <pav@FreeBSD.org>

You take the red pill, you stay in Wonderland, and I show you how deep
the rabbit hole goes....
Comment 2 Pav Lucistnik freebsd_committer freebsd_triage 2004-06-01 12:32:46 UTC
State Changed
From-To: open->feedback

Asked maintainer for approval. 


Comment 3 Pav Lucistnik freebsd_committer freebsd_triage 2004-06-01 12:32:46 UTC
Responsible Changed
From-To: freebsd-ports-bugs->pav

Handle.
Comment 4 Ryan Thompson 2004-06-01 15:10:18 UTC
Pav Lucistnik wrote to freebsd-gnats-submit@FreeBSD.org, odip@bionet.nsc.ru...:

> Dear maintainer of FreeBSD port ftp/mirror, please take a look at
>
> http://www.freebsd.org/cgi/query-pr.cgi?q=67445
>
> Do you approve this patch?

It looks fine to me. Thanks!

- Ryan

-- 
  Ryan Thompson <ryan@sasknow.com>

  SaskNow Technologies - http://www.sasknow.com
  901-1st Avenue North - Saskatoon, SK - S7K 1Y4

        Tel: 306-664-3600   Fax: 306-244-7037   Saskatoon
  Toll-Free: 877-727-5669     (877-SASKNOW)     North America
Comment 5 Pav Lucistnik freebsd_committer freebsd_triage 2004-06-01 15:28:45 UTC
State Changed
From-To: feedback->closed

Committed, thanks! Please also try to contact author 
about this patch