Bug 33170 - zip -T [zip file] fails with message about missing end signature
Summary: zip -T [zip file] fails with message about missing end signature
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Andrey A. Chernov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2001-12-25 15:50 UTC by Idar Tollefsen
Modified: 2002-05-31 11:18 UTC (History)
0 users

See Also:


Attachments
ITCVEN.zip (1.68 KB, application/x-zip-compressed)
2001-12-26 15:10 UTC, Idar Tollefsen
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Idar Tollefsen 2001-12-25 15:50:00 UTC
While building Mozilla wiht --enable-chrome-format=jar (the default),
the build would fail while testing the integrity of en-US.jar, which
is just a zip file.

Upon further investigation, it turned out that doing
zip test.zip anyfile
followed by
zip -T test.zip
always would yield an error about a missing end signature.

Here is the exact message:

zip warning: missing end signature--probably not a zip file (did you
zip warning: remember to use binary mode when you transferred it?)

zip error: Zip file structure invalid (test.zip)

This is done with zip-2.3 on a 21164A running 4.4-STABLE.
It has not been established wether this is spesific for the Alpha or
not, but I suspect it is since breaking the Mozilla build is something
that the i386 crowd would have screamed about a long time ago.

How-To-Repeat: Run the following (on an Alpha):
zip [name of zip file] [any file]
zip -T [name of zip file]
Comment 1 Alan Eldridge 2001-12-25 15:51:50 UTC
On Tue, Dec 25, 2001 at 07:41:11AM -0800, Idar Tollefsen wrote:
>It has not been established wether this is spesific for the Alpha or
>not, but I suspect it is since breaking the Mozilla build is something
>that the i386 crowd would have screamed about a long time ago.

Ooh, look ... another big-endian vs little-endian problem, I'd guess.

-- 
Alan Eldridge
Comment 2 Idar Tollefsen 2001-12-25 23:16:48 UTC
>>It has not been established wether this is spesific for the Alpha
>>or not, but I suspect it is since breaking the Mozilla build is
>>something that the i386 crowd would have screamed about a long
>>time ago.

>Ooh, look ... another big-endian vs little-endian problem,
>I'd guess.

Altough not my strong side, I tought both Intel processors and
Alphas where little-endian.

Anyway, I ment no offence to anyone running i386, which indeed
would be the majority of FreeBSD users. I simply ment that since
there are more i386 users, they would most likely have caught the
problem a long time ago. Minor platforms, as I suppose the Alpha is,
could have these problems laying around for a long periode of time
before anyone discovers them simply because there are fewer people
using this platform.

I felt the need to clearify this as you seemed to take offence
to the way my message was formulated.


_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com
Comment 3 Alan Eldridge 2001-12-25 23:56:07 UTC
Hmm, it's not even zip. All that zip -T does is invoke "unzip -t".
So we need to look at a different program entirely. I'll see if I can
come up with any ideas.

-- 
Alan Eldridge
Pmmfmffmmfmp mmmpppppffmpmfpmpppff $PffMmmPppMpmPpfPpm mfpmmmmmfpmpmpppff.
Comment 4 Alan Eldridge 2001-12-26 00:34:43 UTC
What does i386 zip/unzip say about the file zipped on the Alpha? What does 
the Alpha say about a file zipped on i386? IOW, I'm wondering whether zip is 
writing a bad file.... It is looking for the the sequence "PK\x05\x06" and is 
doing so by converting bytes in the file to a long int and comparing against 
0x06054b50 (little endian).

Can you mail me or post a small zip file that it fails upon?

-- 
Alan Eldridge <ports@geeksrus.net>
Pmmfmffmmfmp mmmpppppffmpmfpmpppff $PffMmmPppMpmPpfPpm mfpmmmmmfpmpmpppff.
Comment 5 Idar Tollefsen 2001-12-26 15:10:22 UTC
>What does i386 zip/unzip say about the file zipped on the Alpha?
>What does the Alpha say about a file zipped on i386? IOW, I'm
>wondering whether zip is writing a bad file.... It is looking for
>the the sequence "PK\x05\x06" and is doing so by converting bytes
>in the file to a long int and comparing against 0x06054b50
>(little endian).

I don't have access to FreeBSD, or any other Unix system, running
on i386.

Just to clarify one thing: Only testing the integrity of the zip
file fails, zipping and unzipping works just fine anyway.

I tested a zip file zipped on the Alpha with WinZip 8.0 on Windows
2000. It worked just fine, no errors. I then zipped a file with
WinZip and testet it on the Alpha. zip -T again complained about
missing end singature, but it unzipped just fine anyway, as usual.

I stumbled upon another thing as well: I forgot to type the name
of the zip file once and the same message poped up. That is,
instead of doing
zip [name of zip file] [file to zip]
I did
zip [file to zip]
and it complained about a missing end signature.

>Can you mail me or post a small zip file that it fails upon?

A small zip file zipped on the Alpha which fails the test is
attached.

And yes, I think it would be a good idea to test this on another
Alpha, even tough it's not running FreeBSD. It could tell us if
this is Alpha spesific or not, independent of OS.


- IT


_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp.
Comment 6 mvp 2002-02-20 04:34:38 UTC
  I have run into the same problem on FreeBSD/alpha system.
Apparently, I have figured out what's going on. It has nothing to do
with 64-bit alignment or endianness. It's all about long-standing bug
in FreeBSD - http://www.freebsd.org/cgi/query-pr.cgi?pr=kern%2F6184

Let's take a look at infozip source, file zipfile.c around line 669:
...
    if (fseek(f, -4096L, SEEK_END) == 0) {
      zipbeg = (ulg) (ftell(f) + 4096L);
      while (!found && zipbeg >= 4096) {
        zipbeg -= 4096L;
        buf[4096] = t[1];
        buf[4097] = t[2];
        buf[4098] = t[3];
/*
 * XXX error check ??
 */
        fread(buf, 1, 4096, f);
        fseek(f, -8192L, SEEK_CUR);
        t = &buf[4095];
...

  "f" in this context file handle of archive to be processed.
If size of that archive is less than 4096 that means first
fseek(f, -4096, SEEK_END) would try to seek to negative
file offset. I found out that in this case fseek does not
return an error and ftell(f) yields negative result!
Of course, this does not make any sense.
  If archive size is between 4096 and 8192 bytes it would mean
that second fseek(f, -8192, SEEK_CUR) also could try to do
negative seek.
  Later this code tries to fseek(f, 0, SEEK_SET) but it does not
help much - errno would be 27 (file too big) and everything blows up.
I think that fseek should return an error (-1) to negative offset.
I have checked what happens on other platforms. On FreeBSD/i386
negative fseek succeeds, but later fseek(f, 0, SEEK_SET) does not
give any errors - i.e. this negative fseek works without being caught.
On linux negative fseek returns an error.
  Proper fix for this situation would be apply fix mentioned
in kern/6184 to kernel. But I also wrote patch for infozip
to avoid even trying negative seeks, so it works without
kernel patching:

===========================================
--- zipfile.c.orig Sun Nov  7 02:30:11 1999
+++ zipfile.c Tue Feb 19 19:41:10 2002
@@ -628,6 +628,16 @@

 #endif /* !UTIL */

+local long fsize(FILE *f) {
+    long size, curpos;
+    if(!f) return 0;
+    curpos = ftell(f);
+    fseek(f, 0, SEEK_END);
+    size = ftell(f);
+    fseek(f, curpos, SEEK_SET);
+    return size;
+}
+
 /*
  * scanzipf_reg starts searching for the End Signature at the end of the
file
  * The End Signature points to the Central Directory Signature which points
@@ -667,7 +677,7 @@
     t[1] = '\0';
     t[2] = '\0';
     t[3] = '\0';
-    if (fseek(f, -4096L, SEEK_END) == 0) {
+    if (fsize(f)>=4096L && fseek(f, -4096L, SEEK_END) == 0) {
       zipbeg = (ulg) (ftell(f) + 4096L);
       while (!found && zipbeg >= 4096) {
         zipbeg -= 4096L;
@@ -678,7 +688,7 @@
  * XXX error check ??
  */
         fread(buf, 1, 4096, f);
-        fseek(f, -8192L, SEEK_CUR);
+        if(ftell(f)>=8192L) fseek(f, -8192L, SEEK_CUR);
         t = &buf[4095];
 /*
  * XXX far pointer arithmetic in DOS
===========================================

  It would be great if someone could commit this patch to ports tree
as /usr/ports/archivers/zip/files/patch-negativeseek or something.
I have checked that with this patch zip works fine for both i386 and alpha.
  In the meantime if someone could commit fix proposed in kern/6184
it won't hurt as well.

Vadim Mikhailov
Comment 7 Андрей Чернов 2002-02-20 13:32:34 UTC
On Tue, Feb 19, 2002 at 20:40:02 -0800, Vadim Mikhailov wrote:
>  
>  +local long fsize(FILE *f) {
>  +    long size, curpos;
>  +    if(!f) return 0;
>  +    curpos = ftell(f);
>  +    fseek(f, 0, SEEK_END);
>  +    size = ftell(f);
>  +    fseek(f, curpos, SEEK_SET);
>  +    return size;
>  +}

You already have file "f" opened at this point, so fsize() can be more
effectively implemented via fstat().

>    In the meantime if someone could commit fix proposed in kern/6184
>  it won't hurt as well.

FYI, negative lseek() and fseek() already disabled in FreeBSD-current

-- 
Andrey A. Chernov
http://ache.pp.ru/
Comment 8 Pete Fritchman freebsd_committer 2002-02-21 01:55:44 UTC
Responsible Changed
From-To: freebsd-ports->ache

Over to maintainer
Comment 9 mvp 2002-05-30 19:13:12 UTC
Hi, All!

  Negative seek kernel bug kern/6184 still has not been fixed
in -stable and probably it would not be fixed in 4.6-RELEASE.
That means the only way to have working zip (and mozilla,
which depend on working zip) on alpha in 4.6-R is to apply
my patch to zip port. Fact that zip works in -current
without patching is no excuse.
  To make zip work properly copy attached patch to
/usr/ports/archivers/zip/files/ and reinstall zip port.
I have checked it works fine on alpha and i386.
Andrey, I have streamlined fsize() function to use fstat()
according to your last comment. If you will commit this patch
probably you need to bump PORTREVISION.
  See this bug history here:
http://www.freebsd.org/cgi/query-pr.cgi?pr=33170

Take care,

Vadim Mikhailov.


begin 666 patch-negativeseek.dat
M+2TM('II<&9I;&4N8RYO<FEG"5-U;B!.;W8@(#<@,3@Z,S Z,3$@,3DY.0HK
M*RL@>FEP9FEL92YC"49R:2!-87D@,S$@,#(Z,3 Z,#@@,C P,@I 0" M-C(X
M+#8@*S8R."PQ,R! 0 H@"B C96YD:68@+RH@(55424P@*B\*( HK;&]C86P@
M;V9F7W0@9G-I>F4H1DE,12 J9BD@>PHK(" @('-T<G5C="!S=&%T(',["BL@
M(" @:68H(68I(')E='5R;B P.PHK(" @(&9S=&%T*&9I;&5N;RAF*2P@)G,I
M.PHK(" @(')E='5R;B!S+G-T7W-I>F4["BM]"BL*("\J"B @*B!S8V%N>FEP
M9E]R96<@<W1A<G1S('-E87)C:&EN9R!F;W(@=&AE($5N9"!3:6=N871U<F4@
M870@=&AE(&5N9"!O9B!T:&4@9FEL90H@("H@5&AE($5N9"!3:6=N871U<F4@
M<&]I;G1S('1O('1H92!#96YT<F%L($1I<F5C=&]R>2!3:6=N871U<F4@=VAI
M8V@@<&]I;G1S"D! ("TV-C<L-R K-C<T+#<@0$ *(" @("!T6S%=(#T@)UPP
M)SL*(" @("!T6S)=(#T@)UPP)SL*(" @("!T6S-=(#T@)UPP)SL*+2 @("!I
M9B H9G-E96LH9BP@+30P.39,+"!3145+7T5.1"D@/3T@,"D@>PHK(" @(&EF
M("AF<VEZ92AF*3X]-# Y-DP@)B8@9G-E96LH9BP@+30P.39,+"!3145+7T5.
M1"D@/3T@,"D@>PH@(" @(" @>FEP8F5G(#T@*'5L9RD@*&9T96QL*&8I("L@
M-# Y-DPI.PH@(" @(" @=VAI;&4@*"%F;W5N9" F)B!Z:7!B96<@/CT@-# Y
M-BD@>PH@(" @(" @("!Z:7!B96<@+3T@-# Y-DP["D! ("TV-S@L-R K-C@U
M+#<@0$ *(" J(%A86"!E<G)O<B!C:&5C:R _/PH@("HO"B @(" @(" @(&9R
M96%D*&)U9BP@,2P@-# Y-BP@9BD["BT@(" @(" @(&9S965K*&8L("TX,3DR
M3"P@4T5%2U]#55(I.PHK(" @(" @("!I9BAF=&5L;"AF*3X].#$Y,DPI(&9S
M965K*&8L("TX,3DR3"P@4T5%2U]#55(I.PH@(" @(" @("!T(#T@)F)U9ELT
M,#DU73L*("\J"B @*B!86%@@9F%R('!O:6YT97(@87)I=&AM971I8R!I;B!$
#3U,*
`
end
Comment 10 Андрей Чернов 2002-05-31 09:42:19 UTC
1) This patch is not needed for FreeBSD-current, so all additional
code must be properly ifdef'ed with __FreeBSD_version check.

2) Return code of fstat() is not checked, but must be.

-- 
Andrey A. Chernov
http://ache.pp.ru/
Comment 11 Andrey A. Chernov freebsd_committer 2002-05-31 11:16:10 UTC
State Changed
From-To: open->closed

Fixed