Bug 84048

Summary: sysutils/ntfsprogs: partition device access requires 512 byte size
Product: Ports & Packages Reporter: Andriy Gapon <avg>
Component: Individual Port(s)Assignee: freebsd-ports-bugs (Nobody) <ports-bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
512.patch
none
patch-Makefile none

Description Andriy Gapon 2005-07-25 11:40:07 UTC
I unsuccessfully tried to use ntfsresize program from this port. ntfsresize -i
was failing early in its run, during verification of "device size" i.e. 
read access to the whole partition that hosted ntfs. The error was EINVAL.
Apparently the problem is that freebsd partition devices (e.g. ad0s2) allow
read/write operations only for sizes that are multiple of sector size. E.g.:
$ dd if=/dev/ad0s1e of=/dev/null bs=1 count=1
dd: /dev/ad0s1e: Invalid argument
0+0 records in
0+0 records out
0 bytes transferred in 0.000099 secs (0 bytes/sec)
$ dd if=/dev/ad0s1e of=/dev/null bs=512 count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.008678 secs (59000 bytes/sec)

To make ntfsresize work on FreeBSD I had to change all read/write calls to use
n*512 byte counts.

The patches to ntfsclone.c and device.c are trivial - 1 byte was read to check
accessibility of a certain offset; 512 bytes are read now instead.
attrib.c patch works as follows: certain data structure is read from disk, its
size may be not multiple of 512, so we round up size before reading, but we
use actual size for bytes read.
ntfsresize.c: certain data structure needs to be written to disk, there is 
a certian algorithm for calculating its size, the result may be not 512*n,
so we round it up.

Probably there are other palces in the code that are affected by the same
restricition.

How-To-Repeat: 
try to use ntfsresize -i on ntfs partition
Comment 1 John Merryweather Cooper 2005-07-25 16:06:19 UTC
Andriy Gapon wrote:

>>Number:         84048
>>Category:       ports
>>Synopsis:       sysutils/ntfsprogs: partition device access requires 512 byte size
>>Confidential:   no
>>Severity:       serious
>>Priority:       medium
>>Responsible:    freebsd-ports-bugs
>>State:          open
>>Quarter:        
>>Keywords:       
>>Date-Required:
>>Class:          sw-bug
>>Submitter-Id:   current-users
>>Arrival-Date:   Mon Jul 25 10:40:07 GMT 2005
>>Closed-Date:
>>Last-Modified:
>>Originator:     Andriy Gapon
>>Release:        FreeBSD 5.4-RELEASE-p3 i386
>>Organization:
>>Environment:
>>    
>>
>System: FreeBSD 5.4-RELEASE-p3 i386
>ntfsprogs-1.9.4_1
>
>
>	
>  
>
>>Description:
>>    
>>
>I unsuccessfully tried to use ntfsresize program from this port. ntfsresize -i
>was failing early in its run, during verification of "device size" i.e. 
>read access to the whole partition that hosted ntfs. The error was EINVAL.
>Apparently the problem is that freebsd partition devices (e.g. ad0s2) allow
>read/write operations only for sizes that are multiple of sector size. E.g.:
>$ dd if=/dev/ad0s1e of=/dev/null bs=1 count=1
>dd: /dev/ad0s1e: Invalid argument
>0+0 records in
>0+0 records out
>0 bytes transferred in 0.000099 secs (0 bytes/sec)
>$ dd if=/dev/ad0s1e of=/dev/null bs=512 count=1
>1+0 records in
>1+0 records out
>512 bytes transferred in 0.008678 secs (59000 bytes/sec)
>
>To make ntfsresize work on FreeBSD I had to change all read/write calls to use
>n*512 byte counts.
>
>The patches to ntfsclone.c and device.c are trivial - 1 byte was read to check
>accessibility of a certain offset; 512 bytes are read now instead.
>attrib.c patch works as follows: certain data structure is read from disk, its
>size may be not multiple of 512, so we round up size before reading, but we
>use actual size for bytes read.
>ntfsresize.c: certain data structure needs to be written to disk, there is 
>a certian algorithm for calculating its size, the result may be not 512*n,
>so we round it up.
>
>Probably there are other palces in the code that are affected by the same
>restricition.
>
>  
>
>>How-To-Repeat:
>>    
>>
>
>try to use ntfsresize -i on ntfs partition
>
>  
>
>>Fix:
>>    
>>
>
>	
>
>--- 512.patch begins here ---
>--- libntfs/attrib.c	Sat Sep  4 13:16:32 2004
>+++ libntfs/attrib.c	Fri Jul 22 01:05:09 2005
>@@ -709,9 +709,9 @@
>  */
> s64 ntfs_attr_pread(ntfs_attr *na, const s64 pos, s64 count, void *b)
> {
>-	s64 br, to_read, ofs, total, total2;
>+	s64 br, old_to_read, to_read, ofs, total, total2;
> 	ntfs_volume *vol;
> 	runlist_element *rl;
> 
> 	Dprintf("%s(): Entering for inode 0x%llx, attr 0x%x, pos 0x%llx, "
> 			"count 0x%llx.\n", __FUNCTION__,
>@@ -741,6 +744,7 @@
> 	if (pos + count > na->data_size) {
> 		if (pos >= na->data_size)
> 			return 0;
>+		Dprintf("trunacting read pos=%lld, na->data_size=%lld, (old) count=%lld\n", pos, na->data_size, count);
> 		count = na->data_size - pos;
> 	}
> 	vol = na->ni->vol;
>@@ -820,6 +825,10 @@
> 		/* It is a real lcn, read it into @dst. */
> 		to_read = min(count, (rl->length << vol->cluster_size_bits) -
> 				ofs);
>+		old_to_read = to_read;
>+		to_read = (to_read + 511) / 512 * 512;
>+		if(old_to_read != to_read)
>+			Dprintf("adjusted %lld->%lld\n", old_to_read, to_read);
> retry:
> 		Dprintf("%s(): Reading 0x%llx bytes from vcn 0x%llx, lcn 0x%llx, "
> 				"ofs 0x%llx.\n", __FUNCTION__, to_read,
>@@ -828,6 +837,8 @@
> 				ofs, to_read, b);
> 		/* If everything ok, update progress counters and continue. */
> 		if (br > 0) {
>+			if(br > old_to_read)
>+				br = old_to_read;
> 			total += br;
> 			count -= br;
> 			b = (u8*)b + br;
>--- libntfs/device.c	Sat Sep  4 13:16:32 2004
>+++ libntfs/device.c	Mon Jul 11 23:27:55 2005
>@@ -441,10 +441,10 @@
>  */
> static inline int ntfs_device_offset_valid(struct ntfs_device *dev, s64 ofs)
> {
>-	char ch;
>+	char ch[512];
> 
> 	if (dev->d_ops->seek(dev, ofs, SEEK_SET) >= 0 &&
>-			dev->d_ops->read(dev, &ch, 1) == 1)
>+			dev->d_ops->read(dev, &ch, 512) > 0)
> 		return 0;
> 	return -1;
> }
>--- ntfsprogs/ntfsclone.c	Mon Jul 25 12:31:30 2005
>+++ ntfsprogs/ntfsclone.c	Mon Jul 25 12:33:32 2005
>@@ -1080,9 +1080,9 @@
> 
> static int device_offset_valid(int fd, s64 ofs)
> {
>-	char ch;
>+	char ch[512];
> 
>-	if (lseek(fd, ofs, SEEK_SET) >= 0 && read(fd, &ch, 1) == 1)
>+	if (lseek(fd, ofs, SEEK_SET) >= 0 && read(fd, &ch, 512) > 0)
> 		return 0;
> 	return -1;
> }
>--- ntfsprogs/ntfsresize.c	Sat Sep  4 13:16:32 2004
>+++ ntfsprogs/ntfsresize.c	Tue Jul 12 01:15:30 2005
>@@ -1783,6 +1783,10 @@
> 
> 	nr_clusters = resize->new_volume_size;
> 	bm_bsize = nr_clusters_to_bitmap_byte_size(nr_clusters);
>+	if(bm_bsize % 512 != 0) {
>+		Dprintf("adjusted bm_bsize: %lld->%lld\n", bm_bsize, (bm_bsize + 511) / 512 * 512);
>+		bm_bsize = (bm_bsize + 511) / 512 * 512;
>+	}
> 	nr_bm_clusters = rounded_up_division(bm_bsize, vol->cluster_size);
> 
> 	if (!(tmp = (u8 *)realloc(resize->lcn_bitmap.bm, bm_bsize)))
>--- 512.patch ends here ---
>
>
>  
>
>>Release-Note:
>>Audit-Trail:
>>Unformatted:
>>    
>>
>_______________________________________________
>freebsd-ports-bugs@freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-ports-bugs
>To unsubscribe, send any mail to "freebsd-ports-bugs-unsubscribe@freebsd.org"
>
>  
>
While we're at it, if the GNOMEVFS2 option is selected, the build fails
as follows:

"/usr/ports/Mk/bsd.gnome.mk", line 598: Malformed conditional
(${_USE_GNOME_ALL:Mgnomevfs2}=="")
"/usr/ports/Mk/bsd.port.mk", line 5234: if-less endif
make: fatal errors encountered -- cannot continue
*** Error code 1

Looks like some of the code in bsd.gnome.mk got a little side-ways in a
recent commit.

jmc
Comment 2 Joe Marcus Clarke 2005-07-26 05:31:00 UTC
On Mon, 2005-07-25 at 09:06 -0600, John Merryweather Cooper wrote:

> >  
> >
> While we're at it, if the GNOMEVFS2 option is selected, the build fails
> as follows:
> 
> "/usr/ports/Mk/bsd.gnome.mk", line 598: Malformed conditional
> (${_USE_GNOME_ALL:Mgnomevfs2}=="")
> "/usr/ports/Mk/bsd.port.mk", line 5234: if-less endif
> make: fatal errors encountered -- cannot continue
> *** Error code 1
> 
> Looks like some of the code in bsd.gnome.mk got a little side-ways in a
> recent commit.


The bug is in ntfsprogs.  They are missing:

WANT_GNOME=	yes

Somewhere before bsd.port.pre.mk.  Without this, none of the GNOME
macros will be defined.

Joe

> 
> jmc
> 
> _______________________________________________
> freebsd-gnome@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-gnome
> To unsubscribe, send any mail to "freebsd-gnome-unsubscribe@freebsd.org"
> 

-- 
PGP Key : http://www.marcuscom.com/pgp.asc
Comment 3 John Merryweather Cooper 2005-07-26 06:05:09 UTC
Joe Marcus Clarke wrote:

>On Mon, 2005-07-25 at 09:06 -0600, John Merryweather Cooper wrote:
>
>  
>
>>> 
>>>
>>>      
>>>
>>While we're at it, if the GNOMEVFS2 option is selected, the build fails
>>as follows:
>>
>>"/usr/ports/Mk/bsd.gnome.mk", line 598: Malformed conditional
>>(${_USE_GNOME_ALL:Mgnomevfs2}=="")
>>"/usr/ports/Mk/bsd.port.mk", line 5234: if-less endif
>>make: fatal errors encountered -- cannot continue
>>*** Error code 1
>>
>>Looks like some of the code in bsd.gnome.mk got a little side-ways in a
>>recent commit.
>>    
>>
>
>The bug is in ntfsprogs.  They are missing:
>
>WANT_GNOME=	yes
>
>Somewhere before bsd.port.pre.mk.  Without this, none of the GNOME
>macros will be defined.
>
>Joe
>
>  
>
>>jmc
>>
>>_______________________________________________
>>freebsd-gnome@freebsd.org mailing list
>>http://lists.freebsd.org/mailman/listinfo/freebsd-gnome
>>To unsubscribe, send any mail to "freebsd-gnome-unsubscribe@freebsd.org"
>>
>>    
>>
This patch to the Makefile fixes the build with WITH_GNOMEVFS2.

jmc
Comment 4 Jean-Yves Lefort freebsd_committer freebsd_triage 2005-07-26 20:22:53 UTC
On Mon, 25 Jul 2005 23:05:09 -0600
John Merryweather Cooper <john_m_cooper@yahoo.com> wrote:

> Joe Marcus Clarke wrote:
> 
> >On Mon, 2005-07-25 at 09:06 -0600, John Merryweather Cooper wrote:
> >
> >>>
> >>While we're at it, if the GNOMEVFS2 option is selected, the build fails
> >>as follows:
> >>
> >>"/usr/ports/Mk/bsd.gnome.mk", line 598: Malformed conditional
> >>(${_USE_GNOME_ALL:Mgnomevfs2}=="")
> >>"/usr/ports/Mk/bsd.port.mk", line 5234: if-less endif
> >>make: fatal errors encountered -- cannot continue
> >>*** Error code 1
> >>
> >>Looks like some of the code in bsd.gnome.mk got a little side-ways in a
> >>recent commit.
> >>    
> >>
> >
> >The bug is in ntfsprogs.  They are missing:
> >
> >WANT_GNOME=	yes
> >
> >Somewhere before bsd.port.pre.mk.  Without this, none of the GNOME
> >macros will be defined.
> >>
> This patch to the Makefile fixes the build with WITH_GNOMEVFS2.


Committed, thanks.

-- 
Jean-Yves Lefort

jylefort@FreeBSD.org
http://lefort.be.eu.org/
Comment 5 Pav Lucistnik freebsd_committer freebsd_triage 2005-07-27 12:30:32 UTC
State Changed
From-To: open->closed

Committed, thanks!