Bug 158950 - arm/sheevaplug fails fsx when mmap operations are enabled
Summary: arm/sheevaplug fails fsx when mmap operations are enabled
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: arm (show other bugs)
Version: 9.0-CURRENT
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-arm (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-15 18:50 UTC by Kirk Russell
Modified: 2018-01-03 05:16 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kirk Russell 2011-07-15 18:50:08 UTC
I can get fsx to run okay if I disable the mmap operations, using the -RW command line options.
When I enable the mmap operations, fsx appears to fail.

See "How to repeat" for the fsx output.

This is the output of the boot rom and the kernel booting.
         __  __                      _ _
        |  \/  | __ _ _ ____   _____| | |
        | |\/| |/ _` | '__\ \ / / _ \ | |
        | |  | | (_| | |   \ V /  __/ | |
        |_|  |_|\__,_|_|    \_/ \___|_|_|
 _   _     ____              _
| | | |   | __ )  ___   ___ | |_ 
| | | |___|  _ \ / _ \ / _ \| __| 
| |_| |___| |_) | (_) | (_) | |_ 
 \___/    |____/ \___/ \___/ \__| 
 ** MARVELL BOARD: SHEEVA PLUG LE 

U-Boot 1.1.4 (Mar 19 2009 - 16:06:59) Marvell version: 3.4.16

U-Boot code: 00600000 -> 0067FFF0  BSS: -> 006CEE80

Soc: 88F6281 A0 (DDR2)
CPU running @ 1200Mhz L2 running @ 400Mhz
SysClock = 400Mhz , TClock = 200Mhz 

DRAM CAS Latency = 5 tRP = 5 tRAS = 18 tRCD=6
DRAM CS[0] base rx00000000   size 256MB 
DRAM CS[1] base rx10000000   size 256MB 
DRAM Total size 512MB  16bit width
Flash:  0 kB
Addresses 8M - 0M are saved for the U-Boot usage.
Mem malloc Initialization (8M - 7M): Done
NAND:512 MB

CPU : Marvell Feroceon (Rev 1)

Streaming disabled 
Write allocate disabled


USB 0: host mode
PEX 0: interface detected no Link.
Net:   egiga0 [PRIME], egiga1
Hit any key to stop autoboot:  0 
Using egiga0 device
TFTP from server 192.168.1.23; our IP address is 192.168.1.79
Filename 'node79.bin'.
Load address: 0x900000
Loading: #################################################################
[...]

done
Bytes transferred = 2997752 (2dbdf8 hex)
## Starting application at 0x00900000 ...
 dtbp = 0xc0bba550
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #0: Fri Jul 15 17:08:59 IST 2011
    toor@freenas:/usr/obj/arm.arm/usr/src/sys/NODE79 arm
CPU: Feroceon 88FR131 rev 1 (Marvell core)
  DC enabled IC enabled WB enabled EABT branch prediction enabled
  16KB/32B 4-way Instruction cache
  16KB/32B 4-way write-back-locking-C Data cache
real memory  = 536870912 (512 MB)
avail memory = 520232960 (496 MB)
SOC: Marvell 88F6281 rev A0, TClock 200MHz
simplebus0: <Flattened device tree simple bus> on fdtbus0
ic0: <Marvell Integrated Interrupt Controller> mem 0xf1020200-0xf102023b on simplebus0
timer0: <Marvell CPU Timer> mem 0xf1020300-0xf102032f irq 1 on simplebus0
Event timer "CPUTimer0" frequency 200000000 Hz quality 1000
Timecounter "CPUTimer1" frequency 200000000 Hz quality 1000
gpio0: <Marvell Integrated GPIO Controller> mem 0xf1010100-0xf101011f irq 35,36,37,38,39,40,41 on simplebus0
rtc0: <Marvell Integrated RTC> mem 0xf1010300-0xf1010307 on simplebus0
mge0: <Marvell Gigabit Ethernet controller> mem 0xf1072000-0xf1073fff irq 12,13,14,11,46 on simplebus0
mge0: Ethernet address: 00:50:43:01:d6:c8
miibus0: <MII bus> on mge0
e1000phy0: <Marvell 88E1116R Gigabit PHY> PHY 0 on miibus0
e1000phy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
uart0: <16550 or compatible> mem 0xf1012000-0xf101201f irq 33 on simplebus0
uart0: console (1066,n,8,1)
uart1: <16550 or compatible> mem 0xf1012100-0xf101211f irq 34 on simplebus0
ehci0: <Marvell Integrated USB 2.0 controller> mem 0xf1050000-0xf1050fff irq 48,19 on simplebus0
usbus0: EHCI version 1.0
usbus0: set host controller mode
usbus0: <Marvell Integrated USB 2.0 controller> on ehci0
Timecounters tick every 1.000 msec
usbus0: 480Mbps High Speed USB v2.0
bootpc_init: wired to interface 'mge0'
Sending DHCP Discover packet from interface mge0 (00:50:43:01:d6:c8)
ugen0.1: <Marvell> at usbus0
uhub0: <Marvell EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
mge0: link state changed to UP
uhub0: 1 port with 1 removable, self powered
Received DHCP Offer packet on mge0 from 192.168.1.23 (accepted) (no root path)
ugen0.2: <Kingston> at usbus0
umass0: <Kingston DataTraveler 102, class 0/0, rev 2.00/1.00, addr 2> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x0000
umass0:0:0:-1: Attached to scbus0

da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Kingston DataTraveler 102 PMAP> Removable Direct Access SCSI-0 device 
da0: 40.000MB/s transfers
da0: 3745MB (7669824 512 byte sectors: 255H 63S/T 477C)
Sending DHCP Request packet from interface mge0 (00:50:43:01:d6:c8)
Received DHCP Ack packet on mge0 from 192.168.1.23 (accepted) (got root path)
mge0 at 192.168.1.79 server 192.168.1.23 boot file node79.bin
subnet mask 255.255.255.0 router 192.168.1.101 root_server 192.168.1.23 rootfs /export/tank/freebsd/arm8le hostname node79 
Adjusted interface mge0
Trying to mount root from nfs: []...
NFS ROOT: 192.168.1.23:/export/tank/freebsd/arm8le
Interface mge0 IP-Address 192.168.1.79 Broadcast 192.168.1.255 
Setting hostuuid: 4e548f07-9f3f-11e0-8815-00504301d6c8.
Setting hostid: 0xb812a530.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
mount_nfs: can't update /var/db/mounttab for 192.168.1.23:/export/tank/freebsd/arm8le
Mounting local file systems:.
Starting Network: lo0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet 127.0.0.1 netmask 0xff000000 
Starting devd.
devd: cannot open pid file: Operation not supported
Mounting NFS file systems:.
Creating and/or trimming log files.
Starting syslogd.
syslogd: cannot open pid file: Operation not supported
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
Starting rpcbind.
Jul 15 17:20:09 node79 rpcbind: cannot get information for udp6
Jul 15 17:20:09 node79 rpcbind: cannot get information for tcp6
NFS access cache time=60
Clearing /tmp.
Starting lockd.
Jul 15 17:20:11 node79 kernel: NLM: failed to contact remote rpcbind, stat = 0, port = 0
Jul 15 17:20:11 node79 kernel: Can't start NLM - unable to contact NSM
Starting local daemons:.
Updating motd:.
Starting sshd.

Fri Jul 15 17:20:13 UTC 2011

FreeBSD/arm (node79) (ttyu0)

login:

How-To-Repeat: This is a USB key.

node79# newfs /dev/da0s2a
Reduced frags per cylinder group from 94088 to 94048 to enlarge last cyl group
/dev/da0s2a: 2945.0MB (6031360 sectors) block size 16384, fragment size 2048
        using 17 cylinder groups of 183.69MB, 11756 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
 160, 376352, 752544, 1128736, 1504928, 1881120, 2257312, 2633504, 3009696,
 3385888, 3762080, 4138272, 4514464, 4890656, 5266848, 5643040, 6019232
node79# /sbin/mount -o noclusterr,noclusterw /dev/da0s2a /mnt
node79# cd /mnt
node79# fsx sheevaplug
truncating to largest ever: 0x13e76
READ BAD DATA: offset = 0xf369, size = 0xbd47
OFFSET  GOOD    BAD     RANGE
0x f400 0x0593  0x0000  0x   9f
operation# (mod 256) for the bad data unknown, check HOLE and EXTEND ops
LOG DUMP (9 total operations):
1(1 mod 256): TRUNCATE UP       from 0x0 to 0x13e76     ******WWWW
2(2 mod 256): WRITE     0x17098 thru 0x26857    (0xf7c0 bytes) HOLE
3(3 mod 256): READ      0xc73e thru 0x1b801     (0xf0c4 bytes)  ***RRRR***
4(4 mod 256): MAPWRITE 0x32e00 thru 0x331fc     (0x3fd bytes)
5(5 mod 256): MAPWRITE 0x7ac1 thru 0x11029      (0x9569 bytes)  ******WWWW
6(6 mod 256): READ      0x1f62e thru 0x2177f    (0x2152 bytes)
7(7 mod 256): WRITE     0x756 thru 0xede        (0x789 bytes)
8(8 mod 256): READ      0x18f13 thru 0x27d18    (0xee06 bytes)
9(9 mod 256): READ      0xf369 thru 0x1b0af     (0xbd47 bytes)  ***RRRR***
Correct content saved for comparison
(maybe hexdump "sheevaplug" vs "sheevaplug.fsxgood")
node79# cmp sheevaplug*
sheevaplug sheevaplug.fsxgood differ: char 62465, line 63
Comment 1 Kristof Provost 2012-04-23 10:58:51 UTC
The problem still occurs in current (r234281).

Interestingly it only seems to occur when using a USB stick. It doesn't
happen when using NFS, or a loop-mounted file system (over NFS).

The following code frequently (but not always!) triggers the problem as
well:

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>

#define FILE_NAME "test.img"

/* Also occurs with offset = 0, but not as frequently */
#define OFFSET 4096

int main(int argc, char **argv) {
	int fd, i;
	char buff[1024];
	char *map;

	(void)argc;
	(void)argv;

	/* Make sure the file doesn't exist when we start */
	unlink(FILE_NAME);

	fd = open(FILE_NAME, O_CREAT | O_RDWR);
	if (fd < 0) {
		perror("Failed to open " FILE_NAME);
		return 1;
	}

	/* mmap write (beyond what is currently written) */
	for (i = 0; i < 1024; i++) {
		buff[i] = i % 128;
	}

	/* Truncate the file up */
	ftruncate(fd, OFFSET + 1024);

	map = mmap(NULL, 1024, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, OFFSET);
	if (map == (char*)-1) {
		perror("Failed to mmap ");
		return 1;
	}

	memcpy(map, buff, 1024);

	msync(map, 1024, MS_SYNC);
	munmap(map, 1024);

	/* Now read() and check if all bytes are written correctly */
	lseek(fd, OFFSET, SEEK_SET);
	read(fd, buff, 1024);

	for (i = 0; i < 1024; i++) {
		if (buff[i] != (i % 128))
			printf("After mmap: offset %d is %d, not %d as expected\n",
			    i, buff[i], i % 128);
	}

	close(fd);

	return 0;
}
Comment 2 marktinguely 2012-04-24 18:17:08 UTC
This may be related to problem:
   arm/162159: [panic] USB errors leading to panic on DockStar 9.0-RC1/arm

What is the output to the mmap program?

--Mark Tinguely.
Comment 3 Kristof Provost 2012-04-25 10:15:17 UTC
The sample program above has the following output:

root@openrd:/mnt# /bin/kp
After mmap: offset 32 is 0, not 32 as expected
After mmap: offset 33 is 0, not 33 as expected
After mmap: offset 34 is 0, not 34 as expected
...
After mmap: offset 61 is 0, not 61 as expected
After mmap: offset 62 is 0, not 62 as expected
After mmap: offset 63 is 0, not 63 as expected
After mmap: offset 224 is 0, not 96 as expected
After mmap: offset 225 is 0, not 97 as expected
...

That's confirmed by hexdump-ing the test.img file afterwards.

I'm also able to reproduce the problem described in arm/162159. You're
probably right about the relation.

The problem looks very similar too: 32-byte sections that are 0 rather
than the expected values.
Comment 4 Kristof Provost 2012-05-07 13:15:51 UTC
Based on a suggestion from Ian Lepore I switched the data cache from 
write-back to write-through mode. In that case the problem doesn't
occur.

Ian suggested this might indicate a problem in the busdma cacheline
flush code, but unfortunately I'm somewhat out of my depth there.
Comment 5 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:19 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped