Bug 87544 - [gbde] mmaping large files on a gbde filesystem deadlocks the system
Summary: [gbde] mmaping large files on a gbde filesystem deadlocks the system
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-geom (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-10-16 21:40 UTC by joonas
Modified: 2017-06-28 23:49 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description joonas 2005-10-16 21:40:16 UTC
The python program does this: write an ~1GB file, mmap it into memory using MAP_SHARED and PROT_READ|PROT_WRITE, write a byte to every 80000'th byte in the file, unmap file; rinse & repeat five more times.  Sometime after writing out the first file it seems like the system deadlocks.  Namely, system cpu usage is up at 75%, 25% idle, with no i/o happening at all.  The system won't come out of deadlock, won't respond to ssh, but must be manually rebooted.  My sysadmin said he could type on the console, but the system didn't respond.

How-To-Repeat: 
Cut & paste the following program into tmp.py and run "python tmp.py" in a directory on an encrypted filesystem.   Open another window, go to the directory, and repeatedly do ls -lrt.  At some point the system will hang.  On a gbde file system io stops completely.  When run on a non-gbde file system the the disks churn like mad, and even though repeated ls -lrt hang also then, it also unhangs after a while and the program completes successfully.

---START---
import sys
import mmap
import os

# python is choking at mmap indices ~ 220M.
MAXCHUNKSIZE = 1024*1024*1024L
WANTCHUNKSIZE = MAXCHUNKSIZE

class charary:

    def __init__(self, basename, width, height, initval='\000'):
        self.width = width
        self.height = height
        self.basename = basename
        self.initval = initval

        totbytes = int(width)*1L*int(height)

        maxchunksize = WANTCHUNKSIZE
        self.rows_per_chunk = (maxchunksize + width - 1) / width
        while self.rows_per_chunk*width > MAXCHUNKSIZE:
                self.rows_per_chunk += -1
        self.chunksize = self.rows_per_chunk * width
        self.nchunks = (totbytes + self.chunksize-1) / self.chunksize

        assert self.rows_per_chunk > 0

        self.curchix = None;
        self.curmap = None

    def initfile(self, fd, size):
        i=0
        len = 16384
        block = self.initval * len
        os.lseek(fd, 0, 0)
        while i + len < size: 
            os.write(fd, block)
            i += len
        while i < size:
            os.write(fd, self.initval)
            i += 1
        os.lseek(fd, 0, 0)


    def mapchunk(self, chix):
        if chix <> self.curchix:
            if self.curmap <> None:
                self.curmap.close()
                self.curmap = None
            filename = "%s%03d" % (self.basename, chix)
            filesize = self.chunksize
            if chix == self.nchunks - 1:
                filesize = (self.height % self.rows_per_chunk)*self.width

            fd = os.open(filename, os.O_RDWR | os.O_CREAT)
            sbuf = os.fstat(fd)
            if sbuf.st_size < filesize:
                self.initfile(fd, filesize)
            self.curchix = chix
            self.curmap = mmap.mmap(fd, filesize, mmap.MAP_SHARED)
        return self.curmap

    def get(self,x,y):
        chix = y / self.rows_per_chunk
        c = self.mapchunk(chix)
        y = y % self.rows_per_chunk
        return c[x + self.width*y]

    def set(self,x,y,val):
        chix = y / self.rows_per_chunk
        c = self.mapchunk(chix)
        y = y % self.rows_per_chunk
        c[x + self.width*y] = val

    def close(self):
        if self.curchix <> None:
                self.curmap.close()
                self.curchix = None
                self.curmap = None

n=80000
a = charary("./ary", n, n)
for i in xrange(n):
        a.get(10, i)
        a.set(10,i,'A')
---END---

Best Regards,

Joonas
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2005-10-24 10:08:59 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-geom

Over to maintainer(s).