The python program does this: write an ~1GB file, mmap it into memory using MAP_SHARED and PROT_READ|PROT_WRITE, write a byte to every 80000'th byte in the file, unmap file; rinse & repeat five more times. Sometime after writing out the first file it seems like the system deadlocks. Namely, system cpu usage is up at 75%, 25% idle, with no i/o happening at all. The system won't come out of deadlock, won't respond to ssh, but must be manually rebooted. My sysadmin said he could type on the console, but the system didn't respond. How-To-Repeat: Cut & paste the following program into tmp.py and run "python tmp.py" in a directory on an encrypted filesystem. Open another window, go to the directory, and repeatedly do ls -lrt. At some point the system will hang. On a gbde file system io stops completely. When run on a non-gbde file system the the disks churn like mad, and even though repeated ls -lrt hang also then, it also unhangs after a while and the program completes successfully. ---START--- import sys import mmap import os # python is choking at mmap indices ~ 220M. MAXCHUNKSIZE = 1024*1024*1024L WANTCHUNKSIZE = MAXCHUNKSIZE class charary: def __init__(self, basename, width, height, initval='\000'): self.width = width self.height = height self.basename = basename self.initval = initval totbytes = int(width)*1L*int(height) maxchunksize = WANTCHUNKSIZE self.rows_per_chunk = (maxchunksize + width - 1) / width while self.rows_per_chunk*width > MAXCHUNKSIZE: self.rows_per_chunk += -1 self.chunksize = self.rows_per_chunk * width self.nchunks = (totbytes + self.chunksize-1) / self.chunksize assert self.rows_per_chunk > 0 self.curchix = None; self.curmap = None def initfile(self, fd, size): i=0 len = 16384 block = self.initval * len os.lseek(fd, 0, 0) while i + len < size: os.write(fd, block) i += len while i < size: os.write(fd, self.initval) i += 1 os.lseek(fd, 0, 0) def mapchunk(self, chix): if chix <> self.curchix: if self.curmap <> None: self.curmap.close() self.curmap = None filename = "%s%03d" % (self.basename, chix) filesize = self.chunksize if chix == self.nchunks - 1: filesize = (self.height % self.rows_per_chunk)*self.width fd = os.open(filename, os.O_RDWR | os.O_CREAT) sbuf = os.fstat(fd) if sbuf.st_size < filesize: self.initfile(fd, filesize) self.curchix = chix self.curmap = mmap.mmap(fd, filesize, mmap.MAP_SHARED) return self.curmap def get(self,x,y): chix = y / self.rows_per_chunk c = self.mapchunk(chix) y = y % self.rows_per_chunk return c[x + self.width*y] def set(self,x,y,val): chix = y / self.rows_per_chunk c = self.mapchunk(chix) y = y % self.rows_per_chunk c[x + self.width*y] = val def close(self): if self.curchix <> None: self.curmap.close() self.curchix = None self.curmap = None n=80000 a = charary("./ary", n, n) for i in xrange(n): a.get(10, i) a.set(10,i,'A') ---END--- Best Regards, Joonas
Responsible Changed From-To: freebsd-bugs->freebsd-geom Over to maintainer(s).