Summary: | mremap() implementation lacking | ||
---|---|---|---|
Product: | Base System | Reporter: | Zachary Amsden <zach> |
Component: | kern | Assignee: | freebsd-bugs (Nobody) <bugs> |
Status: | Open --- | ||
Severity: | Affects Only Me | CC: | doctorwhoguy, shawn.webb, thmu7, trasz |
Priority: | Normal | ||
Version: | 4.2-RELEASE | ||
Hardware: | Any | ||
OS: | Any | ||
Bug Depends on: | |||
Bug Blocks: | 247219 |
Description
Zachary Amsden
2003-12-03 00:00:17 UTC
Responsible Changed From-To: freebsd-bugs->bms I'll look at this. Responsible Changed From-To: bms->freebsd-bugs Back to the free pool Responsible Changed From-To: freebsd-bugs->alc alc's area alc - please reassign to me if you don't think you can get to it i'm forwarding this conversation i had with alc@, because it reveals some details about the issues described in this PR which haven't been mentioned beforehand. this should help any developer deciding to work on this PR. cheers. alex ----- Forwarded message from Alan Cox <alc@cs.rice.edu> ----- Date: Mon, 26 Jul 2010 17:34:47 -0500 From: Alan Cox <alc@cs.rice.edu> To: Alexander Best <arundel@freebsd.org> I wasn't aware of the Google linker using mremap(). I would suggest that you add most of this e-mail to the PR for posterity. [snip] Alexander Best wrote: >On Sun Jul 25 10, Alan Cox wrote: > >>Alexander Best wrote: >> >>>On Wed, Jul 21, 2010 at 04:20:41PM -0500, Alan Cox wrote: [snip] >>>i also stumpled over kern/59912 which provides >>>patches for a mremap() syscall. >>> >>>in 2000 such a syscall was discussed and the result was that freebsd >>>doesn't need mremap(), because doing munamp() and mmap() again is >>>sufficient enough. [1] >>> >>>matthew dillon wrote: >>> >>> "There are a thousand ways to do it, which is why linux's mremap() >>> syscall is stupid. >>> >>> * simply mmap() a larger block in the first place. For example, >>> if you have a 16K file mmap() 1MB. You will seg fault on pages >>> that are beyond the file EOF, but those pages will become valid >>> the moment the file is extended into them without having to lift >>> a finger. >>> >>> * mmap() the tail end of the newly extended file without removing or >>> overwriting the previous mmap, by specifying an absolute address. >>> >>> * munmap() and re-mmap() the file. >>> >>> * Don't depend on a single monolithic mmap(), it won't work for files >>> larger then 2-3GB anyway (on intel architecture), instead mmap the >>> file in chunks on an as-needed basis." >>> >>>my question is: should this pr remain open or is there a consens that >>>freebsd doesn't need mremap()? it looks like linux mremap() can be >>>perfectly simulated in the linuxulator with munmap()/mmap(). so the point >>>that freebsd having mremap() would improve linux compatibility isn't >>>really valid. >>> >>> >>> >>First, do a web search for the e-mail thread on the FreeBSD lists with >>the subject "perl malloc slow?". >> > >thanks a lot. i wasn't aware of the perl performance issue back then and >that >the root cause for it may have been an excessive use of munmap()/mmap() >whereas >linux was getting better performance due to mremap(). > > [snip] >>What if there is no file, that is, what if you want to relocate anonymous >>memory? As I understand mremap(), it can relocate anonymous memory for >>which there is no backing file. >> > >indeed matt didn't comment on the situation you just described. it seems >this >would be a scenario where mremap() could really pose a huge benefit to any >OS >which has implemented it. > > >>It's not clear to me that mremap() is entirely without merit. However, >>it's never really been a priority for anyone with the necessary skill >>set. I haven't looked at this patch in ages, but I seem to recall that >>there were corner cases that it didn't handle. >> > >the only bsd which has implemented mremap() is netbsd [1]. however the >semantics are quite different to the ones of the linux implementation of >mremap(). also the manual mentions that netbsd's mremap() is based upon >mremap() from their linuxulator code. > >i'm not sure if that means that they implemented it properly or simply made >their mremap() to wrap calls into munmap() and mmap(). > > >>I'm content to leave it idle. I don't see a compelling reason to close it. >> > >yeah maybe somebody wants to port an existing implementation of mremap() to >freebsd. the one that's in linux seems to exist since 1995 and thus should >be >quite solid. however there don't seem to exist a lot of benchmark results >for >this scenario. i found [2], but that's still based on the old phkmalloc. > >it would be nice e.g. to see the linux and netbsd implementations of >mremap() >compete against each other. maybe some other OSes also implement it and >could >be included in the bechmarks. > >this is the one from MacOS X 10.5.7 btw: > >int >mremap(void) >{ > /* Not yet implemented */ > return (ENOTSUP); >} > >*hehehe* > >i also saw that the GNU binutils include a very rough implementation of >mremap() [3]. this only occurs in versions of binutils which feature the >GOLD >linker created by google. since the GOLD linker relies upon mremap(), the >binutils have to make sure that the new linker can be used even on systems >that don't have mremap(). > >thanks for clearing things up. > >cheers. >alex > >[1] >http://www.freebsd.org/cgi/man.cgi?query=mremap&apropos=0&sektion=0&manpath=NetBSD+5.0&format=html >[2] http://www.dent.med.uni-muenchen.de/~wmglo/malloc-slides.html >[3] >http://grok.x12.su/source/xref/dragonfly/contrib/binutils-2.20/gold/mremap.c > > >>Regards, >>Alan >> >> ----- End forwarded message ----- State Changed From-To: open->suspended Alan considers mremap() to be a reasonable addition to FreeBSD. However neither he nor any other developer are working on this issue atm. Thus set PR into suspended state. Developers willing to work on this PR should contact Alan. Responsible Changed From-To: alc->freebsd-bugs Alan isn't working on this PR anymore -- assign it back into the pool. For bugs matching the following conditions: - Status == In Progress - Assignee == "bugs@FreeBSD.org" - Last Modified Year <= 2017 Do - Set Status to "Open" FWIW, this is what breaks apt(8); it fails with: E: Dynamic MMap ran out of room. Please increase the size of APT::Cache-Start. The workaround is to: echo "APT::Cache-Start 251658240;" >> "/compat/ubuntu/etc/apt/apt.conf.d/00freebsd" I'm considering adding this workaround to sysutils/deboostrap; apt(8) seems to be only thing I've seen that's affected by the mremap problem. I'm observing linux_mremap() calls issued by RPM 4.16 failing with -12 (ENOMEM). (In reply to Thomas Mueller from comment #9) FreeBSD's linuxulator's mremap doesn't support extending mappings. It only supports shrinking. The Cross-DSO CFI implementation in llvm favors mremap, though it does contain a fallback in the case mremap doesn't exist. If FreeBSD were to provide an official mremap syscall implementation, both the linuxulator and llvm's Cross-DSO CFI implementation would benefit. |