|Summary:||[build] release always fails with "mkimg: partition 2: No space left on device"|
|Component:||misc||Assignee:||freebsd-bugs mailing list <bugs>|
|Severity:||Affects Some People||CC:||guru|
Description leeb 2018-09-30 00:33:37 UTC
Created attachment 197609 [details] svn diff for release/amd64/make-memstick.sh 11.2-RELEASE-p3 r338761 make release always produces this error for me: Populating `mini-memstick.img.part' Image `mini-memstick.img.part' complete mkimg: partition 2: No space left on device *** Error code 74 Stop. make: stopped in /usr/src/release *** Error code 1 Stop. make: stopped in /usr/src/release The cause appears to be the nested use of mkimg, though why it can build memstick.img without problems I don't know. Tested both images on physical hardware, begun installer, went to shell, unmame looked good.
Comment 1 Matthias Apitz 2018-09-30 05:41:49 UTC
I have the same problem in a recent CURRENT; here are the details: # TMPDIR=/usr/tmp export TMPDIR # ./make-memstick.sh /usr/local/r338641/root.r338641 /usr/local/r338641/memstick.im + makefs -B little -M 50331648b -m 50331648b -o 'label=FreeBSD_Install' -o 'version=2' /usr/local/r338641/memstick.im.part /usr/local/r338641/root.r338641 Calculated size of `/usr/local/r338641/memstick.im.part': 25769803776 bytes, 24530 inodes Extent size set to 32768 /usr/local/r338641/memstick.im.part: 24576.0MB (50331648 sectors) block size 32768, fragment size 4096 using 28 cylinder groups of 901.44MB, 28846 blks, 1024 inodes. super-block backups (for fsck -b #) at: 192, 1846336, 3692480, 5538624, 7384768, 9230912, 11077056, 12923200, 14769344, 16615488, 18461632, 20307776, 22153920, 24000064, 25846208, 27692352, 29538496, 31384640, 33230784, 35076928, 36923072, 38769216, 40615360, 42461504, 44307648, 46153792, 47999936, 49846080, Populating `/usr/local/r338641/memstick.im.part' Image `/usr/local/r338641/memstick.im.part' complete + rm /usr/local/r338641/root.r338641/etc/fstab + rm /usr/local/r338641/root.r338641/etc/rc.conf.local + mkimg -C 28G -s mbr -b /usr/local/r338641/root.r338641/boot/mbr -p 'efi:=/usr/local/r338641/root.r338641/boot/boot1.efifat' -p 'freebsd:-mkimg -C 28G -s bsd -b /usr/local/r338641/root.r338641/boot/boot -p freebsd-ufs:=/usr/local/r338641/memstick.im.part' -a 2 -o /usr/local/r338641/memstick.im ... While the cascade of mkimg(1) is running a *big* temp file is created, which at the end eats up all memory of the disk: # ls -lh /usr/local/r338641 /usr/tmp /usr/local/r338641: total 25172008 -rwxr-xr-x 1 root wheel 1.3K Sep 29 16:41 make-memstick.sh -rw-r--r-- 1 root wheel 0B Sep 29 16:50 memstick.im -rw-r--r-- 1 root wheel 24G Sep 29 16:50 memstick.im.part drwxr-xr-x 18 root wheel 512B Sep 24 07:11 root.r338641 /usr/tmp: total 11307168 -rw------- 1 root wheel 456G Sep 29 17:01 mkimg-LmntlL <<<******** 456G !!! -rw------- 1 root wheel 0B Sep 29 16:50 mkimg-yfU8Lr /: write failed, filesystem is full /: write failed, filesystem is full But the 'memstick.im' is created fine at the end: # ls -lh /usr/local/r338641 /usr/tmp /usr/local/r338641: total 2591752 -rwxr-xr-x 1 root wheel 1.3K Sep 29 16:41 make-memstick.sh -rw-r--r-- 1 root wheel 24G Sep 29 17:22 memstick.im drwxr-xr-x 18 root wheel 512B Sep 24 07:11 root.r338641 /usr/tmp: total 0 And the UBS stick produced from 'memstick.im' with dd(1) boots fine and the root file system has around 20 GB free space. Why it is mkimg(1) creating such a big temp. file? I glanced through the sources of src/usr.bin/mkimg and it seems that the big file is some kind of swap space used by mkimg.
Comment 2 Matthias Apitz 2018-09-30 17:15:12 UTC
(In reply to leeb from comment #0) I think, in your case it will help to set an env var to some file system with more space than one normally has below /tmp, for example: # mkdir /usr/tmp # TMPDIR=/usr/tmp export TMPDIR AT least, this helped me for a tiny root file system to make an image off. But if you specify in the command mkfs(8) that your file system should have plenty space for additional thinks, it will create there in TMPDIR some swap file with giga o even terra bytes. There is something broken with mkimg(1). Some time ago, in March 2017 on r314251, this was all working fine and w/o any TMPDIR.
Comment 3 leeb 2018-10-01 01:21:33 UTC
My /tmp is 12G tmpfs. Using the patch I supplied, it doesn't really go over 100M, just checking with frequent df -h in another console during the mkimg process. This happens with a GENERIC kernel on a stock installation. Nothing added. cd /usr/src make -j16 buildworld make -j16 buildkernel cd release make release I don't know why your mkfs is 24G, mine is more like 350M. sh /usr/src/release/amd64/make-memstick.sh bootonly mini-memstick.img Calculated size of `mini-memstick.img.part': 361332736 bytes, 11416 inodes Extent size set to 32768 mini-memstick.img.part: 344.6MB (705728 sectors) block size 32768, fragment size 4096 using 1 cylinder groups of 344.59MB, 11027 blks, 12672 inodes. super-block backups (for fsck -b #) at: 192, Populating `mini-memstick.img.part' Image `mini-memstick.img.part' complete
Comment 4 Matthias Apitz 2018-10-01 06:21:05 UTC
(In reply to leeb from comment #3) My file system produced by mkfs(8) is around 24 GByte by intention because I added '-M 50331648b -m 50331648b' in the script to the parms of mkfs(8). The idea is to produce a file system in which I can install packages (a complete KDE4) for testing before I do move to the new CURRENT and my own packages produced with poudriere. I applied your patch to my script make-memstick.sh and it now produced fine the memestick.img without any memory problems. It seems that the problems is caused by chaining the mkimg(1) calls, which your patch exactly addresses. I will copy over the memstick.img to an USB key and test further.
Comment 5 leeb 2018-10-01 20:43:28 UTC
I pulled the disc1.iso from freebsd.org last night, installed it on a Xen VM (16GB RAM, 32GB Disk, no swap) and ran the make buldworld/buildkernel/release which went fine. I binary updated it to -p4, same result was OK. My build system (16GB RAM, 32GB Disk, no swap, tmpfs) was built from source and is currently -p3, no ports installed, no src.conf, no make.conf. I'll src update it and see if -p4 works correctly or still exhibits the problem, which will narrow it to -p3 (unlikely) or my environment. The Xen hosts are different.
Comment 6 leeb 2018-10-02 03:15:37 UTC
tmpfs was the cause of my problem. After removing it the release worked correctly. It was mounted on /tmp as rw,mode=01777,size=12g Consider this closed from my perspective, but I'll leave it for Matthias.
Comment 7 Matthias Apitz 2018-10-02 05:03:13 UTC
I removed tmpfs from rc.conf, but the memory issue is still there. When this proc chain is running: mkimg -s mbr -b /usr/local/r338641/root.r338641/boot/mbr -p 'efi:=/usr/local/r338641/root.r338641/boot/boot1.efifat' -p 'freebsd:-mkimg -s bsd -b /usr/local/r338641/root.r338641/boot/boot -p freebsd-ufs:=/home/guru/zdata/r338641/memstick.im.part' -a 2 -o /home/guru/zdata/r338641/memstick.im the temp file is growing and growing: # ls -lh /usr/tmp total 7683968 -rw------- 1 root wheel 0B Oct 2 06:52 mkimg-8lenP0 -rw------- 1 root wheel 390G Oct 2 07:00 mkimg-EfzEvr ... # ls -lh /usr/tmp total 9515808 -rw------- 1 root wheel 0B Oct 2 06:52 mkimg-8lenP0 -rw------- 1 root wheel 408G Oct 2 07:01 mkimg-EfzEvr Please re-open the PR again for CURRENT.
Comment 9 Matthias Apitz 2018-10-02 12:25:39 UTC
(In reply to leeb from comment #6) Lee, could you please do a test for double check: modify the script /usr/src/release/amd64/make-memstick.sh and insert size values in the line for mkfs like this; change the line: makefs -B little ... to: makefs -B little -M 50331648b -m 50331648b ... Be prepared for the target memstick.img needs ~25 GByte
Comment 10 leeb 2018-10-02 20:33:27 UTC
That created a 219G file on /tmp which is on a 10G partition
Comment 12 Matthias Apitz 2018-10-03 08:26:32 UTC
(In reply to leeb from comment #10) How is this possible, a 219G file in a 10G partition? What type of file is this?
Comment 13 leeb 2018-10-03 15:32:15 UTC
Sorry, I had increased the size of the partition that /usr/src and /usr/obj were on, but forgot /tmp was on my root partition. They are all UFS. The src and obj are nullfs mounts. I suppose a sparse file would act differently if it were piped vs. stored? That's the only difference that patch makes. The only reason I did that was to better isolate my debug output, but it didn't help in that respect.