Bug 231810

I have the same problem in a recent CURRENT; here are the details:


# TMPDIR=/usr/tmp export TMPDIR
# ./make-memstick.sh /usr/local/r338641/root.r338641 /usr/local/r338641/memstick.im
+ makefs -B little -M 50331648b -m 50331648b -o 'label=FreeBSD_Install' -o 'version=2' /usr/local/r338641/memstick.im.part /usr/local/r338641/root.r338641
Calculated size of `/usr/local/r338641/memstick.im.part': 25769803776 bytes, 24530 inodes
Extent size set to 32768
/usr/local/r338641/memstick.im.part: 24576.0MB (50331648 sectors) block size 32768, fragment size 4096
        using 28 cylinder groups of 901.44MB, 28846 blks, 1024 inodes.
super-block backups (for fsck -b #) at:
      192,  1846336,  3692480,  5538624,  7384768,  9230912, 11077056,
 12923200, 14769344, 16615488, 18461632, 20307776, 22153920, 24000064,
 25846208, 27692352, 29538496, 31384640, 33230784, 35076928, 36923072,
 38769216, 40615360, 42461504, 44307648, 46153792, 47999936, 49846080,
Populating `/usr/local/r338641/memstick.im.part'
Image `/usr/local/r338641/memstick.im.part' complete
+ rm /usr/local/r338641/root.r338641/etc/fstab
+ rm /usr/local/r338641/root.r338641/etc/rc.conf.local
+ mkimg -C 28G -s mbr -b /usr/local/r338641/root.r338641/boot/mbr -p 'efi:=/usr/local/r338641/root.r338641/boot/boot1.efifat' -p 'freebsd:-mkimg -C 28G -s bsd -b /usr/local/r338641/root.r338641/boot/boot -p freebsd-ufs:=/usr/local/r338641/memstick.im.part' -a 2 -o /usr/local/r338641/memstick.im
...


While the cascade of mkimg(1) is running a *big* temp file is created,
which at the end eats up all memory of the disk:

# ls -lh /usr/local/r338641 /usr/tmp
/usr/local/r338641:
total 25172008
-rwxr-xr-x   1 root  wheel   1.3K Sep 29 16:41 make-memstick.sh
-rw-r--r--   1 root  wheel     0B Sep 29 16:50 memstick.im
-rw-r--r--   1 root  wheel    24G Sep 29 16:50 memstick.im.part
drwxr-xr-x  18 root  wheel   512B Sep 24 07:11 root.r338641

/usr/tmp:
total 11307168
-rw-------  1 root  wheel   456G Sep 29 17:01 mkimg-LmntlL    <<<******** 456G !!!
-rw-------  1 root  wheel     0B Sep 29 16:50 mkimg-yfU8Lr

/: write failed, filesystem is full

/: write failed, filesystem is full

But the 'memstick.im' is created fine at the end:

# ls -lh /usr/local/r338641 /usr/tmp
/usr/local/r338641:
total 2591752
-rwxr-xr-x   1 root  wheel   1.3K Sep 29 16:41 make-memstick.sh
-rw-r--r--   1 root  wheel    24G Sep 29 17:22 memstick.im
drwxr-xr-x  18 root  wheel   512B Sep 24 07:11 root.r338641

/usr/tmp:
total 0

And the UBS stick produced from 'memstick.im' with dd(1) boots fine and the root
file system has around 20 GB free space.

Why it is mkimg(1) creating such a big temp. file?

I glanced through the sources of src/usr.bin/mkimg and it seems that the big file
is some kind of swap space used by mkimg.

(In reply to leeb from comment #0)

I think, in your case it will help to set an env var to some file system with more space than one normally has below /tmp, for example:

# mkdir /usr/tmp
# TMPDIR=/usr/tmp export TMPDIR

AT least, this helped me for a tiny root file system to make an image off. 

But if you specify in the command mkfs(8) that your file system should have plenty space for additional thinks, it will create there in TMPDIR some swap file with giga o even terra bytes.

There is something broken with mkimg(1).

Some time ago, in March 2017 on r314251, this was all working fine and w/o any TMPDIR.

My /tmp is 12G tmpfs.  Using the patch I supplied, it doesn't really go over 100M, just checking with frequent df -h in another console during the mkimg process.

This happens with a GENERIC kernel on a stock installation.  Nothing added.
cd /usr/src
make -j16 buildworld
make -j16 buildkernel
cd release
make release

I don't know why your mkfs is 24G, mine is more like 350M.

sh /usr/src/release/amd64/make-memstick.sh bootonly mini-memstick.img
Calculated size of `mini-memstick.img.part': 361332736 bytes, 11416 inodes
Extent size set to 32768
mini-memstick.img.part: 344.6MB (705728 sectors) block size 32768, fragment size 4096
        using 1 cylinder groups of 344.59MB, 11027 blks, 12672 inodes.
super-block backups (for fsck -b #) at:
 192,
Populating `mini-memstick.img.part'
Image `mini-memstick.img.part' complete

(In reply to leeb from comment #3)

My file system produced by mkfs(8) is around 24 GByte by intention because I added '-M 50331648b -m 50331648b' in the script to the parms of mkfs(8). The idea is to produce a file system in which I can install packages (a complete KDE4) for testing before I do move to the new CURRENT and my own packages produced with poudriere.

I applied your patch to my script make-memstick.sh and it now produced fine the memestick.img without any memory problems. It seems that the problems is caused by chaining the mkimg(1) calls, which your patch exactly addresses.

I will copy over the memstick.img to an USB key and test further.

I pulled the disc1.iso from freebsd.org last night, installed it on a Xen VM (16GB RAM, 32GB Disk, no swap) and ran the make buldworld/buildkernel/release which went fine.  I binary updated it to -p4, same result was OK.

My build system (16GB RAM, 32GB Disk, no swap, tmpfs) was built from source and is currently -p3, no ports installed, no src.conf, no make.conf.  I'll src update it and see if -p4 works correctly or still exhibits the problem, which will narrow it to -p3 (unlikely) or my environment.

The Xen hosts are different.

tmpfs was the cause of my problem.  After removing it the release worked correctly.

It was mounted on /tmp as rw,mode=01777,size=12g

Consider this closed from my perspective, but I'll leave it for Matthias.

I removed tmpfs from rc.conf, but the memory issue is still there.
When this proc chain is running:

mkimg -s mbr -b /usr/local/r338641/root.r338641/boot/mbr -p 'efi:=/usr/local/r338641/root.r338641/boot/boot1.efifat' -p 'freebsd:-mkimg -s bsd -b /usr/local/r338641/root.r338641/boot/boot -p freebsd-ufs:=/home/guru/zdata/r338641/memstick.im.part' -a 2 -o /home/guru/zdata/r338641/memstick.im

the temp file is growing and growing:

# ls -lh /usr/tmp
total 7683968
-rw-------  1 root  wheel     0B Oct  2 06:52 mkimg-8lenP0
-rw-------  1 root  wheel   390G Oct  2 07:00 mkimg-EfzEvr

...

# ls -lh /usr/tmp
total 9515808
-rw-------  1 root  wheel     0B Oct  2 06:52 mkimg-8lenP0
-rw-------  1 root  wheel   408G Oct  2 07:01 mkimg-EfzEvr

Please re-open the PR again for CURRENT.

Per comment #7

(In reply to leeb from comment #6)

Lee, could you please do a test for double check: modify the script

/usr/src/release/amd64/make-memstick.sh

and insert size values in the line for mkfs like this; change the line:

makefs -B little ...

to:

makefs -B little -M 50331648b -m 50331648b ...

Be prepared for the target memstick.img needs ~25 GByte

That created a 219G file on /tmp which is on a 10G partition

Created attachment 197721 [details]
Screenshot

(In reply to leeb from comment #10)

How is this possible, a 219G file in a 10G partition? What type of file is this?

Sorry, I had increased the size of the partition that /usr/src and /usr/obj were on, but forgot /tmp was on my root partition.  They are all UFS.  The src and obj are nullfs mounts.

I suppose a sparse file would act differently if it were piped vs. stored?  That's the only difference that patch makes.  The only reason I did that was to better isolate my debug output, but it didn't help in that respect.