Bug 253905 - databases/mysql80-server excessive free storage requirements for building from source
Summary: databases/mysql80-server excessive free storage requirements for building fro...
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Jochen Neumeister
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-02-28 11:18 UTC by Viktor Štujber
Modified: 2021-02-28 11:18 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (joneum)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Viktor Štujber 2021-02-28 11:18:46 UTC
mysql-server at this point requires 1GB of storage to build via portmaster, multiple times more than any other software I'm using. And from what I've observed, it's not due to the inherent complexity of the source code, but due to compounding inefficiencies in the build process.

The distfile 'mysql-boost-8.0.23.tar.gz' is 278 MB.
(mysql-5.5.62.tar.gz size is 20 MB.)
The extracted source tree is 810 MB.
(From that, the mysql-test/ directory is 516 MB.)
The intermediate .build/ directory, without tests, is 947 MB.
The stage/ directory is another 177 MB.

In total, the required free storage is no less than 2212 MB, to produce a 177 MB package - a 12.5 : 1 ratio. Back in the mysql 5.x days, it was possible to fit the whole build, distfile included, onto a 1 GB tmpfs. Perhaps it would be worthwhile to get closer to that kind of number. Here are my observations.

The ports makefile currently has no options. One of them could be to disable tests. I did not find any direct knob in their cmakefile that would do that, but patching it out is easy. That in turn allows having a post-extract step delete the mysql-test subdirectory for immediate savings of ~516 MB.

The build process generates intermediate .o files and resulting executables/libraries for the entire source tree without any cleanup steps inbetween. That is, even though the results are already built, it still keeps the temporary stuff around. And it does in fact require them to be there - they will be recompiled if missing, even though all the outputs have already been built. If the build system instead cleaned its temporary files when they were no longer needed, and did not require them once the corresponding outputs were built, that would save ~200 MB.

When the build completes and the files are staged for installation, the staging is done via copying. It might be possible to do this via symlinks during staging, and then resolving them during install. This would save another ~177 MB.

I understand that these ideas may not be compatible with how freebsd and mysql currently does builds, and changes could require upstream cooperation. I just figured I'd write this to give this issue (or non-issue?) a bit of attention.