Bug 252088 - [NEW PORT] devel/ccache-devel: Tool to minimize the compile time of C/C++ programs (Development Branch)
Summary: [NEW PORT] devel/ccache-devel: Tool to minimize the compile time of C/C++ pro...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-23 18:08 UTC by Ivan Rozhuk
Modified: 2024-01-03 00:50 UTC (History)
9 users (show)

See Also:
koobs: maintainer-feedback? (rozhuk.im)


Attachments
patch (10.62 KB, patch)
2020-12-23 18:08 UTC, Ivan Rozhuk
no flags Details | Diff
patch (10.68 KB, patch)
2021-02-12 13:04 UTC, Ivan Rozhuk
no flags Details | Diff
ccache 4.3 (9.50 KB, patch)
2021-08-10 05:29 UTC, Oleg Sidorkin
osidorkin: maintainer-approval?
Details | Diff
patch for ccache-devel support in Mk scripts (807 bytes, patch)
2021-08-10 05:31 UTC, Oleg Sidorkin
no flags Details | Diff
ccache 4.8.3 (10.88 KB, patch)
2023-09-09 21:01 UTC, Ivan Rozhuk
no flags Details | Diff
patch for ccache-devel support in Mk scripts (1.33 KB, patch)
2023-09-09 21:01 UTC, Ivan Rozhuk
rozhuk.im: maintainer-approval?
Details | Diff
ccache 4.8.3 (10.89 KB, patch)
2023-09-09 21:19 UTC, Ivan Rozhuk
no flags Details | Diff
ccache 4.8.3.20231222 (11.08 KB, patch)
2023-12-29 22:06 UTC, Ivan Rozhuk
no flags Details | Diff
ccache 4.8.3.20231222 (11.08 KB, patch)
2024-01-03 00:50 UTC, Ivan Rozhuk
rozhuk.im: maintainer-approval?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Rozhuk 2020-12-23 18:08:20 UTC
Created attachment 220870 [details]
patch

ccache is a compiler cache.  It acts as a caching pre-processor to C/C++
compilers, using the -E compiler switch and a hash to detect when a
compilation can be satisfied from cache.  This often results in a 5 to 10
times speedup in common compilations.

WWW: https://ccache.samba.org/
Comment 1 Ivan Rozhuk 2021-02-12 13:04:25 UTC
Created attachment 222388 [details]
patch

4.2
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2021-07-11 02:16:12 UTC
^Triage: 4.3 is now GA (released),
Comment 3 Oleg Sidorkin 2021-08-10 05:29:19 UTC
Created attachment 227068 [details]
ccache 4.3

updated patch for ccache 4.3
Comment 4 Oleg Sidorkin 2021-08-10 05:31:02 UTC
Created attachment 227069 [details]
patch for ccache-devel support in Mk scripts
Comment 5 Oleg Sidorkin 2022-03-31 12:09:50 UTC
ccache is 4.6 now. It has automatic management of cache directory tree depth and secondary (i.e. shared) storage support.

Could the new version be committed as a separate port?
Comment 6 Kurt Jaeger freebsd_committer freebsd_triage 2023-03-13 21:08:27 UTC
(In reply to Oleg Sidorkin from comment #5)
Is this -devel still necessary ?
Comment 7 Ivan Rozhuk 2023-09-09 21:01:05 UTC
Created attachment 244736 [details]
ccache 4.8.3
Comment 8 Ivan Rozhuk 2023-09-09 21:01:51 UTC
Created attachment 244737 [details]
patch for ccache-devel support in Mk scripts
Comment 9 Ivan Rozhuk 2023-09-09 21:04:43 UTC
(In reply to Kurt Jaeger from comment #6)

It is easy to add -devel port then try to update mishmash -statc -memcached etc.
Comment 10 Ivan Rozhuk 2023-09-09 21:19:21 UTC
Created attachment 244738 [details]
ccache 4.8.3
Comment 11 Oleg Sidorkin 2023-09-10 13:52:57 UTC
(In reply to Ivan Rozhuk from comment #10)

One more here: 
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271536
Comment 12 Kurt Jaeger freebsd_committer freebsd_triage 2023-09-10 16:49:02 UTC
(In reply to Ivan Rozhuk from comment #10)
is there a good reason to use a ccache-devel instead of a ccache4 port ?
Comment 13 Ivan Rozhuk 2023-09-11 02:03:03 UTC
(In reply to Kurt Jaeger from comment #12)

Yes.

1. ccache will become 5x in next year, and there is no reason to keep ccache4 in ports after that. After version bump users must do some extra actions by hands to update ccache4 to ccache5.
Same time ccache-devel (as opposite to legacy unupdateble ccache*) will not require any extra actions to update port on major version bump.

2. This port have features that ccache4 port does not:
- make test - work
- HIREDIS option
- support current and future versions of clang and gcc
- USES= pkgconfig - not missed
- PORTDOCS= LICENSE.html - not missed
- some ccache build options forced to off
Comment 14 Anton Saietskii 2023-09-11 11:03:46 UTC
(In reply to Ivan Rozhuk from comment #13)

I believe "-devel" is incorrect since 4.8.3 is release version, not a beta/alpha/pre-release/etc.

So, we really have to stick with "ccache4" (at least I don't see other options acceptable considering we can't do anything with devel/ccache at the moment). I'd suggest moving your patches to devel/ccache4.
Comment 15 Oleg Sidorkin 2023-09-11 11:47:18 UTC
(In reply to Anton Saietskii from comment #14)

IMO, after ccache4 runs fine with common port tools (poudriere, portmaster), ccache should be moved to ccache3, and ccache4 should become ccache.

This is somehow inevitable, because compilers gain more and more features ccache-3.x.x doesn't support.

IMO, in ccache mode ccache and all its dependencies should be built first as if there is no ccache requested. Then all other ports should be built.
Comment 16 Anton Saietskii 2023-09-11 12:11:14 UTC
(In reply to Oleg Sidorkin from comment #15)

That may occur sooner or later, but anyway "-devel" is simply not true. And, if I remember correctly, adding major version is common practice in ports (e.g. llvm14, llvm15, gcc12, gcc13, etc)
Of course, we can also have "-devel" ccache version, but it should be in fact development version.
Comment 17 Ivan Rozhuk 2023-09-11 21:18:56 UTC
> I believe "-devel" is incorrect since 4.8.3 is release version, not a beta/alpha/pre-release/etc.

I can change version to 4.8.3.yyyymmdd and set to last commit.


> I'd suggest moving your patches to devel/ccache4

It is good for ccache4 :)


> adding major version is common practice in ports (e.g. llvm14, llvm15, gcc12, gcc13, etc)

These ports can be installed and used in parallel.
Comment 18 Ivan Rozhuk 2023-09-11 21:20:22 UTC
bdrewery@
Is there some reasons to continue keep ccache 3.x as default?
Comment 19 Oleg Sidorkin 2023-09-11 22:28:18 UTC
Uncommited part of https://bz-attachments.freebsd.org/attachment.cgi?id=240831 patch allowed to select the desired version using 

DEFAULT_VERSIONS+=ccache=ccache4

entry in make.conf.

I'm still using updated version of it, so I can submit it again as a separate patch if anyone is interested in reviewing and commiting it.
Comment 20 Nuno Teixeira freebsd_committer freebsd_triage 2023-09-13 09:49:02 UTC
(In reply to Oleg Sidorkin from comment #19)

poudriere shows a lot of:
---
ptsort: cycle involving ccache4-4.8 and ...
---

I have CCACHE_DEFAULT?=	ccache4 in bsd.defaults.mk

Do I missing something?
Comment 21 Oleg Sidorkin 2023-09-13 10:52:46 UTC
(In reply to Nuno Teixeira from comment #20)
No, you are not. That's why we have a separate ccache4 port: it has a lot of build dependencies.

For now (until poudriere is changed to handle it) I disable ccache in poudriere config, run poudriere bulk ... devel/ccache4 and enable it again to workaround cyclic deps. poudriere still produces lots of warnings, but works
Comment 22 Ivan Rozhuk 2023-09-14 00:53:51 UTC
(In reply to Nuno Teixeira from comment #20)

This is because there is loop in deps.
To avoid loop Makefile contains:
NO_CCACHE_DEPEND=yes
.export NO_CCACHE_DEPEND
so every sub make will have set NO_CCACHE_DEPEND.
This removes ccache from deps list.

But portmaster, poudriere and some other build tools try to manage build deps, so they call make few times: first they try get deps list (and recursively install it); second to build port.

As work around you may set env var before run your ports tool:
env NO_CCACHE_DEPEND=yes portmaster
Comment 23 Nuno Teixeira freebsd_committer freebsd_triage 2023-09-14 07:32:53 UTC
(In reply to Ivan Rozhuk from comment #22)

Thanks Oleg and Ivan, now I can understand the procedure to have it working.

Waiting for a comparison building chromium (79h !) on rpi4 with ccache, next week I will have results.

Also very curious about ccache performance on my poudriere build machine with 5 jails...

Is there a prevision when changes to Mk/files will be commited?

Cheers
Comment 24 Ivan Rozhuk 2023-09-14 07:49:53 UTC
(In reply to Nuno Teixeira from comment #23)

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246245
You will see super speed only on rebuild chromium same DISTVERSION, in other cases speed will decreased.

May be I will try one more time to hack ports build system to force build all ports in same dir not depend on port version/git hash.
Comment 25 Oleg Sidorkin 2023-09-16 13:19:28 UTC
(In reply to Nuno Teixeira from comment #23)
I've posted the updated version of Mk patch here: 

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234971#c30

So anyone with commit bit can commit it (or file a report in Ports Framework section).
Comment 26 Ivan Rozhuk 2023-12-29 22:06:57 UTC
Created attachment 247341 [details]
ccache 4.8.3.20231222
Comment 27 Ivan Rozhuk 2024-01-03 00:50:38 UTC
Created attachment 247427 [details]
ccache 4.8.3.20231222

update patch hash