Bug 267346

Summary: devel/llvm13 takes very long to compile on the Raspberry Pi B+
Product: Ports & Packages Reporter: Martin Birgmeier <d8zNeCFG>
Component: Individual Port(s)Assignee: Brooks Davis <brooks>
Status: Closed Works As Intended    
Severity: Affects Only Me CC: diizzy, emaste, freebsd-arm
Priority: --- Flags: bugzilla: maintainer-feedback? (brooks)
Version: Latest   
Hardware: Any   
OS: Any   

Description Martin Birgmeier 2022-10-25 18:02:48 UTC
(this is a little tongue-in-cheek :-))

Scenario:
- Raspberry Pi model B+ https://www.raspberrypi.com/products/raspberry-pi-1-model-b-plus/ and a 2.5" HDD via USB
- FreeBSD 14 at 1ffd352bc25
- ports at 2bc1986c8c9
- upgrading ports using portmaster, including x11/xorg
- this requires rebuilding devel/llvm13 and lang/rust

Result:
- building llvm13-13.0.1_3 takes about 4.5 weeks
- building rust-1.62.1 takes about 3.5 weeks
- rebuilding 443 ports takes about 2.5 months

Expected result:
- This should go much faster...

The nice thing is that it is even working!

There was a time when ports for armv6 for the RPI B+ could be compiled using emulators/qemu-user-static on multiple cores, but alas this does not work anymore.

It would be interesting to know what uses more energy - the native compilation on the RPI B+, or qemu-user-static with four cores.

-- Martin

p.s. Does anyone know of FreeBSD drivers for the peripherals of a GrovePi (I2C is working)?
Comment 1 Brooks Davis freebsd_committer freebsd_triage 2022-10-25 21:28:48 UTC
I'm impressed it works at all. If your HDD is a spinning disk it's remotely possible that switching to an SSD could help a little. It is also possible to cross compile LLVM (though not in the current ports framework). I've got a prototype somewhere to create minimal flavors that have no dependences and could potentially be pre-populated into a cross-built repo for use with poudriere -b. I'll think about dusting those changes off, but don't plan to keep this PR open to track it. 

qemu-user-static is incredibly fragile and sadly pretty much unmaintained.

I'd suggest asking about GrovePi on the forums on mailing lists.
Comment 2 Daniel Engberg freebsd_committer freebsd_triage 2022-10-25 21:36:44 UTC
Hi,

This book called "Learning Raspbian" (https://www.packtpub.com/product/learning-raspbian/9781784392192) says this about performance:

At the heart of Raspberry Pi is the powerful Broadcom BCM2835 "system on a chip". The BCM2835 is similar to the chip at the heart of almost every smartphone and set top box in the world that uses ARM architecture. The BCM2835 CPU on the Raspberry Pi runs at 700 MHz and its performance is roughly equivalent to a 300 MHz Pentium II computer that was available back in 1999. [1]

It was never meant to be a "buildbox" and it will be slow due to hardware limitations. Emulating ARM will most likely be faster even if emulation by design is quite compute intensive.

Building both LLVM and Rust is very optimistic with less than ~8Gb of RAM (UFS) and much swap space available with 2-3 jobs at some points during compilation. Using more jobs and/or ZFS will use a lot more memory.

1: https://subscription.packtpub.com/book/hardware-and-creative/9781784392192/1/ch01lvl1sec10/the-raspberry-pi-hardware

If you have questions please use other means such as mailing lists, forums or IRC.
Comment 3 Martin Birgmeier 2022-10-26 06:55:43 UTC
(In reply to Brooks Davis from comment #1)

Thank you for your earnest reply, even though this PR was not really about llvm13.

Regarding the build time it seems that waiting for the spinning HDD is nearly negligible. Only the installs (to the slow SDHC card) take several seconds or even minutes. This can be observed from the CPU load - only the installs are not CPU-bound.

There are a few PRs regarding qemu-user-static, it is clear that this is incredibly complicated, especially for armv6 (which probably is not even supported anymore upstream).

I'll check the forums etc. regarding the GrovePi.

-- Martin
Comment 4 Martin Birgmeier 2022-10-26 07:05:13 UTC
(In reply to Daniel Engberg from comment #2)

Thank you for your reply and pointers.

Yes, the RPI-B+ is not a build box, however having seemingly no alternative left it had to serve as one. Plus it was interesting to see whether this would succeed at all.

Regarding HW, there are no Gigabytes. The kernel says, real memory  = 469757952 (447 MB), avail memory = 443703296 (423 MB). The swap area (added from the HDD) is nominally 5GB, although not all of this can be handled by the kernel due to memory limitations (it complains when adding the swap). Still, all of this suffices to compile both rust and llvm13, which is quite nice.

And not only compile, but also actually run these programs - they are used when building x11/xorg, and the X server is indeed working in full resolution.

I probably should have posted this to the forums in the first place, however thought the information might be better preserved here.

-- Martin