I'm trying to run Mecrisp Stellaris 2.5.8 on arm64 FreeBSD. This is a Forth system written entirely in armv7 thumb2 assembly. A FreeBSD port has existed for a while and has recently been made available as lang/mecrisp-stellaris in the ports tree.
It appears that with FreeBSD 13, rudimentary support for loading and running armv7 binaries from arm64 FreeBSD exists. And indeed, mecrisp-stellaris loads and executes fine. However, problems crop up once more definitions are loaded. Specifically, the system crashes in random places with a wildly inconsistent program state.
To reproduce the issue, download Mecrisp Stellaris 2.5.8 from Sourceforge  onto an arm64 FreeBSD 13 system. Navigate into the freebsd-ra directory and execute "buildcore-freebsd". You should see a few pages of output followed by a crash. The crash is a segmentation fault or an illegal instruction fault.
The same issue does not seem to appear on armv7 FreeBSD, so I'm a bit at a loss as to what may have caused this.
One clue: the code uses a sequence of barried instructions (DMB; DSB; ISB) instructions to synchronise caches before jumping to recently compiled code. If these instructions are replaced with an appropriate sysarch(ARM_SYNC_ICACHE, ...) call, the problem goes away. It is possible that I am misusing the barrier instructions, though if so, I don't understand how.
My bad. It appears that the synchronisation code is indeed incorrect. Citing from the ARMv8 ARM, section E2.5.3:
> If software requires coherency between instruction execution and
> memory, it must manage this coherency using Context synchronization
> events and cache maintenance instructions. These can only be
> accessed from an Exception level that is higher than EL0, and
> therefore require a system call [...].
Closing this one as invalid.
(In reply to Robert Clausecker from comment #2)
Using armv7 documentation instead of armv8:
If software requires coherency between instruction execution and memory, it must manage this coherency using the ISB and DSB memory barriers and cache maintenance operations, see Ordering of cache and branch predictor maintenance operations. Many of these operations are only available to system software.