Bug 263845 - graphics/libglvnd: Library compiled in ARM mode, but executes in Thumb mode, crashing programs on FreeBSD 13 armv7 with SIGILL
Summary: graphics/libglvnd: Library compiled in ARM mode, but executes in Thumb mode, ...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: arm Any
: --- Affects Some People
Assignee: freebsd-x11 (Nobody)
URL:
Keywords: crash, needs-qa
Depends on:
Blocks: 263849
  Show dependency treegraph
 
Reported: 2022-05-07 15:42 UTC by Robert Clausecker
Modified: 2022-06-05 16:11 UTC (History)
5 users (show)

See Also:
manu: maintainer-feedback+
koobs: merge-quarterly?


Attachments
graphics/libglvnd: fix on armv7 (1.97 KB, patch)
2022-05-31 17:20 UTC, Robert Clausecker
no flags Details | Diff
graphics/libglvnd: fix on armv6/armv7 (2.46 KB, patch)
2022-06-03 13:24 UTC, Robert Clausecker
fuz: maintainer-approval? (x11)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Clausecker freebsd_committer freebsd_triage 2022-05-07 15:42:52 UTC
The following simple program crashes with SIGILL when compiled on armv7 FreeBSD 13:

#include <GL/gl.h>
int main()
{
	glGetError();
}

I am compiling with "cc -I/usr/local/lib -L/usr/local/include test.c -lGL.  My package version is libglvnd 1.4.0_1 and I've tested this both on native armv7 and in an armv7 jail running on arm64.

The behaviour is extremely strange.  I am aware that glvnd does some sort of dispatch logic to get to the platform's native driver, but this does not match what I see.

When I try to debug this code, it seems that it is somehow iterating through the entirety of the the libGL API, only to then crash after having gone through glWriteMaskEXT.  I do not understand what the purpose of this is and it does not seem to match any code I found in the library.

Digging further into this, it seems that the library is compiled in ARM mode, but executed in Thumb mode for some reason (wtf).  I suspect this might be because the source file src/GLdispatch/vnd-glapi/entry_armv7_tsd.c uses ".thumb_func" to label the stub entry points, but then they don't get compiled in thumb mode for some reason.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2022-05-31 00:06:33 UTC
@Reporter Could you please add full uname -a output please along with /var/run/dmesg.boot (as attachment) for full system/environment details, thanks
Comment 2 Robert Clausecker freebsd_committer freebsd_triage 2022-05-31 00:20:06 UTC
(In reply to Kubilay Kocak from comment #1)

Hi there, I'm running my tests in an armv7 jail on a Raspberry Pi 4B (arm64, Cortex-A72, ARMv8-A).  The OS version is FreeBSD 13.0 with a patch to fix debugging (see bug #256468).

$ uname -a
FreeBSD tau.fuz.su 13.0-RELEASE-p7 FreeBSD 13.0-RELEASE-p7 #2: Fri Feb 18 23:08:54 CET 2022     fuz@tau.fuz.su:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC  arm64

$ sed -ne '/CPU  0/,/CPU  3/p' /var/run/dmesg.boot
CPU  0: ARM Cortex-A72 r0p3 affinity:  0
                   Cache Type = <64 byte D-cacheline,64 byte I-cacheline,PIPT ICache,64 byte ERG,64 byte CWG>

 Instruction Set Attributes 0 = <CRC32>
 Instruction Set Attributes 1 = <>
         Processor Features 0 = <AdvSIMD,FP,EL3 32,EL2 32,EL1 32,EL0 32>
         Processor Features 1 = <>
      Memory Model Features 0 = <TGran4,TGran64,SNSMem,BigEnd,16bit ASID,16TB PA>
      Memory Model Features 1 = <8bit VMID>
      Memory Model Features 2 = <32bit CCIDX,48bit VA>
             Debug Features 0 = <2 CTX BKPTs,4 Watchpoints,6 Breakpoints,PMUv3,Debugv8>
             Debug Features 1 = <>
         Auxiliary Features 0 = <>
         Auxiliary Features 1 = <>
CPU  1: ARM Cortex-A72 r0p3 affinity:  1
CPU  2: ARM Cortex-A72 r0p3 affinity:  2
CPU  3: ARM Cortex-A72 r0p3 affinity:  3
Comment 3 Robert Clausecker freebsd_committer freebsd_triage 2022-05-31 13:06:37 UTC
Upstream issue reported: https://gitlab.freedesktop.org/glvnd/libglvnd/-/issues/232
Comment 4 Robert Clausecker freebsd_committer freebsd_triage 2022-05-31 17:20:19 UTC
Created attachment 234353 [details]
graphics/libglvnd: fix on armv7

The .thumb_func directive does not switch to thumb mode in clang's
integrated assembler.  This causes the code to be miscompiled in that
it is generated as ARM code but the symbols are tagged as thumb.  The
resulting library is unusable.  Every call into it crashes.

Add a patch to fix the behaviour and bump PORTREVISION so consumers
get the updated library.

Tested with Poudriere on armv7 FreeBSD 13.
Please MFH if possible.
Comment 5 Li-Wen Hsu freebsd_committer freebsd_triage 2022-06-03 08:39:18 UTC
(In reply to Robert Clausecker from comment #3)
From the thread there, should we use the patch in https://gitlab.freedesktop.org/glvnd/libglvnd/-/merge_requests/267/diffs for entry_armv7_tsd.c ?
Comment 6 Robert Clausecker freebsd_committer freebsd_triage 2022-06-03 09:13:43 UTC
(In reply to Li-Wen Hsu from comment #5)

Yeah, let me rework the patch to use that patch instead.
Comment 7 Robert Clausecker freebsd_committer freebsd_triage 2022-06-03 13:24:45 UTC
Created attachment 234421 [details]
graphics/libglvnd: fix on armv6/armv7

The .thumb_func directive does not switch to thumb mode in clang's
integrated assembler.  This causes the code to be miscompiled in that
it is generated as ARM code but the symbols are tagged as thumb.  The
resulting library is unusable.  Every call into it crashes.

Fix this issue by applying an upstream patch set to have the entry
stubs be in ARM mode instead of Thumb mode.

See also:	https://gitlab.freedesktop.org/glvnd/libglvnd/-/merge_requests/267
See also:	https://gitlab.freedesktop.org/glvnd/libglvnd/-/issues/232
PR:		263845

Tested with Poudriere on armv7 arm64 FreeBSD 13.
Portlint likes the port.
Comment 8 commit-hook freebsd_committer freebsd_triage 2022-06-03 13:42:57 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=239ba7a240fd6fd611ac6a72986de6b26ab5da64

commit 239ba7a240fd6fd611ac6a72986de6b26ab5da64
Author:     Robert Clausecker <fuz@fuz.su>
AuthorDate: 2022-06-03 13:18:05 +0000
Commit:     Emmanuel Vadot <manu@FreeBSD.org>
CommitDate: 2022-06-03 13:41:40 +0000

    graphics/libglvnd: fix on armv6/armv7

    The .thumb_func directive does not switch to thumb mode in clang's
    integrated assembler.  This causes the code to be miscompiled in that
    it is generated as ARM code but the symbols are tagged as thumb.  The
    resulting library is unusable.  Every call into it crashes.

    Fix this issue by applying an upstream patch set to have the entry
    stubs be in ARM mode instead of Thumb mode.

    See also:       https://gitlab.freedesktop.org/glvnd/libglvnd/-/merge_requests/267
    See also:       https://gitlab.freedesktop.org/glvnd/libglvnd/-/issues/232
    PR:             263845

 graphics/libglvnd/Makefile | 7 ++++++-
 graphics/libglvnd/distinfo | 8 +++++++-
 2 files changed, 13 insertions(+), 2 deletions(-)
Comment 9 Robert Clausecker freebsd_committer freebsd_triage 2022-06-03 21:43:26 UTC
Please don't forget to MFH this change.
Comment 10 commit-hook freebsd_committer freebsd_triage 2022-06-05 16:11:25 UTC
A commit in branch 2022Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=ef2d763b3855c82449ac171a538d90e70cbd7ba1

commit ef2d763b3855c82449ac171a538d90e70cbd7ba1
Author:     Robert Clausecker <fuz@fuz.su>
AuthorDate: 2022-06-03 13:18:05 +0000
Commit:     Li-Wen Hsu <lwhsu@FreeBSD.org>
CommitDate: 2022-06-05 16:09:55 +0000

    graphics/libglvnd: fix on armv6/armv7

    The .thumb_func directive does not switch to thumb mode in clang's
    integrated assembler.  This causes the code to be miscompiled in that
    it is generated as ARM code but the symbols are tagged as thumb.  The
    resulting library is unusable.  Every call into it crashes.

    Fix this issue by applying an upstream patch set to have the entry
    stubs be in ARM mode instead of Thumb mode.

    See also:       https://gitlab.freedesktop.org/glvnd/libglvnd/-/merge_requests/267
    See also:       https://gitlab.freedesktop.org/glvnd/libglvnd/-/issues/232
    PR:             263845

    (cherry picked from commit 239ba7a240fd6fd611ac6a72986de6b26ab5da64)

 graphics/libglvnd/Makefile | 7 ++++++-
 graphics/libglvnd/distinfo | 8 +++++++-
 2 files changed, 13 insertions(+), 2 deletions(-)