Bug 209842 - backtrace(3) function can return negative
Summary: backtrace(3) function can return negative
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 10.3-RELEASE
Hardware: arm Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
Depends on:
Reported: 2016-05-29 20:45 UTC by Poul-Henning Kamp
Modified: 2020-09-21 16:47 UTC (History)
1 user (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Poul-Henning Kamp freebsd_committer 2016-05-29 20:45:28 UTC
The backtrace(3) function returns size_t which is unsigned:

     backtrace(void **addrlist, size_t len);

And is documentet as returning:

     The backtrace() function returns the number of elements that were filled
     in the backtrace. [...]

The implementation in contrib/libexecinfo/unwind.c clearly knows that the return value can be all-ones:

        if (ctx.n != (size_t)~0 && ctx.n > 0)
                ctx.arr[--ctx.n] = NULL;        /* Skip frame below __start */

        return ctx.n;

This happens on a BeagleBoneBlack running

    10.3-STABLE FreeBSD 10.3-STABLE #0 r300092

I am not sure what the proper fix is here, nor for that matter what the problem might be that causes it to return the all-ones value in the first place.

But at the very least ctx.n should be slammed to zero before returning it, so the code calling backtrace(3) doesn't run off the end of the world.
Comment 1 commit-hook freebsd_committer 2020-09-21 16:44:19 UTC
A commit references this bug:

Author: phk
Date: Mon Sep 21 16:43:39 UTC 2020
New revision: 365952
URL: https://svnweb.freebsd.org/changeset/base/365952

  Pull in fix from upstream NetBSD rev. 1.5:

      If Unwind_Backtrace is broken, ctx.n will still contain ~0, and we will
      return that which poor behavior for the user, so return 0 instead.
      We could document ~0 to be an error, but that would deviate from the
      Linux behavior which is not desirable. Noted by Poul-Henning Kamp

  PR:		209842