Bug 297297 - net/samba423: smbd shows 400-600MB RSS per process in top
Summary: net/samba423: smbd shows 400-600MB RSS per process in top
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: FreeBSD Samba Team (mail alias)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2026-08-05 13:56 UTC by Ronin Carter
Modified: 2026-08-06 10:59 UTC (History)
3 users (show)

See Also:
linimon: maintainer-feedback? (samba)


Attachments
The patch disables per-hash-bucket robust mutexes for Samba's volatile TDBs on FreeBSD (1.43 KB, patch)
2026-08-05 14:07 UTC, Ronin Carter
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ronin Carter 2026-08-05 13:56:14 UTC

    
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2026-08-05 13:57:49 UTC
^Triage: please let us know which port you have that supplies the smbd binary.
Comment 2 Ronin Carter 2026-08-05 14:07:09 UTC
Created attachment 273486 [details]
The patch disables per-hash-bucket robust mutexes for Samba's volatile TDBs on FreeBSD

  Two changes, both FreeBSD-only (#ifdef __FreeBSD__), zero impact on Linux:

  1. source3/lib/dbwrap/dbwrap_open.c — default try_mutex to false in db_open(), so the six volatile SMB TDBs (brlock, leases, locking, smbXsrv_open/session/tcon_global) no longer set TDB_MUTEX_LOCKING (~60k
     mutexes). Still overridable via the existing dbwrap_tdb_mutexes smb.conf param.
  2. source3/lib/gencache.c — drop the default gencache:hash_size from 10000 to 131, since gencache hardcodes TDB_MUTEX_LOCKING and can't be turned off (~10k mutexes).

  Measured effect on the affected host (Samba 4.23.8, FreeBSD 15.0): smbd parent RSS 585MB → 52MB, ~70,460 → 133 shared 4KB mappings, SMB shares verified working.
Comment 3 Ronin Carter 2026-08-05 14:14:28 UTC
(In reply to Mark Linimon from comment #1)

smbd binary is supplied by the net/samba423 port (4.23.8_1), running as a standalone server inside a jail (samba.dirtybsd.lan) on FreeBSD 15.0-RELEASE amd64. 

The tdb library comes from databases/tdb1 

(1.4.14). /usr/local/sbin/smbd reports smbd version 4.23.8
Comment 4 Ronin Carter 2026-08-05 14:40:29 UTC
## Root cause

1. Samba ≥ 4.18 (upstream commit
   `87fddbad78d9a9f6fe922efb7a87ded01996d6ec`,
   "smbd/locking: make use of the same tdb hash_size and flags for all SMB
   related tdb's", landed 2022-12-19) opens six volatile SMB TDBs
   (`brlock.tdb`, `leases.tdb`, `locking.tdb`, `smbXsrv_open_global.tdb`,
   `smbXsrv_session_global.tdb`, `smbXsrv_tcon_global.tdb`) with
   `SMBD_VOLATILE_TDB_HASH_SIZE = 10007` hash buckets.

2. `databases/tdb1` is built with pthread robust-mutex support (default; the
   port does not pass `--disable-tdb-mutex-locking`). `tdb_open_ex()` then
   initializes one **process-shared robust** pthread mutex per hash bucket —
   `lib/tdb/common/mutex.c:588`:

   ```c
   pthread_mutexattr_setpshared(&ma, PTHREAD_PROCESS_SHARED);
   pthread_mutexattr_setrobust(&ma, PTHREAD_MUTEX_ROBUST);
   for (i=0; i<tdb->hash_size+1; i++) {
       ret = pthread_mutex_init(chain, &ma);
   }
   ```

   6 TDBs × 10008 = 60,048 mutexes; observed total incl. other TDBs ≈ 70,000.

3. FreeBSD `libthr` implements each process-shared mutex as a separate 4KB
   shared-memory object (`_umtx_op(UMTX_OP_SHM)` + `mmap(0, 4096, RW,
   MAP_SHARED)` + `close`). On Linux/glibc the mutexes are embedded in the
   TDB's own mmap region, so the identical Samba build uses ~25MB there.
   Result on FreeBSD: ~70,000 × 4KB ≈ **290MB** of anonymous shared memory.

## Evidence (captured on the affected host)

- `ktrace` startup trace shows a loop of ~70,000 iterations:

  ```
  _umtx_op(0, UMTX_OP_SHM, LOOKUP, name)  -> ESRCH
  _umtx_op(0, UMTX_OP_SHM, MMAP, name)    -> fd
  mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)
  close(fd)
  ```

- `lldb` backtrace at the loop:

  ```
  smbd main -> smbXsrv_tcon_global_init -> db_open -> db_open_tdb
            -> tdb_wrap_open -> tdb_open_ex -> pthread_mutex_init
            -> __sys__umtx_op
  ```

- `procstat -v` on the smbd parent: 70,458 anonymous mappings of exactly
  4096 bytes (290MB), all resident.
- `libtdb.so.1` imports `pthread_mutexattr_setpshared` /
  `pthread_mutexattr_setrobust` / `pthread_mutex_consistent` (mutex support
  compiled in).
- Same symptom reported on the FreeBSD forums (Samba 4.19/4.20, FreeBSD
  14.3, fresh VM, blank config):
  https://forums.freebsd.org/threads/samba-smbd-abnormally-high-startup-memory-usage.98780/

## Impact

- ~290MB wasted per smbd instance on any FreeBSD box running Samba ≥ 4.18
  (standalone, NT4 domain, or AD DC — an AD DC runs more TDBs/processes, so
  worse). Scales with the number of 10007-bucket TDBs opened.
- Not workload-dependent and not a leak, so easy to miss; also affects any
  other FreeBSD application that opens tdb1 databases with large hash sizes.

## Validated fix (tested on the affected host, 2026-08-05)

No rebuild is required — the mutex allocation is controllable at runtime:

```
[global]
   dbwrap_tdb_mutexes:* = no     # turns off TDB_MUTEX_LOCKING in dbwrap_open()
   gencache:hash_size = 131      # gencache hardcodes TDB_MUTEX_LOCKING; shrink its
                                 # default 10000-bucket hash (delete gencache.tdb
                                 # once so it is recreated at the new size)
```

`dbwrap_open.c` only requests `TDB_MUTEX_LOCKING` when
`lp_parm_bool("dbwrap_tdb_mutexes", ..., true)` — so the parameter is an
intentional (if undocumented) kill switch. gencache sets the flag directly
(`source3/lib/gencache.c:74`) with its own `gencache:hash_size` param, so it
needs the second line.

Measured before/after (Samba 4.23.8, FreeBSD 15.0, same workload):

| metric | before | after |
|---|---|---|
| 4KB MAP_SHARED shm mappings (procstat) | 70,460 (~275MB) | 133 (~0.5MB) |
| smbd parent RSS | 585MB | 52MB |
| smbd notifyd/cleanupd RSS | 432MB | 51MB |
| summed smbd in top | ~1.9GB | ~205MB |

SMB shares verified working after the change (client connected, share listed).
Comment 5 Xavier Beaudouin freebsd_committer freebsd_triage 2026-08-06 10:59:06 UTC
Well tried to apply your patch, but... it seems it cannot applys "as is".

How did you applied this patch in the ports? Because on current port this cannot be applied "as is".

Kind regards