Bug 266592 - diff(1): 'diff -rl' called on 2 large directories opens too many files
Summary: diff(1): 'diff -rl' called on 2 large directories opens too many files
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 16.0-CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Dag-Erling Smørgrav
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-25 07:29 UTC by Yuri Victorovich
Modified: 2026-02-11 06:16 UTC (History)
1 user (show)

See Also:
des: mfc-stable15+
des: mfc-stable14+
des: mfc-stable13+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2022-09-25 07:29:48 UTC
I've ran this command and it failed:
> $ diff -rl work.test1 work
> diff: kqueue: Too many open files in system

Both work and work.test1 are build directories of one port.

ktrace(1) shows that the file descriptor value grows continuously until the above error occurs.

Correct behavior: the above command shouldn't keep many files open. It is supposed to find differences between files, and can have 2 compared files open and perhaps also 2 compared directories open.
There should be no need to keep a large number of files open.


13.1-STABLE
Comment 1 commit-hook freebsd_committer freebsd_triage 2026-02-05 14:46:51 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5

commit c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5
Author:     Dag-Erling Smørgrav <des@FreeBSD.org>
AuthorDate: 2026-02-05 14:39:53 +0000
Commit:     Dag-Erling Smørgrav <des@FreeBSD.org>
CommitDate: 2026-02-05 14:39:53 +0000

    diff: Fix pagination leak

    * Drop an unnecessary variable and rename pidfd to procd.

    * Rewinding stdout serves no purpose, so stop doing it.

    * Don't bother freeing memory or setting the global status right
      before erroring out.

    * Error out if dup(2) or dup2(2) fail.

    * In the unlikely case that our pipe is equal to stdout, we need to
      record that information so we don't close it when cleaning up.

    * Don't bother closing a descriptor before dup2(2)ing to it.

    * Don't forget to close the the process descriptor after reaping the
      child process.

    PR:             266592
    MFC after:      1 week
    Sponsored by:   Klara, Inc.
    Reviewed by:    kevans, markj
    Differential Revision:  https://reviews.freebsd.org/D55112

 usr.bin/diff/pr.c | 22 ++++++++++------------
 usr.bin/diff/pr.h |  2 +-
 2 files changed, 11 insertions(+), 13 deletions(-)
Comment 2 commit-hook freebsd_committer freebsd_triage 2026-02-10 15:28:07 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e87b9226551e88cb0649d4aaa54b041156df83f7

commit e87b9226551e88cb0649d4aaa54b041156df83f7
Author:     Dag-Erling Smørgrav <des@FreeBSD.org>
AuthorDate: 2026-02-05 14:39:53 +0000
Commit:     Dag-Erling Smørgrav <des@FreeBSD.org>
CommitDate: 2026-02-10 14:24:08 +0000

    diff: Fix pagination leak

    * Drop an unnecessary variable and rename pidfd to procd.

    * Rewinding stdout serves no purpose, so stop doing it.

    * Don't bother freeing memory or setting the global status right
      before erroring out.

    * Error out if dup(2) or dup2(2) fail.

    * In the unlikely case that our pipe is equal to stdout, we need to
      record that information so we don't close it when cleaning up.

    * Don't bother closing a descriptor before dup2(2)ing to it.

    * Don't forget to close the the process descriptor after reaping the
      child process.

    PR:             266592
    MFC after:      1 week
    Sponsored by:   Klara, Inc.
    Reviewed by:    kevans, markj
    Differential Revision:  https://reviews.freebsd.org/D55112

    (cherry picked from commit c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5)

 usr.bin/diff/pr.c | 22 ++++++++++------------
 usr.bin/diff/pr.h |  2 +-
 2 files changed, 11 insertions(+), 13 deletions(-)
Comment 3 commit-hook freebsd_committer freebsd_triage 2026-02-10 15:28:10 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=144455c333dc0d3db369596038de2e3dd6113b46

commit 144455c333dc0d3db369596038de2e3dd6113b46
Author:     Dag-Erling Smørgrav <des@FreeBSD.org>
AuthorDate: 2026-02-05 14:39:53 +0000
Commit:     Dag-Erling Smørgrav <des@FreeBSD.org>
CommitDate: 2026-02-10 14:24:19 +0000

    diff: Fix pagination leak

    * Drop an unnecessary variable and rename pidfd to procd.

    * Rewinding stdout serves no purpose, so stop doing it.

    * Don't bother freeing memory or setting the global status right
      before erroring out.

    * Error out if dup(2) or dup2(2) fail.

    * In the unlikely case that our pipe is equal to stdout, we need to
      record that information so we don't close it when cleaning up.

    * Don't bother closing a descriptor before dup2(2)ing to it.

    * Don't forget to close the the process descriptor after reaping the
      child process.

    PR:             266592
    MFC after:      1 week
    Sponsored by:   Klara, Inc.
    Reviewed by:    kevans, markj
    Differential Revision:  https://reviews.freebsd.org/D55112

    (cherry picked from commit c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5)

 usr.bin/diff/pr.c               | 22 ++++++++++------------
 usr.bin/diff/pr.h               |  1 +
 usr.bin/diff/tests/diff_test.sh |  1 -
 3 files changed, 11 insertions(+), 13 deletions(-)
Comment 4 commit-hook freebsd_committer freebsd_triage 2026-02-10 15:33:17 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=44cf3a1f6da63b064b4356d2b3c6d7834ee03a97

commit 44cf3a1f6da63b064b4356d2b3c6d7834ee03a97
Author:     Dag-Erling Smørgrav <des@FreeBSD.org>
AuthorDate: 2026-02-05 14:39:53 +0000
Commit:     Dag-Erling Smørgrav <des@FreeBSD.org>
CommitDate: 2026-02-10 15:32:47 +0000

    diff: Fix pagination leak

    * Drop an unnecessary variable and rename pidfd to procd.

    * Rewinding stdout serves no purpose, so stop doing it.

    * Don't bother freeing memory or setting the global status right
      before erroring out.

    * Error out if dup(2) or dup2(2) fail.

    * In the unlikely case that our pipe is equal to stdout, we need to
      record that information so we don't close it when cleaning up.

    * Don't bother closing a descriptor before dup2(2)ing to it.

    * Don't forget to close the the process descriptor after reaping the
      child process.

    PR:             266592
    MFC after:      1 week
    Sponsored by:   Klara, Inc.
    Reviewed by:    kevans, markj
    Differential Revision:  https://reviews.freebsd.org/D55112

    (cherry picked from commit c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5)

 usr.bin/diff/pr.c | 22 ++++++++++------------
 usr.bin/diff/pr.h |  1 +
 2 files changed, 11 insertions(+), 12 deletions(-)
Comment 5 commit-hook freebsd_committer freebsd_triage 2026-02-11 06:16:10 UTC
A commit in branch releng/14.4 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=6761e555376e375f89fe36c72f2813f5e64a7f63

commit 6761e555376e375f89fe36c72f2813f5e64a7f63
Author:     Dag-Erling Smørgrav <des@FreeBSD.org>
AuthorDate: 2026-02-05 14:39:53 +0000
Commit:     Colin Percival <cperciva@FreeBSD.org>
CommitDate: 2026-02-11 06:14:26 +0000

    diff: Fix pagination leak

    * Drop an unnecessary variable and rename pidfd to procd.

    * Rewinding stdout serves no purpose, so stop doing it.

    * Don't bother freeing memory or setting the global status right
      before erroring out.

    * Error out if dup(2) or dup2(2) fail.

    * In the unlikely case that our pipe is equal to stdout, we need to
      record that information so we don't close it when cleaning up.

    * Don't bother closing a descriptor before dup2(2)ing to it.

    * Don't forget to close the the process descriptor after reaping the
      child process.

    Approved by:    re (cperciva)
    PR:             266592
    MFC after:      1 week
    Sponsored by:   Klara, Inc.
    Reviewed by:    kevans, markj
    Differential Revision:  https://reviews.freebsd.org/D55112

    (cherry picked from commit c3904a7de78ca1ca15fcdf4c09f9d4be7f6fe6f5)
    (cherry picked from commit 144455c333dc0d3db369596038de2e3dd6113b46)

 usr.bin/diff/pr.c               | 22 ++++++++++------------
 usr.bin/diff/pr.h               |  1 +
 usr.bin/diff/tests/diff_test.sh |  1 -
 3 files changed, 11 insertions(+), 13 deletions(-)