Bug 232210 - sendfile(2) doesn't percolate up issues with copyout(9) related to sbytes
Summary: sendfile(2) doesn't percolate up issues with copyout(9) related to sbytes
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Enji Cooper
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2018-10-12 23:25 UTC by Enji Cooper
Modified: 2019-02-12 02:58 UTC (History)
2 users (show)

See Also:
ngie: mfc-stable10?
ngie: mfc-stable11?
ngie: mfc-stable12?


Attachments
Please use the pull request instead. (397 bytes, patch)
2018-10-12 23:25 UTC, Enji Cooper
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Enji Cooper freebsd_committer 2018-10-12 23:25:33 UTC
Created attachment 198079 [details]
Please use the pull request instead.

While doing some digging around the syscall, I noticed (via code inspection) that it doesn't percolate issues with copyout back to userspace. This means that failures to call copyout might result in incorrectly communicated values for sbytes.

The attached patch percolates up the EFAULT issues from copyout.
Comment 1 Enji Cooper freebsd_committer 2018-10-13 00:14:53 UTC
(In reply to Enji Cooper from comment #0)

I thought about the proposed patch and I don't think it was a good complete solution.

I have submitted https://github.com/freebsd/freebsd/pull/169 instead for the issue.
Comment 2 Gleb Smirnoff freebsd_committer 2018-10-13 16:28:39 UTC
That's a good catch, but I don't fully agree on the patch. Some return values of sendfile aren't errors, and the patch doesn't cover them. If sendfile returns EAGAIN, which is normal for non-blocking socket, we again ignore copyout error. I think we should take in copyout error always.
Comment 3 commit-hook freebsd_committer 2019-01-23 22:00:58 UTC
A commit references this bug:

Author: ngie
Date: Wed Jan 23 22:00:18 UTC 2019
New revision: 343362
URL: https://svnweb.freebsd.org/changeset/base/343362

Log:
  Add [initial] functional tests for sendfile(2) as lib/libc/sys/sendfile

  These testcases exercise a number of functional requirements for sendfile(2).

  The testcases use IPv4 and IPv6 domain sockets with TCP, and were confirmed
  functional on UFS and ZFS. UDP address family sockets cannot be used per the
  sendfile(2) contract, thus using UDP sockets is outside the scope of
  testing the syscall in positive cases. As seen in
  `:s_negative_udp_socket_test`, UDP is used to test the sendfile(2) contract
  to ensure that EINVAL is returned by sendfile(2).

  The testcases added explicitly avoid testing out `SF_SYNC` due to the
  complexity of verifying that support. However, this is a good next logical
  item to verify.

  The `hdtr_positive*` testcases work to a certain degree (the header
  testcases pass), but the trailer testcases do not work (it is an expected
  failure). In particular, the value received by the mock server doesn't match
  the expected value, and instead looks something like the following (using
  python array notation):

  `trailer[:]message[1:]`

  instead of:

  `message[:]trailer[:]`

  This makes me think there's a buffer overrun issue or problem with the
  offset somewhere in the sendfile(2) system call, but I need to do some
  other testing first to verify that the code is indeed sane, and my
  assumptions/code isn't buggy.

  The `sbytes_negative` testcases that check `sbytes` being set to an
  invalid value resulting in `EFAULT` fails today as the other change
  (which checks `copyout(9)`) has not been committed [1]. Thus, it
  should remain an expected failure (see bug 232210 for more details
  on this item).

  Next steps for testing sendfile(2):
  1. Fix the header/trailer testcases so that they pass.
  2. Setup if_tap interface and test with it, instead of using "localhost", per
     @asomers's suggestion.
  3. Handle short recv(2)'s in `server_cat(..)`.
  4. Add `SF_SYNC` support.
  5. Add some more negative tests outside the scope of the functional contract.

  MFC after:	1 month
  Reviewed by:	asomers
  Approved by:	emaste (mentor)
  PR: 		232210
  Sponsored by:   Netflix, Inc
  Differential Revision: https://reviews.freebsd.org/D18625

Changes:
  head/lib/libc/tests/sys/Makefile
  head/lib/libc/tests/sys/sendfile_test.c
Comment 4 commit-hook freebsd_committer 2019-01-25 19:56:39 UTC
A commit references this bug:

Author: ngie
Date: Fri Jan 25 19:56:02 UTC 2019
New revision: 343444
URL: https://svnweb.freebsd.org/changeset/base/343444

Log:
  Document that `sendfile` will return an invalid value for `sbytes` if provided an invalid address

  This is meant to clarify the fact that the system call will not fail
  with -1/EFAULT, as one might expect, when reading the sendfile(2)
  manpage today.

  While here, pet the mandoc linter, when dealing with the section that
  describes valid values for `flags`.

  PR:	232210
  MFC after:	2 weeks
  Approved by:	emaste (mentor)
  Reviewed by:	glebius, 0mp
  Differential Revision: https://reviews.freebsd.org/D18949

Changes:
  head/lib/libc/sys/sendfile.2
Comment 5 commit-hook freebsd_committer 2019-02-12 02:57:34 UTC
A commit references this bug:

Author: ngie
Date: Tue Feb 12 02:57:29 UTC 2019
New revision: 344037
URL: https://svnweb.freebsd.org/changeset/base/344037

Log:
  MFC r343444:

  Document that `sendfile` will return an invalid value for `sbytes` if provided an invalid address

  This is meant to clarify the fact that the system call will not fail
  with -1/EFAULT, as one might expect, when reading the sendfile(2)
  manpage today.

  While here, pet the mandoc linter, when dealing with the section that
  describes valid values for `flags`.

  PR:		232210
  Approved by:	jtl (mentor)
  Differential Revision: https://reviews.freebsd.org/D19151

Changes:
_U  stable/12/
  stable/12/lib/libc/sys/sendfile.2
Comment 6 commit-hook freebsd_committer 2019-02-12 02:58:37 UTC
A commit references this bug:

Author: ngie
Date: Tue Feb 12 02:57:35 UTC 2019
New revision: 344038
URL: https://svnweb.freebsd.org/changeset/base/344038

Log:
  MFC r339343,r343444:

  r339343 (by allanjude):

  Document that sendfile(2) can return ENOTCAPABLE

  PR:		232207

  r343444:

  Document that `sendfile` will return an invalid value for `sbytes` if provided an invalid address

  This is meant to clarify the fact that the system call will not fail
  with -1/EFAULT, as one might expect, when reading the sendfile(2)
  manpage today.

  While here, pet the mandoc linter, when dealing with the section that
  describes valid values for `flags`.

  PR:		232210
  Approved by:	jtl (mentor)
  Differential Revision: https://reviews.freebsd.org/D19150

Changes:
_U  stable/11/
  stable/11/lib/libc/sys/sendfile.2