Bug 259319 - openpty(3) and forkpty(3) don't work under Linux compat
Summary: openpty(3) and forkpty(3) don't work under Linux compat
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-20 15:47 UTC by Julio Merino,+1 347 694 0576,New York City
Modified: 2021-10-20 19:43 UTC (History)
1 user (show)

See Also:


Attachments
Small repro program that uses openpty (418 bytes, text/plain)
2021-10-20 15:47 UTC, Julio Merino,+1 347 694 0576,New York City
no flags Details
kdump output of the repro program built for FreeBSD (10.94 KB, text/plain)
2021-10-20 15:48 UTC, Julio Merino,+1 347 694 0576,New York City
no flags Details
kdump output of the repro program built for Linux (10.94 KB, text/plain)
2021-10-20 15:48 UTC, Julio Merino,+1 347 694 0576,New York City
no flags Details
kdump output of the repro program built for Linux (15.62 KB, text/plain)
2021-10-20 15:51 UTC, Julio Merino,+1 347 694 0576,New York City
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 15:47:29 UTC
Created attachment 228888 [details]
Small repro program that uses openpty

Linux binaries that call into openpty(3) or forkpty(3) get "No such file or directory" as a result of these calls. This is in combination with linux_base-c7-7.9.2009.

I've found this issue while trying to run the VSCode remote server under Linux compat, but I have attached a trivial repro program that shows the issue.

I have loaded the pty kernel module.

One thing I've found is that these calls are trying to open /dev/ptmx, whereas this device is under /dev/pts/ptmx on Linux. However, by looking at the glibc sources, it seems as if the lack of ptmx is a non-fatal error, as the code then falls back to opening /dev/ptyp0.
Comment 1 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 15:48:13 UTC
Created attachment 228889 [details]
kdump output of the repro program built for FreeBSD
Comment 2 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 15:48:33 UTC
Created attachment 228890 [details]
kdump output of the repro program built for Linux
Comment 3 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 15:50:08 UTC
I've now attached the kdump output of the sample repro program when built on FreeBSD 13 as well as when built on Debian testing with libc6-2.32-4.
Comment 4 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 15:51:54 UTC
Created attachment 228891 [details]
kdump output of the repro program built for Linux
Comment 5 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 16:07:57 UTC
Another thing that seems strange is that the Linux binary is trying to access /dev/ptyp0 on FreeBSD, but it gets a /dev/pts/? device on Linux.

By looking at the FreeBSD kernel sources, I notice that sys/compat/linux/linux_util.c has some special code to handle device paths that start with pts/, but obviously that's not going to work for the /dev/pty? devices. Maybe that's the issue?
Comment 6 Julio Merino,+1 347 694 0576,New York City freebsd_committer 2021-10-20 19:43:38 UTC
Uh... operator error!

I had done "kldload linux64" followed by a "pkg install linux_base-c7", but I had not run "service linux start", which actually mounts a few more missing file systems under /compat/linux/dev. Doing so made the repro program work just fine.