Bug 230068 - ps: processes with non-ASCII characters are hidden from list in "ps auxww"
Summary: ps: processes with non-ASCII characters are hidden from list in "ps auxww"
Status: Closed DUPLICATE of bug 241491
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-07-26 18:47 UTC by Robert
Modified: 2020-06-07 08:01 UTC (History)
2 users (show)

See Also:


Attachments
File contains example scripts and output from test-run. (1.64 KB, text/plain)
2018-07-26 18:47 UTC, Robert
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robert 2018-07-26 18:47:38 UTC
Created attachment 195484 [details]
File contains example scripts and output from test-run.

A process with non-ASCII character in its name does not show up in the listing by 'ps auxww' (or other variants).
At best, it confuses the one who started the process.
At worst, unauthorized processes can act on a system without showing up in the process list.

For example, start a processes named 'GuterProzess' and 'BöserProzess'.
In ps only 'GuterProzess' will appear.
(BTW, e.g. lsof lists the other process too.)
Comment 1 Conrad Meyer freebsd_committer freebsd_triage 2018-07-26 20:48:31 UTC
Have you configured LANG, LC_ALL, or LC_CTYPE?  I don't see any such problem (at least on CURRENT).

$ env|grep LANG
LANG=en_US.UTF-8
$ ./tëst.sh&
[1] 33360
$ ps auxwwwww|grep st.sh
conrad 33360    0.2  0.0  11820  3192  4  S    13:47        0:00.00 /bin/sh ./tëst.sh
Comment 2 Robert 2018-07-27 08:15:42 UTC
My reply for the records:
Hi,

I didn't expect anything from this side, since ls workes fine.
23:16 gluon: admin/src/FreeBSD-Probleme/ps-completeness 0% ls -l
total 923
-rwxrwxr-x  1 admin  work      62 26 Juli 20:23 BöserProzess.sh
-rwxrwxr-x  1 admin  work      64 26 Juli 20:22 GuterProzess.sh
-rw-rw-r--  1 admin  work  881577 26 Juli 23:04 lsof.lst
-rw-rw-r--  1 admin  work    1681 26 Juli 20:27 result.txt
-rwxrwxr-x  1 admin  work     164 26 Juli 20:22 StartProzesse.sh

But now I checked to be sure.
First, I had only LANG set to de_DE.ISO8859-15

Next, I tried
LANG=de_DE.ISO8859-15
LC_ALL=de_DE.ISO8859-15
LC_CTYPE=de_DE.ISO8859-15

Then reverted to
LANG=de_DE.ISO8859-1
LC_ALL=de_DE.ISO8859-1
LC_CTYPE=de_DE.ISO8859-1

and finally used
LANG=de_DE.UTF-8
LC_ALL=de_DE.UTF-8
LC_CTYPE=de_DE.UTF-8

All with the same result.
(With the last setting, in lsof the umlaut showed up as B\xf6serProzess.sh as I did not re-create the file.)

So it seems, it got fixed in CURRENT.
I will upgrade in September to 11.2.
I hope, it made it in there too.

Regards,
Robert
Comment 3 Conrad Meyer freebsd_committer freebsd_triage 2018-07-27 17:07:18 UTC
Hi Robert,

I don't think it's actually fixed in CURRENT.  I just happen to be using UTF-8, so my on-disk names are UTF-8 as well.

I think the problem is something like this:

1. ps(1) now uses libxo for formatting its output.

2. libxo is pretty dumb and seems to expect all input in UTF-8.  (It can maybe format non-UTF8 output if LANG/LC_FOO is set, but still expects programs to give it UTF8 input, if I'm skimming it correctly.)

3. ps(1) is also fairly encoding-naive and just passes through the bytes it gets from disk to libxo.

In my case, that works because the on-disk bytes are UTF-8.

I can reproduce the same issue you describe when the file is named "BöserProzess" in ISO8859-15 (with appropriate LANG set), rather than in UTF-8.
Comment 4 Yuri Pankov freebsd_committer freebsd_triage 2020-06-07 08:01:45 UTC
Will be fixed in bug 241491 (as it was the first I noticed and opened review for). Thank you for the report!

*** This bug has been marked as a duplicate of bug 241491 ***