Bug 142986

Summary: /dev/random can not be source for tr(1) | tr: Illegal byte sequence
Product: Base System Reporter: Slawomir Wojciech Wojtczak <vermaden>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 8.0-RELEASE   
Hardware: Any   
OS: Any   

Description Slawomir Wojciech Wojtczak 2010-01-19 18:50:05 UTC 
While I can use /dev/random for tr(1) source (password generation for example) on Linux (CentO@ 5.4 to be precise) and also on gtr(1) from coreutils port, the BSD tr(1) does not want to cooperate with /dev/random output, as below:

FreeBSD% head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
tr: Illegal byte sequence

FreeBSD% head -c 200 /dev/random| gtr -d -c 'A-Za-z0-9'
x9SMkHI4INJOwwAHhHocHElFdrVJOEMYxgtcVndR

CentOS$ head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
KMLqkAymFbFlpXPNrs5PaQ6McrHW3xzeBo9zm3aaL

How-To-Repeat: FreeBSD% head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
tr: Illegal byte sequence

FreeBSD% head -c 200 /dev/random| gtr -d -c 'A-Za-z0-9'
x9SMkHI4INJOwwAHhHocHElFdrVJOEMYxgtcVndR

CentOS$ head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
KMLqkAymFbFlpXPNrs5PaQ6McrHW3xzeBo9zm3aaL
Comment 1 Yuri Pankov 2010-01-19 20:51:00 UTC 
/dev/random just isn't guaranteed to produce correct multibyte characters
(and I guess you are using multibyte locale). Try setting LC_CTYPE to
single byte locale:
> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=
> head -c 200 /dev/urandom | tr -c -d 'A-Za-z0-9'
tr: Illegal byte sequence
> head -c 200 /dev/urandom | env LC_CTYPE=C tr -c -d 'A-Za-z0-9'
myRUBav1EHfJrD0HCSuurHRPYmqMPzWErJhHRgp6ze0YisDoA4x
Comment 2 Mark Linimon freebsd_committer freebsd_triage 2010-01-20 22:46:52 UTC 
State Changed
From-To: open->closed

From misfiled PR bin/142986: 

Date: 20 Jan 2010 20:46:31 +0100