Bug 243136 - factor(6) gives inconsistent results with and without OpenSSL compiled int
Summary: factor(6) gives inconsistent results with and without OpenSSL compiled int
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
Keywords: patch
Depends on:
Reported: 2020-01-06 20:03 UTC by Steve Kargl
Modified: 2020-05-10 18:52 UTC (History)
3 users (show)

See Also:

patch to fix the bug (4.82 KB, patch)
2020-01-06 20:03 UTC, Steve Kargl
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Steve Kargl freebsd_committer 2020-01-06 20:03:53 UTC
Created attachment 210487 [details]
patch to fix the bug

See the eamil thread that starts here:


Basically, compiling with -DWITH_OPENSSL and -DWITHOUT_OPENSSL generates a binary for factor(6), which gives inconsistent results for hexadecimal input.  The patch fixes the problem.

Here's a svn log entry.

* usr.bin/factor/factor.6:
  . Update documentation to note that hexadecimal strings are accepted.
  . Document that a hexadecimal number can have an optional 0x or 0X prefix.
  . Document that a 0 value in interactive mode terminates factor(6).
  . Fix the maximum value for 'stop' in primes(6).
  . While here, spell "white-space" as "whitespace" and "non-digit" as

* usr.bin/factor/factor.c:
  . Include stdbool to get acces to bool type.
  . Use consistent style for function prototypes.
  . New function. is_hex_str() looks for the longest substring and 
    determines if it is a hexadecimal number.
  . New function.  Factor (pun intended) out common code into convert_str2bn().
  . For the WIHTOUT_OPENSSL case, make BN_dec2bn() and BN_hex2bn() return 0
    on error like their OpenSSL counterparts.

* usr.bin/primes/primes.c:
  . Fix comment.
Comment 1 Kirk McKusick freebsd_committer 2020-01-07 00:49:11 UTC
Thanks for the submission. Garance Alistair Drosehn or I will get it reviewed and committed.
Comment 2 commit-hook freebsd_committer 2020-01-12 20:25:23 UTC
A commit references this bug:

Author: gad
Date: Sun Jan 12 20:25:12 UTC 2020
New revision: 356666
URL: https://svnweb.freebsd.org/changeset/base/356666

  Fix the way 'factor' behaves when using OpenSSL to match the description
  of how it works when not compiled with OpenSSL.

  Also, allow users to specify a hexadecimal number by using a prefix of
  '0x'.  Before this, users could only specify a hexadecimal value if that
  value included a hex digit ('a'-'f') in the value.

  PR:		243136
  Submitted by:	Steve Kargl
  Reviewed by:	gad
  MFC after:	3 weeks

Comment 3 Kirk McKusick freebsd_committer 2020-01-13 01:00:33 UTC
Suggested change made. Will close after MFC.
Comment 4 Kirk McKusick freebsd_committer 2020-05-10 18:52:03 UTC
MFC'ed to 12-stable with -r360835.