Bug 209369 - chinese/sunpinyin: Fix build with libc++ 3.8.0
Summary: chinese/sunpinyin: Fix build with libc++ 3.8.0
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs mailing list
URL:
Keywords:
Depends on:
Blocks: 208158
  Show dependency treegraph
 
Reported: 2016-05-07 20:27 UTC by Dimitry Andric
Modified: 2016-05-09 18:45 UTC (History)
1 user (show)

See Also:
lichray: maintainer-feedback+


Attachments
Explicitly call float versions of log, exp, etc. (2.44 KB, patch)
2016-05-07 20:27 UTC, Dimitry Andric
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitry Andric freebsd_committer 2016-05-07 20:27:16 UTC
Created attachment 170103 [details]
Explicitly call float versions of log, exp, etc.

During the exp-run in bug 208158, it was found that chinese/sunpinyin gives errors with libc++ 3.8.0 [1]:

gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work'
slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm
Loading lexicon...done.
Loading ARPA slm...
Writing out...done!
slmthread lm_sc.3gm lm_sc.t3g.orig
Loading original slm...
first pass...
Compressing pr values...65536 float values ==> 65536 values
Compressing bow values...16384 float values ==> 16384 values
Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364.
/wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed

This is because the code mostly uses floats, but in some parts it uses log(), exp(), etc, which return doubles.  During the parts where it does lookups in std::map<float, int> constructs, this leads to it not being able to find the expected entries.  Fix this by using logf(), expf() and similar, which explicitly return floats.

[1] http://package18.nyi.freebsd.org/data/headamd64PR208158-default/2016-05-01_10h29m48s/logs/errors/zh-sunpinyin-2.0.4.r3.log
Comment 1 lichray 2016-05-07 21:42:32 UTC
Upstream introduced tolerance to fix this issue, but this patch is also correct.  I will report to upstream.
Comment 2 lichray 2016-05-07 21:48:24 UTC
Forgot to mention, declare

#include <cmath>

using std::log;
using std::exp;

Will be an alternative fix, because C++ overloaded them.
Comment 3 Dimitry Andric freebsd_committer 2016-05-07 22:15:47 UTC
(In reply to lichray from comment #2)
> Forgot to mention, declare
> 
> #include <cmath>
> 
> using std::log;
> using std::exp;
> 
> Will be an alternative fix, because C++ overloaded them.

Hm, I've tried this, but it still fails. Possibly because of the macro usages like the following:

   364              std::map<float, int>::iterator prit = pr_map.find(pn->pr);
   365              if (prit == pr_map.end()) { // This would be cause by precision error
   366                  double val = EffectivePr(pn->pr);
   367                  val = OriginalPr(val);
   368                  prit = pr_map.find(val);
   369                  assert(prit != pr_map.end());
   370              }

Because OriginalPr() is called with a double argument, the double versions of log() and exp2() are chosen.  Maybe it would be better to define the macros as inline functions.
Comment 4 lichray 2016-05-07 22:21:21 UTC
Yea, pre-auto code.  Anyway, please just add the patch you attached to unblock libc++.
Comment 5 Dimitry Andric freebsd_committer 2016-05-07 22:34:56 UTC
FWIW, this worked too:

--- src/slm/thread/slmthread.cpp.orig   2014-11-14 14:17:48 UTC
+++ src/slm/thread/slmthread.cpp
@@ -226,6 +226,30 @@ FILE* fp = NULL;
 CThreadSlm::TNode* levels[16];
 CThreadSlm::TLeaf* lastLevel;

+static inline float
+effectivePr(bool usingLogPr, float a)
+{
+    return usingLogPr ? a / log(2.0f) : -log2(a);
+}
+
+static inline float
+originalPr(bool usingLogPr, float b)
+{
+    return usingLogPr ? b * log(2.0f) : exp2(-b);
+}
+
+static inline float
+effectiveBow(bool usingLogPr, float a)
+{
+    return usingLogPr ? exp(-a) : a;
+}
+
+static inline float
+originalBow(bool usingLogPr, float b)
+{
+    return usingLogPr ? -log(b) : b;
+}
+
 int
 main(int argc, char* argv[])
 {
@@ -250,10 +274,10 @@ main(int argc, char* argv[])

     bool usingLogPr = slm.isUseLogPr();

-    #define EffectivePr(a)  (float((usingLogPr) ? ((a) / log(2.0)) : (-log2((a)))))
-    #define OriginalPr(b)   (float((usingLogPr) ? ((b) * log(2.0)) : (exp2(-(b)))))
-    #define EffectiveBow(a) (float((usingLogPr) ? (exp(-(a))) : ((a))))
-    #define OriginalBow(b)  (float((usingLogPr) ? (-log((b))) : ((b))))
+    #define EffectivePr(a)  effectivePr(usingLogPr, (a))
+    #define OriginalPr(b)   originalPr(usingLogPr, (b))
+    #define EffectiveBow(a) effectiveBow(usingLogPr, (a))
+    #define OriginalBow(b)  originalBow(usingLogPr, (b))

     printf("\nfirst pass..."); fflush(stdout);
     for (int lvl = 0; lvl <= slm.getN(); ++lvl) {
Comment 6 commit-hook freebsd_committer 2016-05-07 22:44:49 UTC
A commit references this bug:

Author: dim
Date: Sat May  7 22:44:31 UTC 2016
New revision: 414787
URL: https://svnweb.freebsd.org/changeset/ports/414787

Log:
  During the exp-run in bug 208158, it was found that chinese/sunpinyin
  gives errors with libc++ 3.8.0:

  gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work'
  slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm
  Loading lexicon...done.
  Loading ARPA slm...
  Writing out...done!
  slmthread lm_sc.3gm lm_sc.t3g.orig
  Loading original slm...
  first pass...
  Compressing pr values...65536 float values ==> 65536 values
  Compressing bow values...16384 float values ==> 16384 values
  Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364.
  /wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed

  This is because the code mostly uses floats, but in some parts it uses
  log(), exp(), etc, which return doubles.  During the parts where it does
  lookups in std::map<float, int> constructs, this leads to it not being
  able to find the expected entries.  Fix this by using logf(), expf() and
  similar, which explicitly return floats.

  Approved by:	lichray@gmail.com (maintainer)
  PR:		209369
  MFH:		2016Q2

Changes:
  head/chinese/sunpinyin/Makefile
  head/chinese/sunpinyin/files/patch-src_slm_thread_slmthread.cpp
Comment 7 commit-hook freebsd_committer 2016-05-09 18:45:43 UTC
A commit references this bug:

Author: dim
Date: Mon May  9 18:44:48 UTC 2016
New revision: 414881
URL: https://svnweb.freebsd.org/changeset/ports/414881

Log:
  MFH: r414787

  During the exp-run in bug 208158, it was found that chinese/sunpinyin
  gives errors with libc++ 3.8.0:

  gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work'
  slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm
  Loading lexicon...done.
  Loading ARPA slm...
  Writing out...done!
  slmthread lm_sc.3gm lm_sc.t3g.orig
  Loading original slm...
  first pass...
  Compressing pr values...65536 float values ==> 65536 values
  Compressing bow values...16384 float values ==> 16384 values
  Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364.
  /wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed

  This is because the code mostly uses floats, but in some parts it uses
  log(), exp(), etc, which return doubles.  During the parts where it does
  lookups in std::map<float, int> constructs, this leads to it not being
  able to find the expected entries.  Fix this by using logf(), expf() and
  similar, which explicitly return floats.

  Approved by:	portmgr (junovitch)
  PR:		209369

Changes:
_U  branches/2016Q2/
  branches/2016Q2/chinese/sunpinyin/Makefile
  branches/2016Q2/chinese/sunpinyin/files/patch-src_slm_thread_slmthread.cpp