Created attachment 170103 [details] Explicitly call float versions of log, exp, etc. During the exp-run in bug 208158, it was found that chinese/sunpinyin gives errors with libc++ 3.8.0 [1]: gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work' slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm Loading lexicon...done. Loading ARPA slm... Writing out...done! slmthread lm_sc.3gm lm_sc.t3g.orig Loading original slm... first pass... Compressing pr values...65536 float values ==> 65536 values Compressing bow values...16384 float values ==> 16384 values Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364. /wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed This is because the code mostly uses floats, but in some parts it uses log(), exp(), etc, which return doubles. During the parts where it does lookups in std::map<float, int> constructs, this leads to it not being able to find the expected entries. Fix this by using logf(), expf() and similar, which explicitly return floats. [1] http://package18.nyi.freebsd.org/data/headamd64PR208158-default/2016-05-01_10h29m48s/logs/errors/zh-sunpinyin-2.0.4.r3.log
Upstream introduced tolerance to fix this issue, but this patch is also correct. I will report to upstream.
Forgot to mention, declare #include <cmath> using std::log; using std::exp; Will be an alternative fix, because C++ overloaded them.
(In reply to lichray from comment #2) > Forgot to mention, declare > > #include <cmath> > > using std::log; > using std::exp; > > Will be an alternative fix, because C++ overloaded them. Hm, I've tried this, but it still fails. Possibly because of the macro usages like the following: 364 std::map<float, int>::iterator prit = pr_map.find(pn->pr); 365 if (prit == pr_map.end()) { // This would be cause by precision error 366 double val = EffectivePr(pn->pr); 367 val = OriginalPr(val); 368 prit = pr_map.find(val); 369 assert(prit != pr_map.end()); 370 } Because OriginalPr() is called with a double argument, the double versions of log() and exp2() are chosen. Maybe it would be better to define the macros as inline functions.
Yea, pre-auto code. Anyway, please just add the patch you attached to unblock libc++.
FWIW, this worked too: --- src/slm/thread/slmthread.cpp.orig 2014-11-14 14:17:48 UTC +++ src/slm/thread/slmthread.cpp @@ -226,6 +226,30 @@ FILE* fp = NULL; CThreadSlm::TNode* levels[16]; CThreadSlm::TLeaf* lastLevel; +static inline float +effectivePr(bool usingLogPr, float a) +{ + return usingLogPr ? a / log(2.0f) : -log2(a); +} + +static inline float +originalPr(bool usingLogPr, float b) +{ + return usingLogPr ? b * log(2.0f) : exp2(-b); +} + +static inline float +effectiveBow(bool usingLogPr, float a) +{ + return usingLogPr ? exp(-a) : a; +} + +static inline float +originalBow(bool usingLogPr, float b) +{ + return usingLogPr ? -log(b) : b; +} + int main(int argc, char* argv[]) { @@ -250,10 +274,10 @@ main(int argc, char* argv[]) bool usingLogPr = slm.isUseLogPr(); - #define EffectivePr(a) (float((usingLogPr) ? ((a) / log(2.0)) : (-log2((a))))) - #define OriginalPr(b) (float((usingLogPr) ? ((b) * log(2.0)) : (exp2(-(b))))) - #define EffectiveBow(a) (float((usingLogPr) ? (exp(-(a))) : ((a)))) - #define OriginalBow(b) (float((usingLogPr) ? (-log((b))) : ((b)))) + #define EffectivePr(a) effectivePr(usingLogPr, (a)) + #define OriginalPr(b) originalPr(usingLogPr, (b)) + #define EffectiveBow(a) effectiveBow(usingLogPr, (a)) + #define OriginalBow(b) originalBow(usingLogPr, (b)) printf("\nfirst pass..."); fflush(stdout); for (int lvl = 0; lvl <= slm.getN(); ++lvl) {
A commit references this bug: Author: dim Date: Sat May 7 22:44:31 UTC 2016 New revision: 414787 URL: https://svnweb.freebsd.org/changeset/ports/414787 Log: During the exp-run in bug 208158, it was found that chinese/sunpinyin gives errors with libc++ 3.8.0: gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work' slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm Loading lexicon...done. Loading ARPA slm... Writing out...done! slmthread lm_sc.3gm lm_sc.t3g.orig Loading original slm... first pass... Compressing pr values...65536 float values ==> 65536 values Compressing bow values...16384 float values ==> 16384 values Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364. /wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed This is because the code mostly uses floats, but in some parts it uses log(), exp(), etc, which return doubles. During the parts where it does lookups in std::map<float, int> constructs, this leads to it not being able to find the expected entries. Fix this by using logf(), expf() and similar, which explicitly return floats. Approved by: lichray@gmail.com (maintainer) PR: 209369 MFH: 2016Q2 Changes: head/chinese/sunpinyin/Makefile head/chinese/sunpinyin/files/patch-src_slm_thread_slmthread.cpp
A commit references this bug: Author: dim Date: Mon May 9 18:44:48 UTC 2016 New revision: 414881 URL: https://svnweb.freebsd.org/changeset/ports/414881 Log: MFH: r414787 During the exp-run in bug 208158, it was found that chinese/sunpinyin gives errors with libc++ 3.8.0: gmake[2]: Entering directory '/wrkdirs/usr/ports/chinese/sunpinyin/work' slmpack lm_sc.3gm.arpa dict.utf8 lm_sc.3gm Loading lexicon...done. Loading ARPA slm... Writing out...done! slmthread lm_sc.3gm lm_sc.t3g.orig Loading original slm... first pass... Compressing pr values...65536 float values ==> 65536 values Compressing bow values...16384 float values ==> 16384 values Threading the new model...Assertion failed: (prit != pr_map.end()), function main, file src/slm/thread/slmthread.cpp, line 364. /wrkdirs/usr/ports/chinese/sunpinyin/work/sunpinyin-a8bd811/src/sunpinyin-dictgen.mk:51: recipe for target 'lm_sc.t3g.orig' failed This is because the code mostly uses floats, but in some parts it uses log(), exp(), etc, which return doubles. During the parts where it does lookups in std::map<float, int> constructs, this leads to it not being able to find the expected entries. Fix this by using logf(), expf() and similar, which explicitly return floats. Approved by: portmgr (junovitch) PR: 209369 Changes: _U branches/2016Q2/ branches/2016Q2/chinese/sunpinyin/Makefile branches/2016Q2/chinese/sunpinyin/files/patch-src_slm_thread_slmthread.cpp