Bug 223634 - Bugzilla notices about commits have missing Unicode characters
Summary: Bugzilla notices about commits have missing Unicode characters
Status: Closed FIXED
Alias: None
Product: Services
Classification: Unclassified
Component: Core Infrastructure (show other bugs)
Version: unspecified
Hardware: Any Any
: --- Affects Only Me
Assignee: Oleksandr Tymoshenko
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-12 19:27 UTC by Yuri Victorovich
Modified: 2018-04-02 20:13 UTC (History)
2 users (show)

See Also:


Attachments
utf-8 fix for notify_bz.sh (432 bytes, patch)
2018-02-15 07:23 UTC, Oleksandr Tymoshenko
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2017-11-12 19:27:42 UTC
For example, the ports commit r454075 has the name with a Unicode character in it: "Vinícius".

Yet, the notification in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223593 has this character replaced with '?': 'Vin?cius'.

It's a small problem, but it is very visible when such characters appear in name or message text.
Comment 1 Brad Davis freebsd_committer freebsd_triage 2017-11-13 16:19:04 UTC
Over to the team that maintains bugzilla.
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2017-11-13 17:52:23 UTC
I was thinking that it's maybe the subversion server that isn't UTF8 compliant, not bugzilla.
Comment 3 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2017-11-13 18:36:01 UTC
(In reply to Yuri Victorovich from comment #2)

No, looks like it's Bugzilla fault. mailman archive looks good: https://lists.freebsd.org/pipermail/svn-ports-all/2017-November/166036.html

And SVN mail header seems to use UTF-8 at least for src messages:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Comment 4 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2018-02-15 07:23:34 UTC
Created attachment 190638 [details]
utf-8 fix for notify_bz.sh
Comment 5 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2018-02-15 07:29:22 UTC
commit notification is sent via email by calling notify_bz.sh script in svn commit hook. This script doesn't add Content-Encoding headers so bugzilla parser assumes it is latin-1. Attached patch fixes the problem.

Assigning back to Core Infrastructure team for commit.
Comment 6 Oleksandr Tymoshenko freebsd_committer freebsd_triage 2018-03-07 20:16:16 UTC
(In reply to Oleksandr Tymoshenko from comment #5)

Ping. Any updates on this?
I can commit the change if clusteradm@ approves it.
Comment 7 commit-hook freebsd_committer freebsd_triage 2018-04-02 20:12:50 UTC
A commit references this bug:

Author: gonzo
Date: Mon Apr  2 20:12:25 UTC 2018
New revision: 331884
URL: https://svnweb.freebsd.org/changeset/base/331884

Log:
  Make bugzilla notification email UTF-8 compatible

  Add Content-Transfer-Encoding and Content-Type headers to indicate
  that email body is UTF-8. Without them Bugzilla's email parser
  defaults to latin-1 encoding and if there are UTF-8 charasters
  in commit messages they get corrupted in refrenced PR's comment text.

  PR:		223634

Changes:
  svnadmin/hooks/scripts/notify_bz.sh