Bug 211164 - shells/ksh93: Update to 93v- unofficial release 20180520
Summary: shells/ksh93: Update to 93v- unofficial release 20180520
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Cy Schubert
URL:
Keywords:
Depends on:
Blocks: 208098
  Show dependency treegraph
 
Reported: 2016-07-16 23:51 UTC by Marcin Cieślak
Modified: 2019-04-19 14:09 UTC (History)
7 users (show)

See Also:
pi: maintainer-feedback+


Attachments
Update ksh93 port to 20160716 (1.61 KB, patch)
2016-07-17 00:10 UTC, Marcin Cieślak
no flags Details | Diff
Update ksh93 port to 20160716 (3.82 KB, patch)
2017-01-23 01:27 UTC, Marcin Cieślak
no flags Details | Diff
Update ksh93 port to 20160716 (1.76 KB, patch)
2017-05-28 21:43 UTC, Marcin Cieślak
koobs: maintainer-approval-
Details | Diff
Update to ksh93 20180520 (4.25 KB, patch)
2018-05-21 01:20 UTC, Marcin Cieślak
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Marcin Cieślak 2016-07-16 23:51:18 UTC
This is a small update to my "local" release 20160716 that is basically current git beta version (2016-01-10-beta) plus updated patches to fix compilation on FreeBSD (from http://thread.gmane.org/gmane.comp.programming.tools.ast.devel/2499)

Some tests still fail:
- there are some issues with "long double" arithmetic, I suppose this is because we are faking powl() and tgammal()
- typeset -T does not seem to respect namespaces (probably an upstream issue)
- there are some issues with catching child processes

Minor issues I think:
- ksh tests have some problems detecting which locale is present (but UTF-8 seems to work for me)

I have prepared a "local" 20160716 release to indicate this is a modified git version. The download site is down anyway.
Comment 1 Marcin Cieślak 2016-07-17 00:10:45 UTC
Created attachment 172599 [details]
Update ksh93 port to 20160716

The distfiles have been uploaded to

http://marcincieslak.com/tmp/distfiles/ast-ksh.2016-07-16.tgz

http://marcincieslak.com/tmp/distfiles/INIT.2014-12-24.tgz
Comment 2 Po-Chuan Hsieh freebsd_committer freebsd_triage 2016-08-13 08:18:05 UTC
Would you like to take maintainership?
Comment 3 Marcin Cieślak 2016-08-21 01:19:22 UTC
Why not. Is there a way to upload custom distfiles to local/ if I am not a committer?
Comment 4 Po-Chuan Hsieh freebsd_committer freebsd_triage 2016-11-23 18:59:12 UTC
LOCAL is for committers. Since this new version is your local release. You might consider put them on GitHub or so. And it'll be easier for others to track your commits.

BTW, I'll pass maintainership to you first.
Thanks!
Comment 5 commit-hook freebsd_committer freebsd_triage 2016-11-23 18:59:36 UTC
A commit references this bug:

Author: sunpoet
Date: Wed Nov 23 18:59:23 UTC 2016
New revision: 426954
URL: https://svnweb.freebsd.org/changeset/ports/426954

Log:
  - Pass maintainership to submitter

  PR:		211164
  Submitted by:	Marcin Cieslak <saper@saper.info>

Changes:
  head/shells/ksh93/Makefile
Comment 6 Marcin Cieślak 2017-01-23 01:27:10 UTC
Created attachment 179231 [details]
Update ksh93 port to 20160716

Change distfiles
Comment 7 Marcin Cieślak 2017-01-23 11:00:55 UTC
The current patch unfortunately does not build as it should. Back to the drawing board.
Comment 8 Martin Wilke freebsd_committer freebsd_triage 2017-05-07 10:54:46 UTC
Do we still need this open?
Comment 9 Marcin Cieślak 2017-05-07 21:36:15 UTC
Well, yes, but it will be an update to another version
Comment 10 Marcin Cieślak 2017-05-07 21:36:37 UTC
reopening
Comment 11 Martin Wilke freebsd_committer freebsd_triage 2017-05-13 04:41:34 UTC
Any ETA?
Comment 12 Marcin Cieślak 2017-05-28 21:43:02 UTC
Created attachment 183019 [details]
Update ksh93 port to 20160716

- updated distfile path to something more long-term
- tested w/poudriere 10.3 amd64
Comment 13 Steve Wills freebsd_committer freebsd_triage 2017-07-12 19:28:20 UTC
(In reply to Marcin Cieślak from comment #12)
Current patch fails to fetch and fails to patch if you fix the fetch.
Comment 14 Marcin Cieślak 2018-05-21 01:20:15 UTC
Created attachment 193586 [details]
Update to ksh93 20180520

I have cut another "release" which solves some issues with FreeBSD 11 (ALIGN macro got undefined).

I have traced the aarch64 problem and this ends up caused by the lack of sbrk() on that platform. Actually we only need sbrk(0) to determine the address limit for the heap.

Removed 3 patches to ksh93 source.
Comment 15 Kurt Jaeger freebsd_committer freebsd_triage 2019-01-06 11:23:24 UTC
In September 2018, cy@ introduced the ksh93-devel port, which tracks github att/ast updates. Would it not be useful to base ksh93 on the att/ast version as well ?

It has seen some releases:

https://github.com/att/ast/releases
Comment 16 Kurt Jaeger freebsd_committer freebsd_triage 2019-01-06 11:30:19 UTC
testbuilds@work, testing issue from pr#208098 as well.
Comment 17 Kurt Jaeger freebsd_committer freebsd_triage 2019-01-06 11:34:25 UTC
ksh93 with this patch no longer dumps core (see PR#208098).

Fetching in poudriere causes this problem:

=> INIT.2014-12-24.tgz doesn't seem to exist in /portdistfiles/ksh93.
=> Attempting to fetch https://distfile.net/local-ports-distfiles/INIT.2014-12-24.tgz
Certificate verification failed for /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
34370682880:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/pou/jails/cur/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1924:
fetch: https://distfile.net/local-ports-distfiles/INIT.2014-12-24.tgz: Authentication error
Comment 18 Cy Schubert freebsd_committer freebsd_triage 2019-01-06 18:04:34 UTC
(In reply to Kurt Jaeger from comment #15)
The reasons I created ksh93-devel are:

1. This shells/ksh93 hasn't been updated a long time.

2. att/ast has not released a new release in a long time and frankly their naming/numbering of releases didn't give any comfort that they took this seriously. The fact that there were four releases in the space of two days was also a little disconcerting.

3. Looking at their git log there have been a few IMO important fixes.

If people want I can update this port.

Recommended:

1. Delete this port.

2. svn copy ksh93-devel to ksh93, using the 93v release instead. However this won't guarantee a port with important fixes.

Maybe we should simply delete this port.

Additionally it is my intention (when I find the time) to put together a revision for USES=ksh: where after the : one could us ksh93, pdksh, or mksh, and each could be configured to install ${LOCALBASE}/bin/ksh.
Comment 19 Kurt Jaeger freebsd_committer freebsd_triage 2019-02-10 17:38:09 UTC
Please implement your suggestion, as you are more involved than I am. Marcin, can you give feedback to cy@ about this, as well ?
Comment 20 Cy Schubert freebsd_committer freebsd_triage 2019-02-10 18:22:39 UTC
Ok. I'll accept the PR. Maintainer feedback rules apply.

Marcin,

Do you want the port deleted or updated to 93v as recommended?
Comment 21 Marcin Cieślak 2019-02-10 20:38:20 UTC
Hello,

in this update I have collected various things that made ksh93 working on FreeBSD but without major rework.

I am following the development on Github from time to time and I have some doubts about the quality of changes being made. Sometimes the authors don't even bother to consult the KornShell book to understand shell's behaviour. 

There is also some talk about ditching 93v updates and going back to some earlier version.

My preference would be to keep this port to ensure ksh93 compatibility and follow the GitHub development as ksh93-devel, as it is now.

I think I might be submitting some improvements to this port in the future especially FreeBSD-related fixes if needed.
Comment 22 Marcin Cieślak 2019-02-10 20:39:40 UTC
Regarding USES=ksh - there are significant differences between those shells. I wonder if there most of the scripts will work portably with for example mksh and ksh93...
Comment 23 Cy Schubert freebsd_committer freebsd_triage 2019-02-10 20:50:31 UTC
(In reply to Marcin Cieślak from comment #21)
Let's close this PR and you submit a new PR with new updates or fixes. IS that ok?

If you want to base ksh93 on a new tarball, please use github instead of rolling your own distribution. If you want to roll your own distribution it should be ksh93-mc or something like it. It's not an official distribution.
Comment 24 Cy Schubert freebsd_committer freebsd_triage 2019-02-10 20:55:34 UTC
(In reply to Marcin Cieślak from comment #22)
They're similar enough. Additionally pdksh, the current default ksh, is not ksh. ksh93 is. By rights it (or ksh93-devel) should be the default. People should be able to specify USES=ksh:ksh93-devel or ksh:pdksh. The intent would be to default to ksh:pdksh as it is today, or if a ksh is already installed use it instead (like openssl does), and after a period of time change the default to ksh:ksh93.

I'm not quite there yet as I have more than enough on my plate to keep me busy before I even think about tackling this project.
Comment 25 Marcin Cieślak 2019-02-10 21:08:18 UTC
Fair enough, I can probably roll the patches into local patches in the port.

My original motivation to take this port was to work on it as a dependency of x11-wm/cde that provides ksh library to build dtksh.

Let's close this one, thanks!
Comment 26 Cy Schubert freebsd_committer freebsd_triage 2019-02-10 21:28:46 UTC
(In reply to Marcin Cieślak from comment #25)
That's great! When you're working on cde, can you make selecting one of the ksh's an option? If you want to see my USES=ksh idea sooner I'll try to start working on it.

Also, do you run 13-CURRENT or 12? I haven't found the time to get it to build here yet but if you have the cycles that would be appreciated. I used to use cde about 20 years ago on a DEC Alpha running Tru64 and would love to go back. (I've ditched gnome and mate years ago in favour of simply running mwm for now until I find the cycles to look at cde.)
Comment 27 Marcin Cieślak 2019-02-10 21:38:08 UTC
I don't think that USES=ksh is an option because none of the other ksh's provide ksh as a shared library (like tcl). To be fair, the current ksh93 does not do that either but it could :)

CDE is pretty usable and very fast on current machines. UTF-8 does not (fully) work but that's bearable. 

ksh still needs brk() style memory allocation - that one got removed from aarch64 somet time ago - but it should work on other platforms?

As soon as I overcome all the issues from upgrading from 10.x I am sure I'll get to 12 and 13 as well :)
Comment 28 Cy Schubert freebsd_committer freebsd_triage 2019-02-10 21:55:06 UTC
How about we take this CDE thing offline. I'll contact you privately.
Comment 29 Cy Schubert freebsd_committer freebsd_triage 2019-02-21 06:34:52 UTC
Reopened. Will fetch from GH.
Comment 30 Cy Schubert freebsd_committer freebsd_triage 2019-02-24 07:08:18 UTC
The patch doesn't work. Suggest we delete the port and use ksh93-devel or wait for att to tag on github.
Comment 31 Siteshwar Vashisht 2019-04-19 13:13:49 UTC
(In reply to Cy Schubert from comment #18)

> 2. att/ast has not released a new release in a long time and frankly their naming/numbering of releases didn't give any comfort that they took this seriously. The fact that there were four releases in the space of two days was also a little disconcerting.

We did not choose the naming/numbering scheme for previous versions, previous maintainers were using two different schemes for version number that has been made obsolete now. We have switched to semantic versioning and it's a good thing. We did not even make a release until this week, we had only tagged relevant commits with version numbers.
Comment 32 Siteshwar Vashisht 2019-04-19 13:26:59 UTC
(In reply to Marcin Cieślak from comment #21)

> I am following the development on Github from time to time and I have some doubts about the quality of changes being made. Sometimes the authors don't even bother to consult the KornShell book to understand shell's behaviour. 

Those changes have been discussed, but actually never implemented. In fact, it's quite the opposite, we sometimes reject patches if we suspect they may break backward compatibility.

> There is also some talk about ditching 93v updates and going back to some earlier version.

Nope, we already made the release based on ksh93v- and we are not going back to ksh93u+.

> My preference would be to keep this port to ensure ksh93 compatibility and follow the GitHub development as ksh93-devel, as it is now.

I appreciate your hard work to keep ksh93v- well maintained in FreeBSD. As someone who maintains ksh in RHEL/Fedora, I can attest it's very hard to deal with it. But at the same time, our work in upstream is meant to make life of people like you easier, so that it's easy to debug and fix issues. Instead of resisting our efforts to bring this codebase to 21st century, please help us in getting wider userbase for testing. You should rebase to ksh-2020.0.0-alpha1 (the release that we made this week) in FreeBSD.
Comment 33 Cy Schubert freebsd_committer freebsd_triage 2019-04-19 14:09:13 UTC
Thanks. I'll commit an pdate over the weekend when I find the time.