Bug 195281 - [exp-run] Use Postgresql 9.3 by default
Summary: [exp-run] Use Postgresql 9.3 by default
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: pgsql
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-11-22 11:09 UTC by ftigeot
Modified: 2014-12-08 10:00 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ftigeot 2014-11-22 11:09:57 UTC
Could I get an exp-run with

DEFAULT_VERSIONS=pgsql=9.3

in order to fix bug #187286 ?
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2014-11-22 11:41:57 UTC
assign
Comment 2 Antoine Brodin freebsd_committer freebsd_triage 2014-11-22 18:36:21 UTC
Take for exp-run
Comment 3 Antoine Brodin freebsd_committer freebsd_triage 2014-11-22 23:03:32 UTC
Those 2 ports are the same with pgsql=93:

[00:01:04] ====>> Error: Duplicated origin for postgresql93-plv8js-1.4.2: databases/postgresql93-plv8js AND databases/postgresql-plv8js.

I will comment out one of the two.
Comment 4 Antoine Brodin freebsd_committer freebsd_triage 2014-11-24 07:02:14 UTC
New failures:

+ {"origin"=>"databases/pg_rman", "pkgname"=>"pg_rman-1.2.3_1", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"databases/pg_statsinfo", "pkgname"=>"pg_statsinfo-2.4.1", "phase"=>"build", "errortype"=>"gcc4_error"}
+ {"origin"=>"databases/pgpool-II", "pkgname"=>"pgpool-II-3.1.6_1", "phase"=>"build", "errortype"=>"compiler_error"}
+ {"origin"=>"databases/pgpool-II-30", "pkgname"=>"pgpool-II30-3.0.4_1", "phase"=>"build", "errortype"=>"compiler_error"}
+ {"origin"=>"devel/prepstools", "pkgname"=>"prepstools-2.2.0_2", "phase"=>"build", "errortype"=>"clang"}

Failure logs:

http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-23_13h38m12s/logs/errors/pg_rman-1.2.3_1.log
http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-23_13h38m12s/logs/errors/pg_statsinfo-2.4.1.log
http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-23_13h38m12s/logs/errors/pgpool-II-3.1.6_1.log
http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-23_13h38m12s/logs/errors/pgpool-II30-3.0.4_1.log
http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-23_13h38m12s/logs/errors/prepstools-2.2.0_2.log

Ports skipped due to those failures:

databases/pgpoolAdmin
devel/prepstools
Comment 5 John Marino freebsd_committer freebsd_triage 2014-11-24 07:20:39 UTC
(In reply to Antoine Brodin from comment #4)
> New failures:
> 
> + {"origin"=>"databases/pg_rman", "pkgname"=>"pg_rman-1.2.3_1",
> "phase"=>"build", "errortype"=>"clang"}
> Failure logs:
> 
> http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-
> 23_13h38m12s/logs/errors/pg_rman-1.2.3_1.log


pg_rman is 4 releases behind (1.23 vs 1.27).  I've added the maintainer to CC since the mostly fix is to update the software.
Comment 6 John Marino freebsd_committer freebsd_triage 2014-11-24 07:23:42 UTC
(In reply to Antoine Brodin from comment #4)
> New failures:
> 
> + {"origin"=>"databases/pg_statsinfo", "pkgname"=>"pg_statsinfo-2.4.1",
> "phase"=>"build", "errortype"=>"gcc4_error"}
>
> http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-
> 23_13h38m12s/logs/errors/pg_statsinfo-2.4.1.log


Kuriyama also maintains pg_statsinfo and it's also not up to date.  ON the upstream site:

2013-10-25 : pg_statsinfo-2.5.0 is released. It can work in PG 9.3 and add more useful feature! Smaple report file is here.

so this port just needs updating.
Comment 7 John Marino freebsd_committer freebsd_triage 2014-11-24 07:28:43 UTC
(In reply to Antoine Brodin from comment #4)
> New failures:
> 
> + {"origin"=>"databases/pgpool-II", "pkgname"=>"pgpool-II-3.1.6_1",
> "phase"=>"build", "errortype"=>"compiler_error"}
> + {"origin"=>"databases/pgpool-II-30", "pkgname"=>"pgpool-II30-3.0.4_1",
> "phase"=>"build", "errortype"=>"compiler_error"}
> 
> Failure logs:
> 
> http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-
> 23_13h38m12s/logs/errors/pgpool-II-3.1.6_1.log
> http://package18.nyi.freebsd.org/data/101amd64-default-PR195281/2014-11-
> 23_13h38m12s/logs/errors/pgpool-II30-3.0.4_1.log

The pattern continues.
Kuriyama maintains both pgpool-II ports.
The first is well out of date (3.1 vs 3.4).  On the upstream site, it says 3.4 is compatible with pgsql 9.4

The II-30 port seems to be a legacy port (3.0 only) which means it needs to be marked as not compatible with pgsql 93+.  I'd like to know why there are two versions of pgpool out of curiousity.
Comment 8 John Marino freebsd_committer freebsd_triage 2014-11-24 07:31:31 UTC
the final one, devel/prepstools, is not maintained.  The upstream is gone!

At least, it needs to be marked as incompatible with 93+, at best mark it deprecated for deletion.
Comment 9 Palle Girgensohn freebsd_committer freebsd_triage 2014-11-24 08:53:51 UTC
Just for reference, a thread about the performance degradation in PostgreSQL 9.3+ on FreeBSD:

[http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-CE2258230C05@FreeBSD.org]

In short, in the FreeBSD kernel, the SYSV shared memory implementation is more efficient than the POSTIX shared memory one.

I tried to nag som kernel-knowledgable people about it, but I don't know exactly what the result was.

I have a patch towards PostgreSQL, reimplementing SYSV shared memory as an opntional feature. I've planned to test that just to see if it helps.

If that works, I'd like it an optional feature for port builders.

I agree, let the FreeBSD kernel folks worry about the performance degradations. It seems to be about 20-25%, which is actually not noticeable to the end user (that would require doubled time according to cognitive studies :-)

I'll try the patch. How far is the exp-run, does it work now?
Comment 10 Palle Girgensohn freebsd_committer freebsd_triage 2014-11-24 08:54:21 UTC
And perhaps we can merge / mark as duplicate of 187586?
Comment 11 John Marino freebsd_committer freebsd_triage 2014-11-24 08:59:10 UTC
(In reply to Palle Girgensohn from comment #9)
> Just for reference, a thread about the performance degradation in PostgreSQL
> 9.3+ on FreeBSD:
> 
> [http://www.postgresql.org/message-id/2AE143D2-87D3-4AD1-AC78-
> CE2258230C05@FreeBSD.org]
> 
> In short, in the FreeBSD kernel, the SYSV shared memory implementation is
> more efficient than the POSTIX shared memory one.


Hi Paul, I think this is pretty well known.
pgsql 9.2 isn't going away, those concerned about performance can just stick with that version.  We're only taking about changing the default.  I've been told the pgsql developers are none too impressed that freebsd is defaulting to 9.2 when 9.4 is around the corner.



> I have a patch towards PostgreSQL, reimplementing SYSV shared memory as an
> opntional feature. I've planned to test that just to see if it helps.
> 
> If that works, I'd like it an optional feature for port builders.

Well, I assume a patch like that would have to be heavily vetted -- best to get it into upstream.  It's also out of scope of this PR.


> I'll try the patch. How far is the exp-run, does it work now?

My interpretation is the run is finished with 5 failures.  Final result may be 3 upgrades, 1 exclusion, and one deprecation.  Just a guess though.
Comment 12 ftigeot 2014-11-24 09:23:17 UTC
I don't think it's a good idea to continue mentioning SYSV/mmap performance differences. The original benchmarks I did are now two year old and much work has been done to improve the situation.

Without some benchmarking of recent FreeBSD versions, we can't know if there still is a SYSV shared memory vs mmap() performance gap.
Comment 13 Palle Girgensohn freebsd_committer freebsd_triage 2014-11-24 12:46:16 UTC
My (In reply to ftigeot from comment #12)
> I don't think it's a good idea to continue mentioning SYSV/mmap performance
> differences. The original benchmarks I did are now two year old and much
> work has been done to improve the situation.
> 
> Without some benchmarking of recent FreeBSD versions, we can't know if there
> still is a SYSV shared memory vs mmap() performance gap.

My benchmarks are from April 2014.

I'm planning to do some more benchmarks on FreeBSD-10.1.

Palle
Comment 14 commit-hook freebsd_committer freebsd_triage 2014-11-24 14:54:25 UTC
A commit references this bug:

Author: marino
Date: Mon Nov 24 14:54:07 UTC 2014
New revision: 373216
URL: https://svnweb.freebsd.org/changeset/ports/373216

Log:
  devel/prepstools: Mark broken and for removal in 3 weeks

  The upstream is gone, public distfiles are unavailable.  It also does
  not build with pgsql 9.3.

  PR:	195281

Changes:
  head/devel/prepstools/Makefile
Comment 15 John Marino freebsd_committer freebsd_triage 2014-11-24 15:17:00 UTC
FYI there are 5 pgpool-II ports: 
 - databases/pgpool-II 
 - databases/pgpool-II-22
 - databases/pgpool-II-23 
 - databases/pgpool-II-30 
 - databases/pgpool-II-33

Apparently only one of them doesn't build with pgsql 9.3 ?

Do we really need 5 versions of this port?  I was going to mark -30 as not pgsql 9.3-ready but I'm now not going to do anything and just wait for kuriyama to weigh in.
Comment 16 John Marino freebsd_committer freebsd_triage 2014-12-02 08:58:19 UTC
Hi Kuriyama,
The four ports blocking the move to pgsql 9.3 are maintained by you.  There are two approaches we can take:

1) update the ports if that gives 9.3 compatibility
2) Mark the ports as good for only 9.2 and below

Obviously option 1 is better, but if it's going to be a while, we need to go with option 2.  Can you indicate what your plans are so a decision can be made?

Thanks,
John
Comment 17 commit-hook freebsd_committer freebsd_triage 2014-12-07 09:43:40 UTC
A commit references this bug:

Author: marino
Date: Sun Dec  7 09:43:12 UTC 2014
New revision: 374179
URL: https://svnweb.freebsd.org/changeset/ports/374179

Log:
  Limit pgsql to 9.2 on four databases category ports

  As the result of a recent exp-run for postgresql 9.3, several ports
  failed to build as a result.  Most could likely be fixed by updating the
  port to a later available version.  Until that happens, set the maximum
  version of pgsql to 9.2 for these ports.

    * pg_rman (1.23)       : version 1.27 is available
    * pg_statsinfo (2.4.1) : version 2.5.0 available, works on pgsql 9.3
    * pgpool-II (3.1.6)    : version 3.4 available, works on pgsql 9.3
    * pgpool-II30 (3.0.4)

  PR:	195281

Changes:
  head/databases/pg_rman/Makefile
  head/databases/pg_statsinfo/Makefile
  head/databases/pgpool-II/Makefile
  head/databases/pgpool-II-30/Makefile
Comment 18 John Marino freebsd_committer freebsd_triage 2014-12-07 10:09:25 UTC
(In reply to Antoine Brodin from comment #3)
> Those 2 ports are the same with pgsql=93:
> 
> [00:01:04] ====>> Error: Duplicated origin for postgresql93-plv8js-1.4.2:
> databases/postgresql93-plv8js AND databases/postgresql-plv8js.
> 
> I will comment out one of the two.

I am going to remove postgresql93-plv8js port.
There's no reason for it to exist.  postgresql-plv8js will use pgsql 9.3 if PGSQL_DEFAULT is set to it and WANT_PGSQL through the command line could affect it as well.  No port depends on postgresql-plv8js.
Comment 19 commit-hook freebsd_committer freebsd_triage 2014-12-07 10:35:46 UTC
A commit references this bug:

Author: marino
Date: Sun Dec  7 10:35:03 UTC 2014
New revision: 374185
URL: https://svnweb.freebsd.org/changeset/ports/374185

Log:
  Remove port databases/postgresql93-plv8js (avoids broken index)

  This port was added on 5 October 2014.  The intention for its existence
  was to provide a way to use plv8js with pgsql 9.3 instead of the default
  pgsql 9.2.  It was implemented in such a way that if PGSQL_DEFAULT is
  set to 9.3, the index breaks with a duplicate origin with datbases/
  postgresql-plv8js.  It's possible to adjust the plv8js ports by converting
  the version into an option and using typical master/slave techniques, but
  I can't come up with a good reason to do this at all.

  I don't think this port ever should have been created.  Anyone that would
  need this port would have needed to set PGSQL_DEFAULT anyway (which
  already works).  In the worst case, WANT_PGSQL could be based through a
  command line.  Perhaps the motivation was to have a binary package to
  avoid building it, but this reason disappears soon when the default
  version of pgsql is bumped to 9.3.  Based on all those reasons, I think
  it is better to remove the port outright (pointing to master port) rather
  than adjust it to avoid a broken index.

  PR:	195281

Changes:
  head/MOVED
  head/databases/Makefile
  head/databases/postgresql-plv8js/Makefile
  head/databases/postgresql93-plv8js/
Comment 20 John Marino freebsd_committer freebsd_triage 2014-12-08 10:00:28 UTC
PostgreSQL has been moved to version 9.3 by default, so I'm closing the related exp-run PR.