Bug 269798 - databases/sqlite3: Update to 3.41.0
Summary: databases/sqlite3: Update to 3.41.0
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Robert Clausecker
URL: https://sqlite.org/releaselog/3_41_0....
Keywords:
Depends on:
Blocks:
 
Reported: 2023-02-24 07:04 UTC by Pavel Volkov
Modified: 2023-03-13 17:08 UTC (History)
6 users (show)

See Also:


Attachments
patch (6.71 KB, patch)
2023-02-24 07:04 UTC, Pavel Volkov
pavelivolkov: maintainer-approval+
Details | Diff
portlint log (12 bytes, text/plain)
2023-02-24 07:05 UTC, Pavel Volkov
no flags Details
poudriere log without flavors (59.10 KB, text/plain)
2023-02-24 07:06 UTC, Pavel Volkov
no flags Details
poudriere log @tcl (71.69 KB, text/plain)
2023-02-24 07:06 UTC, Pavel Volkov
no flags Details
poudriere log @icu (63.18 KB, text/plain)
2023-02-24 07:07 UTC, Pavel Volkov
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pavel Volkov 2023-02-24 07:04:22 UTC
Created attachment 240362 [details]
patch

This is a new version sqlite3 3.41
Changed default behavior for option DQS (https://sqlite.org/quirks.html#dblquote).
Comment 1 Pavel Volkov 2023-02-24 07:05:49 UTC
Created attachment 240363 [details]
portlint log
Comment 2 Pavel Volkov 2023-02-24 07:06:18 UTC
Created attachment 240364 [details]
poudriere log without flavors
Comment 3 Pavel Volkov 2023-02-24 07:06:49 UTC
Created attachment 240365 [details]
poudriere log @tcl
Comment 4 Pavel Volkov 2023-02-24 07:07:12 UTC
Created attachment 240366 [details]
poudriere log @icu
Comment 5 Florian Smeets freebsd_committer freebsd_triage 2023-02-26 11:55:09 UTC
I think disabling DQS feature by default warrants and UPDATING entry. I see that there should have been warnings for four years, but adding an UPDATING entry with a little description and how to get the old behavior back doesn't hurt.
Comment 6 Robert Clausecker freebsd_committer freebsd_triage 2023-02-26 13:21:42 UTC
Will add such a note on commit.
Comment 7 commit-hook freebsd_committer freebsd_triage 2023-02-27 22:50:20 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=47912ce2e613211aecf6ecdfc58681b7aca0df33

commit 47912ce2e613211aecf6ecdfc58681b7aca0df33
Author:     Pavel Volkov <pavelivolkov@gmail.com>
AuthorDate: 2023-02-24 09:23:48 +0000
Commit:     Robert Clausecker <fuz@FreeBSD.org>
CommitDate: 2023-02-27 22:46:24 +0000

    databases/sqlite3: update to 3.41.0

    Disable option DQS by default.
    Add a note to UPDATING about this.

    See also: https://sqlite.org/quirks.html#dblquote
    Changelog: https://sqlite.org/releaselog/3_41_0.html

    PR:             269798
    Approved by:    flo (mentor)
    Differential Revision: https://reviews.freebsd.org/D38774

 UPDATING                                           | 11 +++++++
 databases/sqlite3/Makefile                         |  6 ++--
 databases/sqlite3/distinfo                         |  6 ++--
 .../sqlite3/files/patch-test_e__uri.test (gone)    | 13 ---------
 .../sqlite3/files/patch-test_fts3expr4.test (gone) | 11 -------
 .../files/patch-test_scanstatus.test (gone)        | 34 ----------------------
 .../sqlite3/files/patch-test_stat.test (gone)      | 11 -------
 databases/sqlite3/files/patch-test_uri.test (gone) | 15 ----------
 8 files changed, 17 insertions(+), 90 deletions(-)
Comment 8 Robert Clausecker freebsd_committer freebsd_triage 2023-02-27 22:54:36 UTC
Thank you for your contribution.
Comment 9 Franco Fichtner 2023-03-09 13:42:46 UTC
FWIW, I think disabling DQS without discussion/warning was a rather unpractical move given the amount of tickets reporting breakage now.
Comment 10 Robert Clausecker freebsd_committer freebsd_triage 2023-03-09 16:00:38 UTC
(In reply to Franco Fichtner from comment #9)

SQLite3 has warned about this usage for the past four years.  Applications that had not fixed their syntax problems in that time frame would likely not have reacted to a notice period from our side either.
Comment 11 Franco Fichtner 2023-03-09 16:15:07 UTC
The QA process (poudriere) employed here was inappropriate for the change made. I've seen smaller things being reverted because of the averse effects it had on the ports.

I don't mind the lack of empathy, but I think this directly impacts the operability of the ports and wasn't properly vetted.


Cheers,
Franco
Comment 12 Robert Clausecker freebsd_committer freebsd_triage 2023-03-09 16:24:51 UTC
(In reply to Franco Fichtner from comment #11)

An exp-run would have not caught these problems in 4 out of hundreds of ports using sqlite3 as it does not occur until the programs are used in some way and then not always.  For example, the www/qutebrowser issue seems to only affect a small number of hand-written queries that do not execute unless certain features are use.

Out of about 600 ports using SQLite3, 5 ports are known to dislike the new default.  2 of them already have patches, the other three are work in progress. 
 None of these problems could have been discovered with Poudriere.

Please tell me what sort of reasonable extra testing you believe would have caught these issues ahead of time.  I'm always happy to take reasonable effort to improve the quality of my ports work.
Comment 13 Franco Fichtner 2023-03-09 16:27:21 UTC
Exactly my point.  Since this cannot be easily verified the approach to slip the disable in there without a notice is a critical mistake. 

And ports aside, production systems have broken and will break if this stands as is.  If you don't take my word for it, perhaps more tickets will arrive as soon as this hits the package mirror.


Cheers,
Franco
Comment 14 Franco Fichtner 2023-03-09 16:27:51 UTC
Correction: notice as in forward notice
Comment 15 Robert Clausecker freebsd_committer freebsd_triage 2023-03-09 16:44:25 UTC
(In reply to Franco Fichtner from comment #13)

Advance notice has been given by the SQLite project 4 years ago and by us in the UPDATING section.  You all read UPDATING before updating your packages, don't you?

If you have software that breaks with the new default in production use, this tells you everything you need to know to locally work around the issue (i.e. that you need to reenable the DQS option and then build your own sqlite3 package).

I'd also wager that additional advance notice by us would have caught precisely none of these issues as people don't actually check these things until they break.  I mean to get to this point, upstream must have ignored the SQLite warnings for 4 years.
Comment 16 Franco Fichtner 2023-03-09 16:47:12 UTC
Honestly I don't expect you to defect blame because I'm not blaming. I'm merely pointing out this was carried out in a suboptimal way.

You imply in UPDATING that people know about sqlite internals, which most running complex systems don't.  Some don't even know they have sqlite components until they break. The approach you show is displaying a lack of empathy towards the complexity of the user end and there are easy solutions such as pkg-message notices and a planned phase out of the option.


Cheers,
Franco
Comment 17 ctyxyj9wa7al4 2023-03-13 17:08:34 UTC
MARKED AS SPAM