Bug 257309 - sysutils/beats7: 7.13 compatibility to non-Elastic distributions
Summary: sysutils/beats7: 7.13 compatibility to non-Elastic distributions
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: Juraj Lutter
URL: https://www.elastic.co/guide/en/beats...
Keywords: needs-qa
Depends on:
Reported: 2021-07-21 10:27 UTC by bgdnlp
Modified: 2021-07-21 19:55 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (elastic)
koobs: maintainer-feedback? (otis)


Note You need to log in before you can comment on or make changes to this bug.
Description bgdnlp 2021-07-21 10:27:56 UTC
Version 7.13 broke compatibility with 7.10 or earlier open source distributions of Elasticsearch [1]. To point at the elephant, 7.13 doesn't connect to AWS ElasticSearch any more. This is a breaking change on a minor version with no warning to users, not even the standard message in UPDATING as far as I can tell. Not nice.

That means that some users of quarterly pkgs will soon find that beats7 isn't working any more with no documented workaround and no warning. Are there any plans to mitigate this?

[1] https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-7.13.html#breaking-changes-7.13
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2021-07-21 10:35:11 UTC
^Triage: Request feedback from committer of ports afd8461bb

@Reporter Have you seen reports, conversations, issues, or other references online with respect to what the community may be doing, or have already done with respect to this change? That might help identify possible / appropriate options for us.
Comment 2 bgdnlp 2021-07-21 11:14:57 UTC
(In reply to Kubilay Kocak from comment #1)

Yes, I can add more details, but I can't yet confirm the workaround works. Pretty sure it does. Basically we downgraded to beats 7.10.

The error that shows up in logs is this:

ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://my.elasticsearch.url:443)): Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: unauthorized access, could not connect to the xpack endpoint, verify your credentials

The pull request that broke it is: https://github.com/elastic/beats/pull/25351/files

This other pull request that would have restored functionality was rejected: https://github.com/elastic/beats/pull/26305

(I don't know what the code does or if it works as intended)

My workaround is to downgrade to 7.10, which is thankfully still available in quarterly repo, then lock the package:

sudo service filebeat stop
sudo pkg remove beats7
fetch https://pkg.freebsd.org/FreeBSD:12:amd/quarterly/All/beats7-7.10.1.txz
sudo pkg install -y beats7-7.10.1.txz
sudo pkg lock -y beats7-7.10.1

I guess one way forward would be to provide a 'beats7-oss' package or similar which would only be upgraded after beats7 is (inevitably) forked to be patched to work with non-Elastic distributions of ES.
Comment 3 bgdnlp 2021-07-21 11:20:14 UTC
(In reply to bgdnlp from comment #2)
Looks like I can't edit my comment. The link to the package in quarterly is wrong. For x64 it's, of course, https://pkg.freebsd.org/FreeBSD:12:amd64/quarterly/All/beats7-7.10.1.txz
Comment 4 Juraj Lutter freebsd_committer 2021-07-21 15:01:04 UTC
I will update the ports to 7.13.4 in my WIP area, then you can test them (as I don't have any access to AWS elasticsearch) so they can be committed.
Comment 5 bgdnlp 2021-07-21 19:55:29 UTC
If it helps I can provide a small 1-node cluster for testing for a week or two. Public access, completely open. Something like 2 CPU, 4 GB RAM, 20 GB disk. Should be enough, right? It would probably take me a couple of days to have it up and running.

If that would be useful I just need a private point of contact to send access details to when ready, like an email or something.