Bug 254093

Summary: net/kafkacat: Update to 1.6.0
Product: Ports & Packages Reporter: Sergey Akhmatov <sergey>
Component: Individual Port(s)Assignee: Rainer Hurling <rhurlin>
Status: Closed FIXED    
Severity: Affects Some People CC: rhurlin, sergey
Priority: --- Keywords: buildisok
Version: Latest   
Hardware: Any   
OS: Any   
Bug Depends on: 254099    
Bug Blocks:    
Attachments:
Description Flags
kafkacat-1.6.0
sergey: maintainer-approval+
patch with devel/yail dependencies and reorderd GNU_CONFIGURE entry
none
kafka-1.6.0
sergey: maintainer-approval+
kafka-1.6.0 sergey: maintainer-approval+

Description Sergey Akhmatov 2021-03-07 10:54:18 UTC
Created attachment 223049 [details]
kafkacat-1.6.0

Update net/kafkacat to 1.6.0

ChangeLog: https://github.com/edenhill/kafkacat/releases/tag/1.6.0

- Transactional Producer support 
- Honour -k <key> when producing files
- Honour -o <offset> in -G high-level consumer mode
- Added -m <seconds> argument to set metadata/query/transaction timeouts.
- Allow schema.registry.url to be configured in config file and not only by -r.
- Print broker-id message was produced to (if -v),or was consumed from (if -J.
Comment 1 Automation User 2021-03-07 11:10:19 UTC
Build and package info is available at https://gitlab.com/swills/freebsd-ports/pipelines/266636500
Comment 2 Rainer Hurling freebsd_committer freebsd_triage 2021-03-07 12:15:34 UTC
Created attachment 223055 [details]
patch with devel/yail dependencies and reorderd GNU_CONFIGURE entry

Hi Sergey,

Thanks for the patch!

(1) portclippy suggests to reorder GNU_CONFIGURE=yes before the PLIST_FILES entry.

(2) 'make' gives the following error, if DEVELOPER=yes is set in /etc/make.conf:
====> Running Q/A tests (stage-qa)
Error: /usr/local/bin/kafkacat is linked to /usr/local/lib/libyajl.so.2 from devel/yajl but it is not declared as a dependency
Warning: you need LIB_DEPENDS+=libyajl.so:devel/yajl

In the updated patch you will find both changes. If you agree, I will commit this version of the patch ;)


---------------------------
One more issue: 'make configure' outputs
    checking for avroc (by pkg-config)... failed
    checking for avroc (by compile)... failed (disable)

and later in the 'Configuration summary' it prints
    BUILT_WITH               JSON AVRO CC PKGCONFIG

Interestingly enough, in WRKSRC there is an avro.c file. 

The 'Requirements' section in https://github.com/edenhill/kafkacat contains avro-c and libserdes as optional dependencies. Did you investigate into this? Should we wait with the commit for the existing commit?
Comment 3 Sergey Akhmatov 2021-03-07 12:52:32 UTC
(In reply to Rainer Hurling from comment #2)
Hello, thanks for the review!

>(1) portclippy suggests to reorder GNU_CONFIGURE=yes before the PLIST_FILES entry.

I'm using portlint by old habbit, and it was silent. Should I prefer portclippy? I'm ok with this change.

> (2) 'make' gives the following error, if DEVELOPER=yes is set in /etc/make.conf:
> ====> Running Q/A tests (stage-qa)
> Error: /usr/local/bin/kafkacat is linked to /usr/local/lib/libyajl.so.2 from devel/yajl but it is not declared as a dependency
> Warning: you need LIB_DEPENDS+=libyajl.so:devel/yajl

My poudriere build doesn't link kafkacat with this lib.
% ldd /usr/local/bin/kafkacat
/usr/local/bin/kafkacat:
        librdkafka.so.1 => /usr/local/lib/librdkafka.so.1 (0x800250000)
        libc.so.7 => /lib/libc.so.7 (0x8003f3000)
        liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x800804000)
        libm.so.5 => /lib/libm.so.5 (0x800833000)
        libzstd.so.1 => /usr/local/lib/libzstd.so.1 (0x800866000)
        libssl.so.111 => /usr/lib/libssl.so.111 (0x80095d000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x8009f6000)
        libz.so.6 => /lib/libz.so.6 (0x800ceb000)
        libdl.so.1 => /usr/lib/libdl.so.1 (0x800d07000)
        libthr.so.3 => /lib/libthr.so.3 (0x800d0b000)
        librt.so.1 => /usr/lib/librt.so.1 (0x800d38000)

It seems that configure detects if its installed and links with it automatically.
There are options in configure --help:

kafkacat options:
   --enable-json                        JSON support (requires libyajl2) [y]
   --disable-json                       JSON support (requires libyajl2)
   --enable-avro                        Avro/Schema-Registry support (requires libserdes) [y]
   --disable-avro                       Avro/Schema-Registry support (requires libserdes)

I think we should add port options for JSON and AVRO support and pass this options to configure explicitly, simultaneously adding LIB_DEPENDS. I'll come up with updated patch shortly.

I've never used this features myself and not sure if this options should be ON by default. What would you suggest?
Comment 4 Rainer Hurling freebsd_committer freebsd_triage 2021-03-07 13:02:48 UTC
(In reply to Sergey Akhmatov from comment #3)

> It seems that configure detects if its installed and links with it
> automatically.

Yes, seems you are right.

> There are options in configure --help:
>
> kafkacat options:
>  --enable-json    JSON support (requires libyajl2) [y]
>  --disable-json   JSON support (requires libyajl2)
>  --enable-avro    Avro/Schema-Registry support (requires libserdes) [y]
>  --disable-avro   Avro/Schema-Registry support (requires libserdes)
>
> I think we should add port options for JSON and AVRO support and pass
> this options to configure explicitly, simultaneously adding
> LIB_DEPENDS. I'll come up with updated patch shortly.

:)

> I've never used this features myself and not sure if this options
> should be ON by default. What would you suggest?

Hmm, not sure. The Github pages shows them as optional, so perhaps most users don't expect them to be enabled?

Whether the default settings are on or off, both should work regardless of whether a dependency is already installed (see libyail).

And I am not clear if the functionality of libserdes is already included in the portstree?
Comment 5 Sergey Akhmatov 2021-03-07 13:53:38 UTC
Created attachment 223059 [details]
kafka-1.6.0

Add new options for JSON and AVRO support
Comment 6 Sergey Akhmatov 2021-03-07 13:57:42 UTC
(In reply to Rainer Hurling from comment #4)
I've created port for libsedes and linked it as a dependency to this bug.

I've also updated patch with optional support for JSON and AVRO.
Everything builds and passes poudriere testport and runtime smoketests on my machine.
Comment 7 Sergey Akhmatov 2021-03-07 14:09:01 UTC
Sorry, something wrong: doesn't actually link with libserdes.
Have to investigate further. Simple update turned out to be not so simple.
Comment 8 Rainer Hurling freebsd_committer freebsd_triage 2021-03-07 14:12:26 UTC
(In reply to Sergey Akhmatov from comment #6 and comment #7)
Wow, that's fast!

I took PR 254099 and will do some tests know, before committing.

Please take the time it takes. After that, I will test again the cat ;)
Comment 9 Sergey Akhmatov 2021-03-07 15:06:12 UTC
Created attachment 223060 [details]
kafka-1.6.0

configure fails to detect avro-c with pkgconf:
DBG 67181: 'pkgconf --short-errors --cflags avro-c' failed: Package '@ZLIB_PKG@', required by 'avro-c', not found
Package '@LZMA_PKG@', required by 'avro-c', not found
Package '@SNAPPY_PKG@', required by 'avro-c', not found

because /usr/local/libdata/pkgconfig/avro-c.pc is broken, and it's broken upstream: https://github.com/apache/avro/pull/1018/commits

I suggest leaving AVRO option there, but mark it as broken till better times.
Comment 10 Rainer Hurling freebsd_committer freebsd_triage 2021-03-07 15:12:07 UTC
(In reply to Sergey Akhmatov from comment #9)

Maybe, this is because of the naming? 

pkg-config is looking for libavro-c.so and the ports one is libavro.so.

Just a guess ...
Comment 11 commit-hook freebsd_committer freebsd_triage 2021-03-08 18:12:42 UTC
A commit references this bug:

Author: rhurlin
Date: Mon Mar  8 18:12:34 UTC 2021
New revision: 567864
URL: https://svnweb.freebsd.org/changeset/ports/567864

Log:
  net/kafkacat: Update to 1.6.0

  New options AVRO and JSON.
  AVRO broken for now because of upstream problem with pkg-config

  Changelog: https://github.com/edenhill/kafkacat/releases/tag/1.6.0

  PR:		254093
  Submitted by:	Sergey Akhmatov <sergey@akhmatov.ru> (maintainer)

Changes:
  head/net/kafkacat/Makefile
  head/net/kafkacat/distinfo
  head/net/kafkacat/files/
Comment 12 Rainer Hurling freebsd_committer freebsd_triage 2021-03-08 18:16:33 UTC
Committed, thanks!

I did it with your suggested version for options: AVRO broken (for now) ;)