Bug 254628 - sysutils/fluent-bit: tail plugin not supported
Summary: sysutils/fluent-bit: tail plugin not supported
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Palle Girgensohn
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-28 16:09 UTC by pete
Modified: 2023-04-06 14:16 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (girgen)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description pete 2021-03-28 16:09:38 UTC
I am working on rolling out fluent-bit and currently make use of the tail plugin for my ruby based fluentd systems.  When trying to run this on FreeBSD it get an error:

$ fluent-bit -s 6000 -i tail  -p path=/var/log/messages -o stdout
Fluent Bit v1.7.2
* Copyright (C) 2019-2021 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2021/03/28 09:00:34] [ info] [engine] started (pid=50457)
[2021/03/28 09:00:34] [ info] [storage] version=1.1.1, initializing...
[2021/03/28 09:00:34] [ info] [storage] in-memory
[2021/03/28 09:00:34] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2021/03/28 09:00:34] [ info] [sp] stream processor started
[2021/03/28 09:00:34] [error] [plugins/in_tail/tail_file.c:1088 errno=63] File name too long


After some digging it looks like their tail implementation is only supported on Linux/OSX and Windows:
https://github.com/fluent/fluent-bit/issues/208
https://github.com/fluent/fluent-bit/issues/2897

Filing this ticket to see if it would be possible to get a patch together to add support for the tail plugin.
Comment 1 Christopher Beppler 2021-05-03 13:22:14 UTC
Maybe it would be worthwile to set the CMake option FLB_IN_TAIL the this port to to False until this has been implemented upstream?
Comment 2 bgdnlp 2021-07-15 11:17:16 UTC
Any idea of the status of tail plugin in FreeBSD please?

Installed fluent-bit 1.7.9 from packages, enabled tail plugin, we get this in logs every couple of seconds or so:

[error] [/wrkdirs/usr/ports/sysutils/fluent-bit/work/.build/plugins/in_tail/CMakeFiles/flb-plugin-in_tail.dir/compiler_depend.ts:1104 errno=2] No such file or directory

The plugin seems to work though, it reads files and sends the data, but I'm not sure what the error means or if it will still work later, on file rotation for example.

(there's also the issue that fluent-bit dies if the `-d` flag is passed to it (daemon on), but I can report that separately)
Comment 3 Christopher Beppler 2021-07-15 14:46:01 UTC
Hi, the second issue with -d is possibly related to #255593.
Comment 4 bgdnlp 2021-07-16 06:31:26 UTC
(In reply to Christopher Beppler from comment #3)

I've seen that ticket, but I wasn't sure what to make of it. It starts, but not as daemon. I'll add my comments there.
Comment 5 Mårten Lindblad 2022-09-13 20:40:06 UTC
Trying to enable the loki plugin i get a similar error:

[2022/09/13 20:39:09] [error] [/wrkdirs/usr/ports/sysutils/fluent-bit/work/.build/plugins/in_tail/CMakeFiles/flb-plugin-in_tail.dir/compiler_depend.ts:1421 errno=2] No such file or directory
[2022/09/13 20:39:13] [error] [/wrkdirs/usr/ports/sysutils/fluent-bit/work/.build/plugins/in_tail/CMakeFiles/flb-plugin-in_tail.dir/compiler_depend.ts:1421 errno=2] No such file or directory
Comment 6 Hiroo Ono 2023-04-01 22:51:20 UTC
Now that fluent-bit is updated and tail plugin should be supported for FreeBSD, this bug may be closed?
Comment 7 pete 2023-04-06 14:16:13 UTC
(In reply to Hiroo Ono from comment #6)

Thanks Hiroo for updating this bug.  I can confirm things work now on my 13.1-RELEASE system using the command at the beginning of this ticket:

$ fluent-bit -s 6000 -i tail  -p path=/var/log/messages -o stdout
[2023/04/06 14:13:55] [ info] [config] changing coro_stack_size from 3072 to 4096 bytes
Fluent Bit v2.0.10
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/04/06 14:13:55] [ info] [fluent bit] version=2.0.10, commit=, pid=60160
[2023/04/06 14:13:55] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/04/06 14:13:55] [ info] [cmetrics] version=0.5.8
[2023/04/06 14:13:55] [ info] [ctraces ] version=0.2.7
[2023/04/06 14:13:55] [ info] [input:tail:tail.0] initializing
[2023/04/06 14:13:55] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2023/04/06 14:13:55] [ info] [sp] stream processor started
[2023/04/06 14:13:55] [ info] [output:stdout:stdout.0] worker #0 started
^C[2023/04/06 14:13:59] [engine] caught signal (SIGINT)
[2023/04/06 14:13:59] [ warn] [engine] service will shutdown in max 5 seconds
[2023/04/06 14:13:59] [ info] [input] pausing tail.0
[2023/04/06 14:14:00] [ info] [engine] service has stopped (0 pending tasks)
[2023/04/06 14:14:00] [ info] [input] pausing tail.0
[2023/04/06 14:14:00] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/04/06 14:14:00] [ info] [output:stdout:stdout.0] thread worker #0 stopped