Bug 247523

Summary: www/nginx: module ngx_http_upstream_check stopped working after migrating to dynamic in 1.18.0_12,2
Product: Ports & Packages Reporter: Felipe Zipitria <fzipitria>
Component: Individual Port(s)Assignee: Jochen Neumeister <joneum>
Status: Closed Unable to Reproduce    
Severity: Affects Some People CC: adam.strohl-fbsd, joneum
Priority: --- Flags: bugzilla: maintainer-feedback? (joneum)
Version: Latest   
Hardware: Any   
OS: Any   

Description Felipe Zipitria 2020-06-24 16:27:22 UTC
Hi joneum,

Yesterday we upgraded to 1.18.0_15, and the module upstream_check stopped working properly.

You can do a test using a simple config file:

<begin config file>
worker_processes 1;
worker_rlimit_nofile 262144;

load_module "/usr/local/libexec/nginx/ngx_http_upstream_check_module.so";

error_log /tmp/test/error.log debug;

events {
    worker_connections  16384;
    use kqueue;
    kqueue_changes 2048;
}


http {

        upstream cluster {

            # simple round-robin
            server 192.168.0.1:80;
            server 192.168.0.2:80;
            check interval=5000 rise=1 fall=3 timeout=4000;
        }

        server {
            listen 50080;

            location / {
                proxy_pass http://cluster;
            }

            location /status {
                check_status;

                access_log   off;
           }
        }

    }
<end config file>

<begin commands>
nginx -p /tmp/test -c test.conf
curl http://localhost:50080/status
<end commands>

And logs show:
<begin logs>
2020/06/23 12:35:00 [debug] 20275#101354: worker cycle
2020/06/23 12:35:00 [debug] 20275#101354: kevent timer: 60000, changes: 1
2020/06/23 12:35:00 [debug] 20275#101354: kevent events: 1
2020/06/23 12:35:00 [debug] 20275#101354: kevent: 3: ft:-1 fl:0020 ff:00000000 d:85 ud:000000080680D111
2020/06/23 12:35:00 [debug] 20275#101354: *3 http wait request handler
2020/06/23 12:35:00 [debug] 20275#101354: *3 malloc: 0000000802270000:1024
2020/06/23 12:35:00 [debug] 20275#101354: *3 recv: eof:0, avail:85, err:0
2020/06/23 12:35:00 [debug] 20275#101354: *3 recv: fd:3 85 of 1024
2020/06/23 12:35:00 [debug] 20275#101354: *3 reusable connection: 0
2020/06/23 12:35:00 [debug] 20275#101354: *3 posix_memalign: 000000080227F000:4096 @16
2020/06/23 12:35:00 [debug] 20275#101354: *3 http process request line
2020/06/23 12:35:00 [debug] 20275#101354: *3 http request line: "GET /status HTTP/1.1"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http uri: "/status"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http args: ""
2020/06/23 12:35:00 [debug] 20275#101354: *3 http exten: ""
2020/06/23 12:35:00 [debug] 20275#101354: *3 posix_memalign: 000000080227B000:4096 @16
2020/06/23 12:35:00 [debug] 20275#101354: *3 http process request header line
2020/06/23 12:35:00 [debug] 20275#101354: *3 http header: "Host: localhost:50080"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http header: "User-Agent: curl/7.67.0"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http header: "Accept: */*"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http header done
2020/06/23 12:35:00 [debug] 20275#101354: *3 event timer del: 3: 7021985672
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 0
2020/06/23 12:35:00 [debug] 20275#101354: *3 rewrite phase: 1
2020/06/23 12:35:00 [debug] 20275#101354: *3 test location: "/"
2020/06/23 12:35:00 [debug] 20275#101354: *3 test location: "status"
2020/06/23 12:35:00 [debug] 20275#101354: *3 using configuration "/status"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http cl:-1 max:1048576
2020/06/23 12:35:00 [debug] 20275#101354: *3 rewrite phase: 3
2020/06/23 12:35:00 [debug] 20275#101354: *3 rewrite phase: 4
2020/06/23 12:35:00 [debug] 20275#101354: *3 post rewrite phase: 5
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 6
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 7
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 8
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 9
2020/06/23 12:35:00 [debug] 20275#101354: *3 access phase: 10
2020/06/23 12:35:00 [debug] 20275#101354: *3 access phase: 11
2020/06/23 12:35:00 [debug] 20275#101354: *3 access phase: 12
2020/06/23 12:35:00 [debug] 20275#101354: *3 post access phase: 13
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 14
2020/06/23 12:35:00 [debug] 20275#101354: *3 generic phase: 15
2020/06/23 12:35:00 [debug] 20275#101354: *3 http set discard body
2020/06/23 12:35:00 [error] 20275#101354: *3 http upstream check module can not find any check server, make sure you've added the check servers, client: 127.0.0.1, server: , request: "GET /status HTTP/1.1", host: "localhost:50080"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http finalize request: 500, "/status?" a:1, c:1
2020/06/23 12:35:00 [debug] 20275#101354: *3 http special response: 500, "/status?"
2020/06/23 12:35:00 [debug] 20275#101354: *3 http set discard body
2020/06/23 12:35:00 [debug] 20275#101354: *3 HTTP/1.1 500 Internal Server Error
Server: nginx/1.18.0
Date: Tue, 23 Jun 2020 19:35:00 GMT
Content-Type: text/html
Content-Length: 177
Connection: close
<end logs>

Looks as if it wasn't installed properly.
Comment 1 Jochen Neumeister freebsd_committer 2020-07-03 21:32:59 UTC
The module was tested with Poudriere and a test server (FreeBSD 12.1 / amd64) before release. There were no problems.

If you want me to test your configuration here in more detail, please contact me privately.
Comment 2 Felipe Zipitria 2020-07-05 00:42:57 UTC
Just to confirm that using a vanilla VM and building from ports everything worked as expected. So this should be a particular problem, not related to the package. Will do better testing next time, sorry.
Comment 3 Adam Strohl 2020-07-15 22:20:20 UTC
I would add that we see this issue as well with our our poudriere pkg repo -- anyone have any ideas what's causing this?
Comment 4 Jochen Neumeister freebsd_committer 2020-07-15 23:25:36 UTC
(In reply to Adam Strohl from comment #3)

see #2
Comment 5 Adam Strohl 2020-07-16 00:58:19 UTC
(In reply to Jochen Neumeister from comment #4)

I don't see what was actually wrong in comment #2, just that building it from ports apparently works?
Comment 6 Adam Strohl 2020-08-05 01:52:26 UTC
Can confirm that this is 100% broken if you build with Poudriere however the port itself works.

Are Poudriere builds not supported?  How is the FreeBSD project building this successfully for it's own repo?