Bug 231848 - accept4() doesn't return a socket address when threading is involved
Summary: accept4() doesn't return a socket address when threading is involved
Status: Closed Not A Bug
Alias: None
Product: Base System
Classification: Unclassified
Component: threads (show other bugs)
Version: 11.2-STABLE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-threads (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-01 10:53 UTC by Andreas Schneider
Modified: 2018-11-10 18:05 UTC (History)
3 users (show)

See Also:


Attachments
server code (1.90 KB, text/plain)
2018-11-09 18:01 UTC, Anoop C S
no flags Details
client code (1.85 KB, text/plain)
2018-11-09 18:01 UTC, Anoop C S
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Schneider 2018-10-01 10:53:06 UTC
We've added threading support to socket_wrapper [1] which is a tool to create artificial networks for testing.

We wrote several tests to make sure socket_wrapper works correctly. However there are two tests which fail on FreeBSD but work correctly on Linux.

When running with socket_wrapper we tracked it down to accept4() not filling out the passed sockaddr. socket_wrapper uses unix socket and we would expect the unix socket path returned by accept4, which isn't the case.

If I change the echo_srv to write to a file and enable debugging of socket_wapper this slows down the whole thing and the tests pass correclty. This means accept4 correctly filling out the sockaddr structure. So this looks like a race condition in the freebsd libc code.

You can find the code we are currently working on at [2].

git clone https://git.cryptomilk.org/users/asn/socket_wrapper.git
cd socket_wrapper
mkdir obj
cd obj
cmake -DCMAKE_BUILD_TYPE=Debug -DUINIT_TESTING=ON ..
make -j8

To run the whole test suite, you can use:
ctest --output-on-failure

Which should result in:

The following tests FAILED:
         20 - test_thread_echo_tcp_connect (Failed)
         21 - test_thread_echo_tcp_write_read (Failed)
         22 - test_thread_echo_tcp_sendmsg_recvmsg (Failed)

To run an individual test you can use:

ctest -V -R test_thread_echo_tcp_write_read


This will also print you how it is called, normally this is something like:

CMOCKA_TEST_ABORT=1 \
LD_PRELOAD=/home/vagrant/workspace/projects/socket_wrapper/obj/src/libsocket_wrapper.so \
tests/test_thread_echo_tcp_write_read


The test could also be run without socket_wrapper using real sockets:

CMOCKA_TEST_ABORT=1 \
TORTURE_SERVER_ADDRESS_IPV4="127.0.0.1" \
TORTURE_SERVER_ADDRESS_IPV6="::1" \
TORTURE_SERVER_PORT=7777 \
tests/test_thread_echo_tcp_write_read


[1] https://cwrap.org/socket_wrapper.html
[2] https://git.cryptomilk.org/users/asn/socket_wrapper.git/log/?h=master-fix
Comment 1 Anoop C S 2018-11-09 18:01:21 UTC
Created attachment 199095 [details]
server code
Comment 2 Anoop C S 2018-11-09 18:01:45 UTC
Created attachment 199096 [details]
client code
Comment 3 Anoop C S 2018-11-09 18:12:27 UTC
See attachment for a client-server program and Compile it as follows to reproduce the issue:

# cc sv.c -o sv
# cc cl.c -lpthread -o cl

Run server in one shell:
# ./sv

Execute client program from another shell with thread count value:
# ./cl 10

Output samples:
with thread count = 5

client:
-------
$ ./cl 5
hello world
hello world
hello world
hello world
hello world

server:
-------
$ ./sv 
1 CONNECTION/S ESTABLISHED
2 CONNECTION/S ESTABLISHED
3 CONNECTION/S ESTABLISHED
4 CONNECTION/S ESTABLISHED
5 CONNECTION/S ESTABLISHED
^C

with thread count = 10

client:
$ ./cl 10
Failed to connect to socket: Connection refused

server:
$ ./sv 
1 CONNECTION/S ESTABLISHED
2 CONNECTION/S ESTABLISHED
3 CONNECTION/S ESTABLISHED
4 CONNECTION/S ESTABLISHED
5 CONNECTION/S ESTABLISHED
6 CONNECTION/S ESTABLISHED
7 CONNECTION/S ESTABLISHED
8 CONNECTION/S ESTABLISHED
^C

client program is written in such a way that it terminates on receiving refusal during a connect() to server.
Comment 4 Konstantin Belousov freebsd_committer 2018-11-09 19:18:21 UTC
Your server limits the length of the accept queue to 5.
Comment 5 Anoop C S 2018-11-10 05:20:29 UTC
(In reply to Konstantin Belousov from comment #4)
Ah.. My bad.

Interestingly, echo server inside socket_wrapper code also have a backlog count of 5. But the threaded test cases were all executed with 10 threads. This will most probably be the reason for test failures.

I will verify and close this bug report accordingly.

Thanks for mentioning the wrong backlog count.
Comment 6 Andreas Schneider 2018-11-10 18:05:15 UTC
Closing as this is an issue in the server implementation.