It seems like https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190824 broke getaddrinfo in jail with configured only lo0. Real life jail example: [lonerr@tonal ~]% ifconfig re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> ether d4:3d:7e:ec:e1:49 media: Ethernet autoselect (1000baseT <full-duplex>) status: active plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500 lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet 127.0.1.4 netmask 0xffffffff pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33152 After upgrade to 9.3 fetchmail failed to start: fetchmail: 6.3.26 querying google (protocol IMAP) at Mon Jul 21 19:18:03 2014: poll started fetchmail: getaddrinfo("imap.gmail.com","imaps") error: Non-recoverable failure in name resolution Simple test program: [lonerr@tonal ~]% cat gai.c #include <netdb.h> #include <stdio.h> #include <sys/socket.h> int main() { struct addrinfo hints, *res; int rc; memset(&hints, 0, sizeof(hints)); hints.ai_socktype = SOCK_STREAM; hints.ai_family = AF_UNSPEC; hints.ai_flags |= AI_ADDRCONFIG; printf("rc:%d\n",getaddrinfo("www.freebsd.org", NULL, &hints, &res)); } Run test program on 9.3-RELEASE-p0 with libc.so.7 from base ditribution: [lonerr@tonal ~]% ./gai rc:4 Simple commented out loopback ignoring: [lonerr@tonal ~]% svn di Index: lib/libc/net/getaddrinfo.c =================================================================== --- lib/libc/net/getaddrinfo.c (revision 268939) +++ lib/libc/net/getaddrinfo.c (working copy) @@ -1557,8 +1557,10 @@ if (seen_inet) continue; sin = (struct sockaddr_in *)(ifa->ifa_addr); + /* if (IN_LOOPBACK(htonl(sin->sin_addr.s_addr))) continue; + */ seen_inet = 1; break; #ifdef INET6 Run again after rebuild/reinstall libc.so.7: [lonerr@tonal ~]% ./gai rc:0
I've hit this bug here too, as it causes phantomjs (ports/lang/phantomjs) to not be able to resolve host names. I've tried to run the following snippet on phantomjs: var page = require('webpage').create(); page.open('http://freebsd.org', function(status) { console.log("Status: " + status); if(status === "success") { page.render('freebsd.png'); } phantom.exit(); }); Running it in debug mode in phantomjs: 2015-06-25T12:22:36 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=<filename>' to enable persisten cookie storage) 2015-06-25T12:22:36 [DEBUG] Phantom - execute: Configuration 2015-06-25T12:22:36 [DEBUG] 0 objectName : "" 2015-06-25T12:22:36 [DEBUG] 1 cookiesFile : "" 2015-06-25T12:22:36 [DEBUG] 2 diskCacheEnabled : "false" 2015-06-25T12:22:36 [DEBUG] 3 maxDiskCacheSize : "-1" 2015-06-25T12:22:36 [DEBUG] 4 ignoreSslErrors : "false" 2015-06-25T12:22:36 [DEBUG] 5 localToRemoteUrlAccessEnabled : "false" 2015-06-25T12:22:36 [DEBUG] 6 outputEncoding : "UTF-8" 2015-06-25T12:22:36 [DEBUG] 7 proxyType : "http" 2015-06-25T12:22:36 [DEBUG] 8 proxy : ":1080" 2015-06-25T12:22:36 [DEBUG] 9 proxyAuth : ":" 2015-06-25T12:22:36 [DEBUG] 10 scriptEncoding : "UTF-8" 2015-06-25T12:22:36 [DEBUG] 11 webSecurityEnabled : "true" 2015-06-25T12:22:36 [DEBUG] 12 offlineStoragePath : "" 2015-06-25T12:22:36 [DEBUG] 13 offlineStorageDefaultQuota : "-1" 2015-06-25T12:22:36 [DEBUG] 14 printDebugMessages : "true" 2015-06-25T12:22:36 [DEBUG] 15 javascriptCanOpenWindows : "true" 2015-06-25T12:22:36 [DEBUG] 16 javascriptCanCloseWindows : "true" 2015-06-25T12:22:36 [DEBUG] 17 sslProtocol : "tlsv1" 2015-06-25T12:22:36 [DEBUG] 18 sslCertificatesPath : "" 2015-06-25T12:22:36 [DEBUG] 19 webdriver : ":" 2015-06-25T12:22:36 [DEBUG] 20 webdriverLogFile : "" 2015-06-25T12:22:36 [DEBUG] 21 webdriverLogLevel : "INFO" 2015-06-25T12:22:36 [DEBUG] 22 webdriverSeleniumGridHub : "" 2015-06-25T12:22:36 [DEBUG] Phantom - execute: Script & Arguments 2015-06-25T12:22:36 [DEBUG] script: "example.js" 2015-06-25T12:22:36 [DEBUG] Phantom - execute: Starting normal mode 2015-06-25T12:22:36 [DEBUG] WebPage - setupFrame "" 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/package.json" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./parser.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./nodes.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././scope.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././helpers.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/././lexer.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/../coffee-script/./lib/coffee-script/./././rewriter.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 10 2015-06-25T12:22:36 [DEBUG] Network - Resource request error: 3 ( "Host codigo23.net not found" ) URL: "http://freebsd.org/" 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 100 Status: fail 2015-06-25T12:22:36 [DEBUG] WebPage - setupFrame "" 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 10 2015-06-25T12:22:36 [DEBUG] WebPage - setupFrame "" 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/_coffee-script.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r") ) ) 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 100 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 10 2015-06-25T12:22:36 [DEBUG] WebPage - updateLoadingProgress: 100 2015-06-25T12:22:36 [DEBUG] WebPage - setupFrame "" 2015-06-25T12:22:36 [DEBUG] WebPage - setupFrame "" After compiled phantomjs with debug symbols and ran it in a gdb session, I traced the error back to this call to getaddrinfo(): https://github.com/ariya/phantomjs/blob/1.9/src/qt/src/network/kernel/qhostinfo_unix.cpp#L213 which fails in a similar way to the sample code provided by Oleg. This server is running: 10.1-STABLE FreeBSD 10.1-STABLE #4 r280361: Mon Mar 23 16:31:23 CET 2015 built from sources (GENERIC kernel) and the jail.conf file is: # # /etc/jails.conf # # Common configuration options path = "/jails/${name}"; interface = "lo0"; allow.raw_sockets = 0; exec.clean; exec.system_user = "root"; exec.jail_user = "root"; exec.start += "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.consolelog = "/var/log/jail_${name}_console.log"; mount.devfs; allow.set_hostname = 0; allow.sysvipc = 1; betabug { host.hostname = "phantomjs.codigo23.net"; ip4.addr += "lo0|127.0.0.2/32"; } I had not tried Oleg fix (this is a production server), so I was wondering if there would be any workaround while a fix gets into the official sources... ?
The workaround that helped me was assigning addresses from 192.168.0.0/16 private block (instead of 127.0.0.0/8) for all jails.
Yes, thanks Oleg, that worked for me here too.
I have a totally different question: is your resolver in /ec/resolv.conf inside said jail also on 127/8 or not? If not, then that's not the jail's fault.
> cat /etc/resolv.conf search your-server.de nameserver 213.133.98.98 nameserver 213.133.99.99 nameserver 213.133.100.100 > That's currently the resolv.conf on that jail. I tried setting up the local unbound resolver too, but did not change anything (it was failing too)
It doesn't matter whether resolver from 127/8 or not. The getaddrinfo wouldn't work anyway.
This bug was identified as a high priority issue at the EuroBSDCon 2015 dev summit
This is not a bug but a feature. RFC 3493 says: If the AI_ADDRCONFIG flag is specified, IPv4 addresses shall be returned only if an IPv4 address is configured on the local system, and IPv6 addresses shall be returned only if an IPv6 address is configured on the local system. The loopback address is not considered for this case as valid as a configured address. I'm curious why you use loopbak address?
Because it is handy when you have a system with a number of local-only Jails. You give them addresses in the 127.0.0.0 range and then you use PF (or whatever) to redirect traffic there if needed. I've doing that for quite a while, until I hit this bug^feature. If it is indeed the expected behaviour, maybe a note about it should be added to the Jails docs, to prevent people from assigning jails addresses in that range.
Created attachment 164104 [details] Ignore just 127.0.0.1 instead of the whole loopback address range. Yes, it is expected behavior from the spec of AI_ADDRCONFIG. Since it was not implemented correctly, your setup used to be working by chance. Ignoring the loopback address is necessary to make AI_ADDRCONFIG usable. However, the loopback address except for 127.0.0.1 isn't used in a usual setup. Though it breaks a spec to be exact, there may be room for modifying getaddrinfo(3) to ignore just 127.0.0.1 instead of the whole loopback address range. Does the attached patch work for you?
A commit references this bug: Author: ume Date: Sat Dec 12 14:42:51 UTC 2015 New revision: 292129 URL: https://svnweb.freebsd.org/changeset/base/292129 Log: RFC 3493 requires ignoring the loopback address for A_ADDRCONFIG. Since it breaks certain jail setup, we ignore just 127.0.0.1 instead of whole loopback address range. PR: 192014 Reviewed by: hrs MFC after: 1 week Changes: head/lib/libc/net/getaddrinfo.c
Yes, I can confirm that patch works for me. Thank you!
A commit references this bug: Author: ume Date: Sat Dec 19 01:21:29 UTC 2015 New revision: 292456 URL: https://svnweb.freebsd.org/changeset/base/292456 Log: MFC r292129: RFC 3493 requires ignoring the loopback address for AI_ADDRCONFIG. Since it breaks certain jail setup, we ignore just 127.0.0.1 instead of whole loopback address range. PR: 192014 Reviewed by: hrs Changes: _U stable/10/ stable/10/lib/libc/net/getaddrinfo.c
A commit references this bug: Author: ume Date: Sat Dec 19 01:24:08 UTC 2015 New revision: 292457 URL: https://svnweb.freebsd.org/changeset/base/292457 Log: MFC r292129: RFC 3493 requires ignoring the loopback address for AI_ADDRCONFIG. Since it breaks certain jail setup, we ignore just 127.0.0.1 instead of whole loopback address range. PR: 192014 Reviewed by: hrs Changes: _U stable/9/lib/libc/ stable/9/lib/libc/net/getaddrinfo.c