Bug 203111 - security/openvpn: TLS handshake fails on any freebsd box, even new
Summary: security/openvpn: TLS handshake fails on any freebsd box, even new
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: Normal Affects Only Me
Assignee: Matthias Andree
URL:
Keywords:
: 203146 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-09-15 00:21 UTC by brad
Modified: 2015-12-13 06:23 UTC (History)
1 user (show)

See Also:
koobs: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description brad 2015-09-15 00:21:31 UTC
I am trying to configure a new VPN server but the TLS handshake fails. I worked with #openvpn for some time and we narrowed it down to a failure at the server but what exactly, no one has a clue.

All firewalls are completely down for both server and client and the testing client 'can' connect to OpenBook free Openvpn servers just fine. Just not my own that are hosted on FreeBSD.

I jumped on a second brand new freebsd server and applied the config, same error. someone sent me a working config from one of their non freebsd servers and, same error.

What ever it is, it appears to be very FreeBSD specific.

server config:

[\u@vader:/usr/local/etc] # cat openvpn/openvpn.conf
local 108.61.175.20
mode server
port 1194
;proto tcp
proto udp
;dev tap
dev tun
;dev-node MyTap
ca /usr/local/etc/easy-rsa/keys/ca.crt
cert /usr/local/etc/easy-rsa/keys/serverP.crt
key /usr/local/etc/easy-rsa/keys/serverP.key
dh /usr/local/etc/easy-rsa/keys/dh2048.pem
topology subnet
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
;server-bridge 10.8.0.4 255.255.255.0 10.8.0.50 10.8.0.100
;server-bridge
;push "route 192.168.10.0 255.255.255.0"
;push "route 192.168.20.0 255.255.255.0"
;client-config-dir /usr/local/etc/ccd

;route 192.168.40.128 255.255.255.248
;client-config-dir ccd
;route 10.9.0.0 255.255.255.252
;learn-address ./script
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 4.2.2.2"

;client-to-client
;duplicate-cn
keepalive 10 120

tls-server
tls-timeout 120
tls-auth /usr/local/etc/openvpn/ta.key 0 # This file is secret
tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA
auth SHA256

;remote-cert-tls server
;cipher BF-CBC        # Blowfish (default)
cipher AES-128-CBC   # AES
;cipher DES-EDE3-CBC  # Triple-DES
comp-lzo
max-clients 100
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
log         /var/log/openvpn.log
log-append  /var/log/openvpn.log
verb 9
;mute 20

client config:

client
dev tun0
dev-type tun
proto udp
remote 108.61.175.20 1194
resolv-retry infinite
remote-cert-tls server
tls-auth C:\\Program\ Files\\OpenVPN\\config\\ta.key 1
tls-client
auth SHA256
dev-node {D1F4080E-CD73-4F64-9213-CBF0FB3C3D71}

resolv-retry infinite
nobind

persist-key
persist-tun
verb 3

;cipher AES-128-CBC
;route-delay 2
;redirect-gateway


inactive 3600
comp-lzo

ca       [inline]
cert     [inline]
key      [inline]


<ca>
-----BEGIN CERTIFICATE-----
MIIE7jCCA9agAwIBAgIJAMDi1PIJghxnMA0GCSqGSIb3DQEBCwUAMIGqMQswCQYD
VQQGEwJVUzELMAkGA1UECBMCTlkxETAPBgNVBAcTCEJyb29rbHluMRQwEgYDVQQK
EwtOWUNUZWxlY29tbTEUMBIGA1UECxMLTllDVGVsZWNvbW0xFzAVBgNVBAMTDk5Z
Q1RlbGVjb21tIENBMRAwDgYDVQQpEwdFYXN5UlNBMSQwIgYJKoZIhvcNAQkBFhVh
ZG1pbkBueWN0ZWxlY29tbS5jb20wHhcNMTUwOTE0MTEyNDU0WhcNMjUwOTExMTEy
NDU0WjCBqjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAk5ZMREwDwYDVQQHEwhCcm9v
a2x5bjEUMBIGA1UEChMLTllDVGVsZWNvbW0xFDASBgNVBAsTC05ZQ1RlbGVjb21t
MRcwFQYDVQQDEw5OWUNUZWxlY29tbSBDQTEQMA4GA1UEKRMHRWFzeVJTQTEkMCIG
CSqGSIb3DQEJARYVYWRtaW5AbnljdGVsZWNvbW0uY29tMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEA4UdREqvqO4SShs54q/m6hHxcm2Bc5jONAtk2I64p
vbDbFmkpyLpibPI+rENgi4o1jfvQJJECpsU8ycvVJ2dPb2k0OmWldmxjHO2GuIc2
LzhqbnycPH2zW1NOO1XmwxIi4USJBvUewHkxclefVh9VFzFxel37RV7rdeeizaeR
3T7udjCP2887RSh5ZQ/TG7P1GbEEeiD56kwM/NVOoUZonxGTaK8JulYZAEAAjmYk
7kzl98GL/RgqE3SKqyXtHSl5GWcSnGHIBoypRt0CALS4t/qQ2Dyr1SW2PJeFkOaP
ddEwjK/We8NiyDamLpahAq9Cj6ZRN+D2rr89z9MQXOH5DQIDAQABo4IBEzCCAQ8w
HQYDVR0OBBYEFCdr+IAjZA59H9EegFpZa5wwniR9MIHfBgNVHSMEgdcwgdSAFCdr
+IAjZA59H9EegFpZa5wwniR9oYGwpIGtMIGqMQswCQYDVQQGEwJVUzELMAkGA1UE
CBMCTlkxETAPBgNVBAcTCEJyb29rbHluMRQwEgYDVQQKEwtOWUNUZWxlY29tbTEU
MBIGA1UECxMLTllDVGVsZWNvbW0xFzAVBgNVBAMTDk5ZQ1RlbGVjb21tIENBMRAw
DgYDVQQpEwdFYXN5UlNBMSQwIgYJKoZIhvcNAQkBFhVhZG1pbkBueWN0ZWxlY29t
bS5jb22CCQDA4tTyCYIcZzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IB
AQBMtUC9Q5u6o9IyOayC3v+pIhmZncYGrxk03G6Odf3mU2ZeyCYja2qTlQOPB22V
XQEJ856KiOqOj0yyTyBJSG6UvpjD5iixf85WE/vBENOSDfzjhydmy8BgLWcRe0Dx
cFbEYv+qZr456s2W8Dt7+AI8VJauEQ5SPhf2WUK4XSH+7lLq2CDLN1qAHblyNks0
dxGaStTe38Pxb6FD0UpjFhSgoJqNZKuGjp5eeWdo0pAWu6T7QQ/9c/RYuTaz5/Kt
RSFUrJ/t0cYVz5sxUVR4KNR26QbVAI5J42n/BL00K5+xB5bP9yGUb5MzwRgFX2nI
J94Euf1q8OXN+kYa53Ca3W/J
-----END CERTIFICATE-----
</ca>

<cert>
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 2 (0x2)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=NY, L=Brooklyn, O=NYCTelecomm, OU=NYCTelecomm, CN=NYCTelecomm CA/name=EasyRSA/emailAddress=admin@nyctelecomm.com
        Validity
            Not Before: Sep 14 11:26:00 2015 GMT
            Not After : Sep 11 11:26:00 2025 GMT
        Subject: C=US, ST=NY, L=Brooklyn, O=NYCTelecomm, OU=NYCTelecomm, CN=client1P/name=EasyRSA/emailAddress=admin@nyctelecomm.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:b2:6f:5d:dc:14:b6:72:d1:80:42:34:4d:14:7d:
                    14:b0:c6:da:50:e0:e8:7f:bd:b4:28:b2:98:33:9f:
                    cd:d0:c1:a9:7c:6f:31:d5:17:cd:18:cf:50:d1:eb:
                    ef:ea:9b:c9:54:0d:03:c2:78:3f:2d:66:8b:a5:1b:
                    ba:39:28:f1:a8:9e:e6:0a:de:56:bc:c0:1a:ab:71:
                    92:ed:77:2d:6f:5d:1e:13:13:60:2a:08:94:76:49:
                    d0:b0:f7:a8:3c:6e:f0:a3:4a:95:25:0a:15:f4:63:
                    87:64:5d:70:0d:a3:89:08:f8:e1:88:72:d4:7c:6b:
                    b7:cb:68:55:ed:bb:23:73:f2:54:9c:7c:03:7f:c5:
                    24:20:ba:d2:de:eb:9f:e7:2c:6c:45:e6:09:f9:af:
                    6d:b5:e3:9d:6f:a5:37:7e:f7:f6:c3:d8:fc:91:dd:
                    7e:0c:c1:10:23:44:23:1c:6a:ee:05:cd:bd:6a:d4:
                    14:3e:71:f4:40:12:85:0d:6f:33:09:21:35:ba:26:
                    42:c1:f0:89:dd:1e:83:4e:e4:31:73:e3:1b:7b:68:
                    af:6d:5f:fd:a0:5f:64:24:6b:51:19:bd:ca:60:47:
                    f2:0f:a6:f5:3e:9d:94:90:f1:83:5a:21:02:8e:eb:
                    ee:45:8e:93:f0:cc:c2:da:6c:32:51:30:98:b3:0c:
                    5d:d9
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Comment:
                Easy-RSA Generated Certificate
            X509v3 Subject Key Identifier:
                3D:94:5E:09:B3:6F:E0:EF:B0:3D:3E:40:4D:AD:2F:DC:3C:52:86:90
            X509v3 Authority Key Identifier:
                keyid:27:6B:F8:80:23:64:0E:7D:1F:D1:1E:80:5A:59:6B:9C:30:9E:24:7D
                DirName:/C=US/ST=NY/L=Brooklyn/O=NYCTelecomm/OU=NYCTelecomm/CN=NYCTelecomm CA/name=EasyRSA/emailAddress=admin@nyctelecomm.com
                serial:C0:E2:D4:F2:09:82:1C:67

            X509v3 Extended Key Usage:
                TLS Web Client Authentication
            X509v3 Key Usage:
                Digital Signature
    Signature Algorithm: sha256WithRSAEncryption
         dd:15:70:12:67:c6:88:fa:c6:f6:01:16:54:df:c7:e1:ee:74:
         ee:00:75:11:fc:70:76:16:90:54:5a:1b:4f:8e:69:c5:c3:44:
         7f:79:9b:9f:98:01:71:2a:ec:59:15:3f:3d:27:b9:9d:0f:ce:
         cc:d1:05:1b:a1:f7:30:f3:e9:cc:37:bb:93:48:e7:14:ce:37:
         03:ee:c5:d8:cd:bb:ef:b2:b9:f3:94:a6:7b:23:49:16:c7:8f:
         73:ef:85:f9:8a:d5:98:24:bf:af:33:f0:19:4c:0c:a7:44:3b:
         c2:b8:43:10:d9:9a:65:6c:7c:50:00:9a:e3:69:21:d6:23:e0:
         66:80:a1:18:50:ef:58:a5:49:90:fc:27:41:f7:4a:39:c4:0b:
         5b:a4:8f:b6:d3:a1:6c:69:56:d9:13:96:0a:2a:32:48:fd:24:
         9c:94:20:5b:74:d6:54:b6:18:ea:f1:6c:bc:ee:bf:f8:86:ac:
         52:17:74:19:ce:f6:ae:ce:4d:84:a1:4f:99:06:ad:e7:29:a3:
         09:96:e7:e7:81:3f:7f:59:2a:83:bb:f1:0b:a5:d5:0b:36:86:
         4b:4d:d8:0c:67:1a:2a:5c:d1:a4:a1:4f:30:4f:c6:7b:7d:87:
         39:f3:93:05:5e:69:24:e8:81:e0:18:82:9e:7c:18:9d:6d:10:
         01:7a:08:e3
-----BEGIN CERTIFICATE-----
MIIFLjCCBBagAwIBAgIBAjANBgkqhkiG9w0BAQsFADCBqjELMAkGA1UEBhMCVVMx
CzAJBgNVBAgTAk5ZMREwDwYDVQQHEwhCcm9va2x5bjEUMBIGA1UEChMLTllDVGVs
ZWNvbW0xFDASBgNVBAsTC05ZQ1RlbGVjb21tMRcwFQYDVQQDEw5OWUNUZWxlY29t
bSBDQTEQMA4GA1UEKRMHRWFzeVJTQTEkMCIGCSqGSIb3DQEJARYVYWRtaW5Abnlj
dGVsZWNvbW0uY29tMB4XDTE1MDkxNDExMjYwMFoXDTI1MDkxMTExMjYwMFowgaQx
CzAJBgNVBAYTAlVTMQswCQYDVQQIEwJOWTERMA8GA1UEBxMIQnJvb2tseW4xFDAS
BgNVBAoTC05ZQ1RlbGVjb21tMRQwEgYDVQQLEwtOWUNUZWxlY29tbTERMA8GA1UE
AxMIY2xpZW50MVAxEDAOBgNVBCkTB0Vhc3lSU0ExJDAiBgkqhkiG9w0BCQEWFWFk
bWluQG55Y3RlbGVjb21tLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBALJvXdwUtnLRgEI0TRR9FLDG2lDg6H+9tCiymDOfzdDBqXxvMdUXzRjPUNHr
7+qbyVQNA8J4Py1mi6Ubujko8aie5greVrzAGqtxku13LW9dHhMTYCoIlHZJ0LD3
qDxu8KNKlSUKFfRjh2RdcA2jiQj44Yhy1Hxrt8toVe27I3PyVJx8A3/FJCC60t7r
n+csbEXmCfmvbbXjnW+lN3739sPY/JHdfgzBECNEIxxq7gXNvWrUFD5x9EAShQ1v
MwkhNbomQsHwid0eg07kMXPjG3tor21f/aBfZCRrURm9ymBH8g+m9T6dlJDxg1oh
Ao7r7kWOk/DMwtpsMlEwmLMMXdkCAwEAAaOCAWEwggFdMAkGA1UdEwQCMAAwLQYJ
YIZIAYb4QgENBCAWHkVhc3ktUlNBIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNV
HQ4EFgQUPZReCbNv4O+wPT5ATa0v3DxShpAwgd8GA1UdIwSB1zCB1IAUJ2v4gCNk
Dn0f0R6AWllrnDCeJH2hgbCkga0wgaoxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJO
WTERMA8GA1UEBxMIQnJvb2tseW4xFDASBgNVBAoTC05ZQ1RlbGVjb21tMRQwEgYD
VQQLEwtOWUNUZWxlY29tbTEXMBUGA1UEAxMOTllDVGVsZWNvbW0gQ0ExEDAOBgNV
BCkTB0Vhc3lSU0ExJDAiBgkqhkiG9w0BCQEWFWFkbWluQG55Y3RlbGVjb21tLmNv
bYIJAMDi1PIJghxnMBMGA1UdJQQMMAoGCCsGAQUFBwMCMAsGA1UdDwQEAwIHgDAN
BgkqhkiG9w0BAQsFAAOCAQEA3RVwEmfGiPrG9gEWVN/H4e507gB1EfxwdhaQVFob
T45pxcNEf3mbn5gBcSrsWRU/PSe5nQ/OzNEFG6H3MPPpzDe7k0jnFM43A+7F2M27
77K585SmeyNJFsePc++F+YrVmCS/rzPwGUwMp0Q7wrhDENmaZWx8UACa42kh1iPg
ZoChGFDvWKVJkPwnQfdKOcQLW6SPttOhbGlW2ROWCioySP0knJQgW3TWVLYY6vFs
vO6/+IasUhd0Gc72rs5NhKFPmQat5ymjCZbn54E/f1kqg7vxC6XVCzaGS03YDGca
KlzRpKFPME/Ge32HOfOTBV5pJOiB4BiCnnwYnW0QAXoI4w==
-----END CERTIFICATE-----
</cert>

<key>
-----BEGIN PRIVATE KEY-----
mmmmmyyyyy    ppppprrrriiiivvvaaatttteeee kkkkeeeeyyyy
yyyyyooouuuurrrr    nnnnooootttt ssuuuuppppoossseeeddd
tttttooo   ssseeeeee
-----END PRIVATE KEY-----
</key>
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
86ba44e5a40fb955687e62db59b7c747
6f71fc10c8a72eec1be7f4785d7700ee
cd88530490853a441666dcf1423d52c2
b22ff5d7f9abc4cfad581e8c4e5537da
3fd2d20901e5388efb7c4c9898ae1b42
3a74dcfb77352bd2d711a01d1d8e8382
ebc267eaec22ae0c027bd0f25ae6f0a6
b66a514c96078fc8f4437e98b778b202
9fbc3cda8325130570959bb729cdf325
307df71569aa4a1ef91a9c15ed2dc67f
c0491568e0c20f1e64b79f774fef764f
b9f56aa05b69f21cd2b5bc343c6ab645
8e4dd75a122c5418c3f005440f6de858
0dba19cc250a8f6da7c1302c8944f2b6
4b909dce9b8bf4721272e93f50573f4d
97517e2ec05d227a6a73f81292d866ce
-----END OpenVPN Static key V1-----
</tls-auth>

output from client:

PS C:\Program Files\OpenVPN\config> openvpn --config .\client1r.ovpn
Mon Sep 14 10:44:05 2015 OpenVPN 2.3.8 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Aug  4 2015
Mon Sep 14 10:44:05 2015 library versions: OpenSSL 1.0.1p 9 Jul 2015, LZO 2.08
Mon Sep 14 10:44:06 2015 Control Channel Authentication: tls-auth using INLINE static key file
Mon Sep 14 10:44:06 2015 Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
Mon Sep 14 10:44:06 2015 Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
Mon Sep 14 10:44:06 2015 Socket Buffers: R=[65536->65536] S=[65536->65536]
Mon Sep 14 10:44:06 2015 UDPv4 link local: [undef]
Mon Sep 14 10:44:06 2015 UDPv4 link remote: [AF_INET]108.61.175.20:1194
Mon Sep 14 10:45:06 2015 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Mon Sep 14 10:45:06 2015 TLS Error: TLS handshake failed
Mon Sep 14 10:45:06 2015 SIGUSR1[soft,tls-error] received, process restarting
Mon Sep 14 10:45:06 2015 Restart pause, 2 second(s)
Mon Sep 14 10:45:08 2015 Socket Buffers: R=[65536->65536] S=[65536->65536]
Mon Sep 14 10:45:08 2015 UDPv4 link local: [undef]
Mon Sep 14 10:45:08 2015 UDPv4 link remote: [AF_INET]108.61.175.20:1194
Mon Sep 14 10:46:08 2015 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)

Oddly, the Win10 testing client has 2 really weird errors that are probably not related but are worth mentioning. A packet capture from the client looks pretty unhealthy http://i.imgur.com/N45bHtv.png 

And if I change the inline keys to files, I get a 'file does not exist' error for the client.crt and the client.key, even if I put them in c:\ We were beating the client up until a couple of people tried the server and found that it wasn't properly responding.

All systems have been properly updated.
Comment 1 brad 2015-09-15 00:24:05 UTC
A switch to tcp from udp indicates deeper TLS issues  https://bpaste.net/show/4dc07da02e01
Comment 2 brad 2015-09-15 00:25:05 UTC
my rc.conf

[\u@vader:/root] # cat /etc/rc.conf
hostname="vader.ex-mailer.com"
ifconfig_vtnet0="dhcp"
sshd_enable="YES"
static_routes=linklocal
route_linklocal="-net 169.254.0.0/16 -interface vtnet0"
crypto_load=YES
cryptodev_load=YES
aesni_load=YES
virtio_random_load=YES
ifconfig_vtnet0_ipv6="inet6 2001:19f0:7400:84c6::64 prefixlen 64"
rtsold_enable=YES
ipv6_activate_all_interfaces=YES
rtsold_flags="-aF"
apache24_enable="yes"
mysql_enable="YES"
linux_enable="YES"
accf_data_load="YES"
export TERM=xterm-color
export GREP_OPTIONS='--color=auto' GREP_COLOR='1;32'
export CLICOLOR=1
export LSCOLORS=ExFxCxDxBxegedabagacad
export COLOR_NC='\e[0m' # No Color
export COLOR_WHITE='\e[1;37m'
export COLOR_BLACK='\e[0;30m'
export COLOR_BLUE='\e[0;34m'
export COLOR_LIGHT_BLUE='\e[1;34m'
export COLOR_GREEN='\e[0;32m'
export COLOR_LIGHT_GREEN='\e[1;32m'
export COLOR_CYAN='\e[0;36m'
export COLOR_LIGHT_CYAN='\e[1;36m'
export COLOR_RED='\e[0;31m'
export COLOR_LIGHT_RED='\e[1;31m'
export COLOR_PURPLE='\e[0;35m'
export COLOR_LIGHT_PURPLE='\e[1;35m'
export COLOR_BROWN='\e[0;33m'
export COLOR_YELLOW='\e[1;33m'
export COLOR_GRAY='\e[0;30m'
export COLOR_LIGHT_GRAY='\e[0;37m'

swapfile="/home/swap0"
pf_enable="YES"
pf_rules="/home/pf.conf"
pflog_enable="YES"
pflog_logfile="/var/log/pflog"

firewall_enable="YES"
firewall_type="open"

gateway_enable="YES"
natd_enable="YES"
natd_interface="vtnet0"
natd_flags="-dynamic -m"
openvpn_enable="YES"
openvpn_config="/usr/local/etc/openvpn.conf"
openvpn_if="tun"
openvpn_if="tap"
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2015-09-15 03:29:14 UTC
Assign to maintainer.

Brad, for future issues, prefixing the title with "category/portname: " will automatically assign it to the correct/current MAINTAINER

Also, please create attachments instead of inline comments for information that is longer than a few lines

Thanks for your report!
Comment 4 Matthias Andree freebsd_committer freebsd_triage 2015-09-15 07:03:05 UTC
Brad,

this looks like an issue specific to your server rather than FreeBSD - there is more than one OpenVPN rig based on FreeBSD in the world, and I am not aware of any large-scale issues.

To permit us to help you, please:

- make sure that the server knows its IP address - either I've overlooked it, or your rc.conf does not provide any cues about 108.61.175.20 or the network.
This seems to be the culprit.

If that's not it:

- report the output of ifconfig -a -v on the server and the equivalent on the client

- prove that the client can reach the server via ICMP echo, UDP and TCP on suitable ports

- report your operating system's version of the server, per: env -i uname -a ; freebsd-version -u

- retry with a Unix (preferred) or Windows 7 client in order to rule out incompatibilities on the Windows 10 client side
(I cannot debug anything but trivial setups from a Windows 7 client to a FreeBSD 10 server)

- retry with external certificate files (you need to put them in the right path and possibly give the --cd option on the command-line) - I have only ever used inline data on Android clients with Android's specific version as provided by Arne Schwabe.

- double-check anything that has to do with the 169.254.x.y APIPA range. This usually has no business with VPNs.

- double-check the necessary certificates and keys and other TLS data are matched properly

- upload the configuration files with comment lines stripped for clarity

- upload the pasted logs as attachments to the bug report else they will be dropped before we may be able to deal with them

- upload packet captures in a format that tshark/wireshark can dissect and without truncation

- finally, reassign the bug report to me.

Bottom line: it is unclear that OpenVPN is to fault here.
Comment 5 Matthias Andree freebsd_committer freebsd_triage 2015-09-15 07:06:57 UTC
I see that the server is using VirtIO - make sure that the virtual instance actually sees all necessary network traffic, and possibly retry on bare metal to rule out hypervisor and/or configuration issues.
Comment 6 brad 2015-09-15 12:42:58 UTC
@Matthias Andree  108.61.175.20 is a routable IP fully accessible fro anywhere on the internet. Network routing issues were eliminated in the first 2 seconds. Did you think to try and ping it brother?

specifically it is a vultr vps, business class not total junk  (keep in mind, more than one server is failing here)



[\u@vader:/root] # ifconfig -a -v
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c03bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 56:00:00:05:72:d5
        inet6 2001:19f0:7400:84c6::64 prefixlen 64
        inet6 fe80::5400:ff:fe05:72d5%vtnet0 prefixlen 64 scopeid 0x1
        inet 108.61.175.20 netmask 0xffffff00 broadcast 108.61.175.255
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
        groups: pflog
tap0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:1c:5d:00:00
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
        groups: tap
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::2bd:1cff:fe5d:0%tun0 prefixlen 64 scopeid 0x5
        inet 10.8.0.1 --> 10.8.0.2 netmask 0xffffff00
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: tun
        Opened by PID 3609





[\u@r2d2:/root] # freebsd-version -u
10.1-RELEASE-p5



- retry with a Unix (preferred)
12 people on the irc support channel #openvpn had the same issue while trying to assist me. I'll do another one just for you.


- retry with external certificate files:
on 4 different retries using easy-rsa and this method via copy/paste to cli  https://openvpn.net/index.php/open-source/documentation/howto.html  the keys were produce. the server has zero issues finding the file. zero 'file not found' error in server log. You can force an error here (rename the file to some silly test name, see it upchuck an error, correct the issues and error is gone). nope, zero issues finding or reaching the keyfile.
We ALSO deleted the keys and created a 5th set using openssl, same error in the end.  keys from both easy-rsa and openssl have been applied. Neither worked for us.


- double-check anything that has to do with the 169.254.x.y
I already addressed this routable network issue.



- double-check the necessary certificates and keys and other TLS
other than producing keys via easy-rsa or openssl are the only 2 methods to test this. Both methods have been used.


- upload the configuration files with comment lines stripped for clarity
o.0   did you even look at the bug report or did you just copy/paste this list of questions where over half of them have already been answered?


- upload the pasted logs as attachments to the bug report
see question just answered

- upload packet captures in a format that tshark/wireshark
once again, you didn't read the posted data, wireshark is already there. Its only port 1194 filtered, I couldn't make it any easier than that brother, not without reading it for you.
Comment 7 brad 2015-09-15 12:45:58 UTC
Note:
We beat up tun for abut 28 hrs. 
eventually we tried tap and tcp or any combination there of.
Basically it just changed the verbosity of the error. always failing with the same tls handshake.
Comment 8 brad 2015-09-15 12:56:23 UTC
server side tcpdump

[\u@vader:/root] # tcpdump -i vtnet0 udp port 1194
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
6 packets received by filter
0 packets dropped by kernel
[\u@vader:/root] # tcpdump -i vtnet0 udp port 1194
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet0, link-type EN10MB (Ethernet), capture size 65535 bytes



12:55:25.096968 IP 32.209.66.82.52617 > 108.61.175.20.vultr.com.openvpn: UDP, length 54
12:55:27.349876 IP 32.209.66.82.52617 > 108.61.175.20.vultr.com.openvpn: UDP, length 54
12:55:30.736073 IP 32.209.66.82.52617 > 108.61.175.20.vultr.com.openvpn: UDP, length 54
12:55:39.645217 IP 32.209.66.82.52617 > 108.61.175.20.vultr.com.openvpn: UDP, length 54
Comment 9 brad 2015-09-15 12:59:07 UTC
[\u@vader:/root] # netstat -na |grep 1194
udp4       0      0 *.1194                 *.*


we also tried assigning ip address to local and tried switching to well used ports such as 443
Comment 10 brad 2015-09-15 16:17:59 UTC
[\u@vader:/root] # netstat -na |grep 1194
udp4       0      0 *.1194                 *.*


we also tried assigning ip address to local and tried switching to well used ports such as 443

verb 9 debugging from a FreeBSD client confirms initial prognosis of server failing to respond

https://bpaste.net/show/d122bd3831b4
Comment 11 Matthias Andree freebsd_committer freebsd_triage 2015-09-16 07:21:05 UTC
I'll say it again, for one last time:

Do NOT post links to logs where they expire, I'll close the bug report as invalid with insufficient information otherwise.

Instead, use "Add an attachment" to upload and attach the logs to the bug report.


Now, more to the point, do you have a chance to use an emulated network card, rather than the VirtIO one?  It may not be as efficient, but let's see if we can nail at which stage the problem happens.

Now, "failure to respond", the log linked to in comment #10 (please upload to the bug report) reads as though the server is trying to talk to itself in both roles, and a "failure to repond" can also happen on implausible data during handshake.

Try removing the "local" statement. Try removing the statements that list [inline] for ca and thereabouts. It should suffice to include the <ca>...</ca> sections.

Be sure to match all relevant parameters on servers and client. I don't see a cipher statement on the client, for example. 

Also, be sure to check the manual pages, nearly half of the configuration parameters you are giving are redundant with what openvpn does internally after, for instance, a "server IP NETMASK" statement.

I still fail to see how this is an OpenVPN-on-FreeBSD specific issue that warrants a bug report, rather than a configuration issue.
Comment 12 Matthias Andree freebsd_committer freebsd_triage 2015-09-16 07:24:40 UTC
From your log:

Mon Sep 14 20:39:51 2015 us=453034 32.209.66.82:65437 Authenticate/Decrypt packet error: packet HMAC authentication failed
Mon Sep 14 20:39:51 2015 us=453047 32.209.66.82:65437 TLS Error: incoming packet authentication failed from [AF_INET]32.209.66.82:65437
...
Mon Sep 14 20:39:51 2015 us=453189 32.209.66.82:65437 Fatal TLS error (check_tls_errors_co), restarting

That pretty much sums it up.  Double-check your TLS data to make sure key/certs and everything is matched up properly between client and server.
Comment 13 Matthias Andree freebsd_committer freebsd_triage 2015-09-16 07:26:12 UTC
(In reply to Matthias Andree from comment #12)
(...meaning that the server won't respond due to the HMAC mismatch...)

So can we close this report as invalid because it's a local configuration issue, or is there any substantiated evidence that it's really a software bug specific to FreeBSD?
Comment 14 Matthias Andree freebsd_committer freebsd_triage 2015-09-17 05:53:32 UTC
*** Bug 203146 has been marked as a duplicate of this bug. ***
Comment 15 Matthias Andree freebsd_committer freebsd_triage 2015-09-17 05:58:11 UTC
Do NOT file duplicate bug reports. 

If we believe this to happen deliberately, we'll consider this spam, and I will also close this bug report.  

Personally, I will NOT respond to pressure.  
Remember this is a volunteer effort - trying to abuse it will not improve anything.

Given the other findings and unanswered questions below, I am considering closing this bug report as describing a local configuration issue with "works as intended".
Comment 16 Matthias Andree freebsd_committer freebsd_triage 2015-09-17 05:58:35 UTC
reopen inadvertently closed PR
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2015-12-13 06:22:47 UTC
@Matthias, I'm not sure if it was accidental or not, but in the event that it was intended, we don't assign issues to issue reporters unless they are also committers and you're happy to have them resolve the issue as they see fit. 

From my reading of this issue, we are currently at the point where the issue has been isolated (potentially not agreed) to the local environment and/or configuration, rather than the port or software itself.

Given the investigation that has taken place and the current conclusion, I will close this issue with an appropriate resolution.

@Brad, if you obtain further evidence of a fault either in FreeBSD itself or the OpenVPN port, we welcome you to re-open the issue at any time to provide (as attachments) that additional information.