Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dco_get_peer_stats fails at startup #643

Closed
ordex opened this issue Nov 18, 2024 · 50 comments
Closed

dco_get_peer_stats fails at startup #643

ordex opened this issue Nov 18, 2024 · 50 comments

Comments

@ordex
Copy link
Member

ordex commented Nov 18, 2024

2024-11-18 02:32:30 OpenVPN 2.6.12 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-11-18 02:32:30 library versions: OpenSSL 3.0.13 30 Jan 2024, LZO 2.10
2024-11-18 02:32:30 DCO version: 0.2.20240712
2024-11-18 02:32:30 net_route_v4_best_gw query: dst 0.0.0.0
2024-11-18 02:32:30 net_route_v4_best_gw result: via 87.90.28.1 dev eth0
2024-11-18 02:32:30 WARNING: experimental option --capath ca
2024-11-18 02:32:30 net_iface_new: add FRAvpn type ovpn-dco
2024-11-18 02:32:30 DCO device FRAvpn opened
2024-11-18 02:32:30 net_iface_mtu_set: mtu 1420 for FRAvpn
2024-11-18 02:32:30 net_iface_up: set FRAvpn up
2024-11-18 02:32:31 net_addr_ptp_v4_add: 10.11.3.1 peer 10.11.3.2 dev FRAvpn
2024-11-18 02:32:31 net_iface_mtu_set: mtu 1420 for FRAvpn
2024-11-18 02:32:31 net_iface_up: set FRAvpn up
2024-11-18 02:32:31 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-18 02:32:31 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)
2024-11-18 02:32:41 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-18 02:32:41 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

linux-image-6.11.8-x64v4-xanmod1

Originally posted by @liuxyon in OpenVPN/ovpn-dco#50 (comment)

@liuxyon
Copy link

liuxyon commented Nov 19, 2024

I may have found the cause of the problem, a permission issue. Using DCO requires ROOT permissions.

user root
group root

Now I can connect to ubuntu 24.0.4, but I still have problems connecting to pfsense freebsd openvpn DCO

Nov 19 11:40:30 openvpn 33936 SIGUSR1[soft,process-push-msg-failed] received, process restarting
Nov 19 11:40:30 openvpn 33936 Failed to open tun/tap interface
Nov 19 11:40:30 openvpn 33936 ERROR: Failed to apply push options
Nov 19 11:40:30 openvpn 33936 OPTIONS ERROR: pushed options are incompatible with data channel offload. Use --disable-dco to connect to this server
Nov 19 11:40:30 openvpn 33936 OPTIONS IMPORT: Server did not request DATA_V2 packet format required for data channel offload
Nov 19 11:40:28 openvpn 33936 [tv1.com] Peer Connection Initiated with [AF_INET6]
Nov 19 11:40:28 openvpn 33936 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 256 bits ED25519, signature: ED25519, peer temporary key: 253 bits X25519

@liuxyon
Copy link

liuxyon commented Nov 19, 2024

@ordex
Copy link
Member Author

ordex commented Nov 19, 2024

Using DCO requires ROOT permissions.

yes, normally you'd start openvpn as root, unless you're dropping privileges later.
Alternatively, you can provide the openvpn binary with the CAP_NET_ADMIN capability and avoid running as root.

@ordex ordex closed this as completed Nov 19, 2024
@liuxyon
Copy link

liuxyon commented Nov 20, 2024

After I set the permissions,It seems there is still a problem

2024-11-20 11:37:08 UID set to root
2024-11-20 11:37:08 GID set to root
2024-11-20 11:37:08 Capabilities retained: CAP_NET_ADMIN
2024-11-20 11:37:08 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 11:37:08 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)
2024-11-20 11:37:18 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 11:37:18 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)
2024-11-20 11:37:28 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 11:37:28 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

sudo setcap cap_setpcap,cap_net_admin,CAP_IPC_LOCK,CAP_NET_BIND_SERVICE,CAP_SETGID,CAP_SETUID,CAP_SYS_CHROOT,CAP_DAC_OVERRIDE,CAP_AUDIT_WRITE,cap_net_raw+ep /usr/sbin/openvpn
root@v7251:~# getcap /usr/sbin/openvpn
/usr/sbin/openvpn cap_dac_override,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_sys_chroot,cap_audit_write=ep

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

If I remove CAP_SETPCAP , then log show 12:13:59 --user specified but lacking CAP_SETPCAP. Cannot retain CAP_NET_ADMIN. Disabling data channel offload
when i add CAP_SETPCAP, log show
dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

So the problem is not solved.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

Setting up ROOT did not solve the problem.

user root
group root

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

this added CAP_SETPCAP

2024-11-20 13:22:24 us=494734 Current Parameter Settings:
2024-11-20 13:22:24 us=494818 config = '/etc/openvpn/server.conf'
2024-11-20 13:22:24 us=494826 mode = 0
2024-11-20 13:22:24 us=494836 persist_config = DISABLED
2024-11-20 13:22:24 us=494842 persist_mode = 1
2024-11-20 13:22:24 us=494848 show_ciphers = DISABLED
2024-11-20 13:22:24 us=494853 NOTE: --mute triggered...
2024-11-20 13:22:24 us=494867 288 variation(s) on previous 20 message(s) suppressed by --mute
2024-11-20 13:22:24 us=494873 OpenVPN 2.6.12 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-11-20 13:22:24 us=494888 library versions: OpenSSL 3.0.13 30 Jan 2024, LZO 2.10
2024-11-20 13:22:24 us=494922 DCO version: 0.2.20240712
2024-11-20 13:22:24 us=495100 net_route_v4_best_gw query: dst 0.0.0.0
2024-11-20 13:22:24 us=495153 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=495168 sitnl_send: rtnl: received 792 bytes
2024-11-20 13:22:24 us=495187 net_route_v4_best_gw result: via 77.80.28.1 dev eth0
2024-11-20 13:22:24 us=496697 tls-crypt-v2 server key: Cipher 'AES-256-CTR' initialized with 256 bit key
2024-11-20 13:22:24 us=496721 tls-crypt-v2 server key: Using 256 bit message hash 'SHA256' for HMAC authentication
2024-11-20 13:22:24 us=496897 Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
2024-11-20 13:22:24 us=497002 open_tun_dco: FRAvpn
2024-11-20 13:22:24 us=497010 net_iface_new: add FRAvpn type ovpn-dco
2024-11-20 13:22:24 us=497631 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=497668 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=497706 ovpn_dco_register
2024-11-20 13:22:24 us=497795 DCO device FRAvpn opened
2024-11-20 13:22:24 us=497804 do_ifconfig, ipv4=1, ipv6=1
2024-11-20 13:22:24 us=497822 net_iface_mtu_set: mtu 1430 for FRAvpn
2024-11-20 13:22:24 us=638386 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=638480 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=638557 net_iface_up: set FRAvpn up
2024-11-20 13:22:24 us=734011 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=734110 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=734144 net_addr_ptp_v4_add: 10.11.3.1 peer 10.11.3.2 dev FRAvpn
2024-11-20 13:22:24 us=811193 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=811294 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=811371 net_iface_mtu_set: mtu 1430 for FRAvpn
2024-11-20 13:22:24 us=811406 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=811413 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=811431 net_iface_up: set FRAvpn up
2024-11-20 13:22:24 us=811482 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=811489 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=811499 net_addr_v6_add: 2a0d:2408:513:a::2/124 dev FRAvpn
2024-11-20 13:22:24 us=811703 sitnl_send: checking for received messages
2024-11-20 13:22:24 us=811715 sitnl_send: rtnl: received 36 bytes
2024-11-20 13:22:24 us=811816 Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1430 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-11-20 13:22:24 us=812083 Socket Buffers: R=[67108864->67108864] S=[67108864->67108864]
2024-11-20 13:22:24 us=812095 setsockopt(IPV6_V6ONLY=0)
2024-11-20 13:22:24 us=812116 UDPv6 link local (bound): [AF_INET6]2a04:e9c0:18:71a::1:51758
2024-11-20 13:22:24 us=812122 UDPv6 link remote: [AF_UNSPEC]
2024-11-20 13:22:24 us=812157 UID set to root
2024-11-20 13:22:24 us=812166 GID set to root
2024-11-20 13:22:24 us=812172 Capabilities retained: CAP_NET_ADMIN
2024-11-20 13:22:24 us=812188 dco_get_peer_stats: peer-id -1
2024-11-20 13:22:24 us=812226 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 13:22:24 us=812234 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

2024-11-20 13:22:24 us=812188 dco_get_peer_stats: peer-id -1

this seems a different problem.
Can you share the config file? It seems openvpn is trying to fetch the stats of the peer before having configured any peer.

@ordex ordex reopened this Nov 20, 2024
@liuxyon
Copy link

liuxyon commented Nov 20, 2024

mode p2p
#dco
local 2a04:e8c:18:71a::1
port 51758
proto udp6
dev-type tun
dev FRAvpn
sndbuf 0
rcvbuf 0
tun-mtu 1430
txqueuelen 10000
reneg-sec 86400
tls-groups X25519
tls-server
data-ciphers AES-256-GCM:AES-128-GCM
data-ciphers-fallback AES-128-GCM
remote-cert-tls client
ca ca.crt
cert tv1.com.crt
key tv1.com.key
float
dh none
auth SHA3-256
tls-crypt-v2 v2crypt-server.key
topology p2p
ifconfig 10.11.3.1 10.11.3.2
ifconfig-ipv6 2a0d:2408:513:a::2/124 2a0d:2408:513:a::3
keepalive 10 60
pull-filter ignore peer-id
ping-timer-rem
allow-compression no
user root
group root
persist-key
persist-tun
status openvpn-status.log
log openvpn.log
verb 6
mute 20
fast-io
tls-version-min 1.3

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

other side pfsense openvpn log show:

Nov 20 17:10:42 | openvpn | 85204 | OPTIONS IMPORT: Server did not request DATA_V2 packet format required for data channel offload
Nov 20 17:10:42 | openvpn | 85204 | PUSH: Received control message: 'PUSH_REPLY,cipher AES-256-GCM,tun-mtu 1430'
Nov 20 17:10:42 | openvpn | 85204 | SENT CONTROL [tv1.com]: 'PUSH_REQUEST' (status=1)
Nov 20 17:10:41 | openvpn | 85204 | TLS: tls_multi_process: initial untrusted session promoted to trusted
Nov 20 17:10:41 | openvpn | 85204 | TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

I am not fully understanding. In the config you posted you wrote mode p2p, how come the other side is doing push-request/reply? These are specific for client/server mode, not for p2p

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

is the other side configured as client?

@cron2
Copy link
Contributor

cron2 commented Nov 20, 2024

This is... interesting. So, for DCO to work, you need DATA_V2 packet format, which is enabled in server/client mode by the server pushing a peer-id to the client. In p2p mode, everything recent enough to support DCO(!) should create a random peer-id itself, and move over to DATA_V2.

Seeing the pfsense side send a PUSH_REQUEST, my guess right now is that the pfsense is using client while we can see from the other side config that there is no server but mode p2p. This will not work with DCO - if you want p2p, both sides need to agree on using mode p2p.

This all said, I think you are triggering a bug on the mode p2p, tls-server side - it wants to do DCO, but it fails since the p2p peer-id negotiation did not take place (yet). I'm a bit surprised that it fails before the other side even connects, while you do have a log from the pfsense that shows a successful TLS handshake - so the logs do not really match (as can be seen by the timestamps).

Anyway, we should look into that case (that is, mode p2p, tls-serverand error at startup / or error on connect from a client client)

@cron2
Copy link
Contributor

cron2 commented Nov 20, 2024

is the other side configured as client?

My crystal ball says it is ;-)

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

I am not fully understanding. In the config you posted you wrote mode p2p, how come the other side is doing push-request/reply? These are specific for client/server mode, not for p2p

i using openvpn for p2p mode tunnel.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

pfsense side config

dev ovpnc8
verb 3
dev-type tun
dev-node /dev/tun8
writepid /var/run/openvpn_client8.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp6
auth SHA3-256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local 2418:827:404:ac0:ecf5:aaff:fe25:4394
tls-client
lport 0
management /var/etc/openvpn/client8/sock unix
remote 2a04:e8c:18:71a::1 51758 udp6
pull
remote-cert-tls server
capath /var/etc/openvpn/client8/ca
cert /var/etc/openvpn/client8/cert
key /var/etc/openvpn/client8/key
data-ciphers AES-256-GCM:AES-128-GCM
data-ciphers-fallback AES-128-GCM
allow-compression no
resolv-retry infinite
tls-crypt-v2 /root/v2crypt-client-1.key
ifconfig 10.11.3.2 10.11.3.1
ifconfig-ipv6 2a0d:2408:513:a::3/124 2a0d:2408:513:a::2
pull-filter ignore peer-id
tun-mtu 1430
float

Ewd5AAo 1

@schwabe
Copy link
Contributor

schwabe commented Nov 20, 2024

pull
That is a client onfig and not a p2p mode config

pull-filter ignore peer-id

That will most certainly cause problems.

tls-crypt-v2 /root/v2crypt-client-1.key

You will not be able to connnect to a p2p peer with tls-crypt-v2. I doubt that the config matches the log from the other side/config from the other side.

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

I doubt that the config matches the log from the other side/config from the other side.

I also have this feeling. Did you post your /etc/openvpn/server.conf file?

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

pull That is a client onfig and not a p2p mode config

pull-filter ignore peer-id

That will most certainly cause problems.

tls-crypt-v2 /root/v2crypt-client-1.key

You will not be able to connnect to a p2p peer with tls-crypt-v2. I doubt that the config matches the log from the other side/config from the other side.

i try deleted the two tls-crypt-v2 /root/v2crypt-client-1.key pull-filter ignore peer-id The problem is still not solved

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

I doubt that the config matches the log from the other side/config from the other side.

I also have this feeling. Did you post your /etc/openvpn/server.conf file?

yes, i had post it.

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

I also have this feeling. Did you post your /etc/openvpn/server.conf file?

yes, i had post it.

it seems that the configuration you have posted and the log do not truly match.

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

2024-11-20 13:22:24 us=812234 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

question: does openvpn stop after this message? Or it simply continue to run?

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

btw make sure to remove the pull directive on BSD too. These options are not truly expected to be mixed this way.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

this now pfsense side config, some ips hide.

dev ovpnc8
verb 3
dev-type tun
dev-node /dev/tun8
writepid /var/run/openvpn_client8.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp6
auth SHA3-256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local
tls-client
lport 0
management /var/etc/openvpn/client8/sock unix
remote 2a:e8:18:71a::1 51758 udp6
pull
remote-cert-tls server
capath /var/etc/openvpn/client8/ca
cert /var/etc/openvpn/client8/cert
key /var/etc/openvpn/client8/key
data-ciphers AES-256-GCM:AES-128-GCM
data-ciphers-fallback AES-128-GCM
allow-compression no
resolv-retry infinite
ifconfig 10.11.3.2 10.11.3.1
ifconfig-ipv6 2a0d:2408:513:a::3/124 2a0d:2408:513:a::2
tun-mtu 1430
float

this ubuntu side cnfig:

#mode p2p

port 51758
proto udp6
#proto-force udp
#mode server
#multihome

dev-type tun
dev FRAvpn
#sndbuf 0
#rcvbuf 0

#explicit-exit-notify 1

tun-mtu 1430
txqueuelen 10000
reneg-sec 86400
tls-groups X25519
tls-server
data-ciphers AES-256-GCM:AES-128-GCM
data-ciphers-fallback AES-128-GCM

remote-cert-tls client
ca ca.crt
cert tv1.com.crt
key tv1.com.key

float
dh none

auth SHA3-256

#tls-crypt-v2 v2crypt-server.key

#topology p2p
#topology subnet

ifconfig 10.11.3.1 10.11.3.2
ifconfig-ipv6 2a0d:2408:513:a::2/124 2a0d:2408:513:a::3

keepalive 10 60
ping-timer-rem
allow-compression no
user root
group root
persist-key
persist-tun
status openvpn-status.log
log openvpn.log
verb 6
mute 20
tls-version-min 1.3

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

btw make sure to remove the pull directive on BSD too. These options are not truly expected to be mixed this way.

yes, i had remove pull

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

ubuntu side log:

2024-11-20 18:50:30 us=322724 Current Parameter Settings:
2024-11-20 18:50:30 us=322801 config = '/etc/openvpn/server.conf'
2024-11-20 18:50:30 us=322808 mode = 0
2024-11-20 18:50:30 us=322813 persist_config = DISABLED
2024-11-20 18:50:30 us=322818 persist_mode = 1
2024-11-20 18:50:30 us=322823 show_ciphers = DISABLED
2024-11-20 18:50:30 us=322828 NOTE: --mute triggered...
2024-11-20 18:50:30 us=322845 286 variation(s) on previous 20 message(s) suppressed by --mute
2024-11-20 18:50:30 us=322851 OpenVPN 2.6.12 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-11-20 18:50:30 us=322863 library versions: OpenSSL 3.3.1 4 Jun 2024, LZO 2.10
2024-11-20 18:50:30 us=322896 DCO version: 0.2.20240712
2024-11-20 18:50:30 us=323105 net_route_v4_best_gw query: dst 0.0.0.0
2024-11-20 18:50:30 us=323161 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=323174 sitnl_send: rtnl: received 792 bytes
2024-11-20 18:50:30 us=323194 net_route_v4_best_gw result: via 7.90.28.1 dev eth0
2024-11-20 18:50:30 us=324597 Control Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1250 tun_max_mtu:0 headroom:126 payload:1600 tailroom:126 ET:0 ]
2024-11-20 18:50:30 us=324710 open_tun_dco: FRAvpn
2024-11-20 18:50:30 us=324718 net_iface_new: add FRAvpn type ovpn-dco
2024-11-20 18:50:30 us=325187 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=325202 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=325222 ovpn_dco_register
2024-11-20 18:50:30 us=325968 DCO device FRAvpn opened
2024-11-20 18:50:30 us=325981 do_ifconfig, ipv4=1, ipv6=1
2024-11-20 18:50:30 us=326009 net_iface_mtu_set: mtu 1430 for FRAvpn
2024-11-20 18:50:30 us=471847 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=471933 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=471999 net_iface_up: set FRAvpn up
2024-11-20 18:50:30 us=579927 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=580634 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=580727 net_addr_ptp_v4_add: 10.11.3.1 peer 10.11.3.2 dev FRAvpn
2024-11-20 18:50:30 us=664418 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=664527 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=664585 net_iface_mtu_set: mtu 1430 for FRAvpn
2024-11-20 18:50:30 us=664616 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=664621 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=664635 net_iface_up: set FRAvpn up
2024-11-20 18:50:30 us=664648 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=664653 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=664662 net_addr_v6_add: 2a0d:2408:513:a::2/124 dev FRAvpn
2024-11-20 18:50:30 us=664741 sitnl_send: checking for received messages
2024-11-20 18:50:30 us=664756 sitnl_send: rtnl: received 36 bytes
2024-11-20 18:50:30 us=664787 Data Channel MTU parms [ mss_fix:0 max_frag:0 tun_mtu:1430 tun_max_mtu:1600 headroom:136 payload:1768 tailroom:562 ET:0 ]
2024-11-20 18:50:30 us=665042 Socket Buffers: R=[67108864->67108864] S=[67108864->67108864]
2024-11-20 18:50:30 us=665052 setsockopt(IPV6_V6ONLY=0)
2024-11-20 18:50:30 us=665071 UDPv6 link local (bound): [AF_INET6][undef]:51758
2024-11-20 18:50:30 us=665076 UDPv6 link remote: [AF_UNSPEC]
2024-11-20 18:50:30 us=665102 UID set to root
2024-11-20 18:50:30 us=665106 GID set to root
2024-11-20 18:50:30 us=665111 Capabilities retained: CAP_NET_ADMIN
2024-11-20 18:50:30 us=665125 dco_get_peer_stats: peer-id -1
2024-11-20 18:50:30 us=665156 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 18:50:30 us=665165 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)
2024-11-20 18:50:40 us=188231 UDPv6 READ [14] from [AF_INET6]2418:827:404:ac0::58394: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
2024-11-20 18:50:40 us=188322 TLS: Initial packet from [AF_INET6]2418:827:404:ac0::58394, sid=767b5194 b90127d0
2024-11-20 18:50:40 us=188336 dco_get_peer_stats: peer-id -1
2024-11-20 18:50:40 us=188402 dco_get_peer_stats: netlink reports object not found, ovpn-dco unloaded?
2024-11-20 18:50:40 us=188410 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

2024-11-20 13:22:24 us=812234 dco_get_peer_stats: failed to send netlink message: No such file or directory (-2)

question: does openvpn stop after this message? Or it simply continue to run?

continue to run

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

if i want to using ifconfig 10.11.3.1 10.11.3.2, ifconfig-ipv6 2a0d:2408:513:a::2/124 2a0d:2408:513:a::3
How to configure it correctly?

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

continue to run

ok, then this is not a real problem. it's just the statistics polling that fails because no peer is yet connected. You can ignore that message

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

if i want to using ifconfig 10.11.3.1 10.11.3.2, ifconfig-ipv6 2a0d:2408:513:a::2/124 2a0d:2408:513:a::3

this looks good to me, as long as it's inverted on the other peer.

Please make sure to remove pull from the config as it is not expected to work with mode p2p

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

2024-11-20 18:50:40 us=188322 TLS: Initial packet from [AF_INET6]2418:827:404:ac0::58394, sid=767b5194 b90127d0

some connection is happening here on ubuntu. Please show the log from the pfsense side.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

2024-11-20 18:50:40 us=188322 TLS: Initial packet from [AF_INET6]2418:827:404:ac0::58394, sid=767b5194 b90127d0

some connection is happening here on ubuntu. Please show the log from the pfsense side.

Nov 20 19:16:15 openvpn 57098 UDPv6 link remote: [AF_INET6]2a04:e8c:18:71a::1:51758
Nov 20 19:16:15 openvpn 57098 UDPv6 link local (bound): [AF_INET6]2418:82:404:ac0::4394:0
Nov 20 19:16:15 openvpn 57098 setsockopt(IPV6_V6ONLY=0)
Nov 20 19:16:15 openvpn 57098 Socket Buffers: R=[33554432->33554432] S=[57344->57344]
Nov 20 19:16:15 openvpn 57098 TCP/UDP: Preserving recently used remote address: [AF_INET6]2a04:e8c:18:71a::1:51758
Nov 20 19:16:15 openvpn 57098 WARNING: experimental option --capath /var/etc/openvpn/client8/ca
Nov 20 19:16:15 openvpn 57098 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Nov 20 19:16:15 openvpn 57098 WARNING: using --pull/--client and --ifconfig together is probably not what you want
Nov 20 19:16:15 openvpn 57098 MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client8/sock
Nov 20 19:16:15 openvpn 56909 DCO version: FreeBSD 15.0-CURRENT #0 plus-RELENG_24_03-n256311-e71f834dd81: Fri Apr 19 00:28:14 UTC 2024 root@freebsd:/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/obj/amd64/Y4MAEJ2R/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/sources/FreeBS
Nov 20 19:16:15 openvpn 56909 library versions: OpenSSL 3.0.13 24 Oct 2023, LZO 2.10
Nov 20 19:16:15 openvpn 56909 OpenVPN 2.6.8 amd64-portbld-freebsd15.0 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [MH/RECVDA] [AEAD] [DCO]
Nov 20 19:16:15 openvpn 56909 Note: ignoring --dev-node as it has no effect when using data channel offload
Nov 20 19:16:15 openvpn 94504 SIGTERM[hard,] received, process exiting
Nov 20 19:16:15 openvpn 94504 event_wait : Interrupted system call (fd=-1,code=4)
Nov 20 19:16:09 openvpn 94504 MANAGEMENT: Client disconnected
Nov 20 19:16:09 openvpn 94504 MANAGEMENT: CMD 'state 1'
Nov 20 19:16:09 openvpn 94504 MANAGEMENT: Client connected from /var/etc/openvpn/client8/sock
Nov 20 19:16:09 openvpn 12964 MANAGEMENT: Client disconnected
Nov 20 19:16:09 openvpn 12964 MANAGEMENT: CMD 'status 2'
Nov 20 19:16:09 openvpn 12964 MANAGEMENT: CMD 'state 1'
Nov 20 19:16:09 openvpn 12964 MANAGEMENT: Client connected from /var/etc/openvpn/client5/sock

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

Nov 20 19:16:15 openvpn 57098 WARNING: using --pull/--client and --ifconfig together is probably not what you want

here it says that you are still using pull and something it taking down the process immediately

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

Nov 20 19:16:15 openvpn 57098 WARNING: using --pull/--client and --ifconfig together is probably not what you want

here it says that you are still using pull and something it taking down the process immediately

I did not configure additional pull parameters. pfsense only has

ifconfig 10.11.3.2 10.11.3.1
ifconfig-ipv6 2a0d:2408:513:a::3/124 2a0d:2408:513:a::2
tun-mtu 1430
tls-crypt-v2 /root/v2crypt-client-1.key

@schwabe
Copy link
Contributor

schwabe commented Nov 20, 2024

Your config is still extremely questionable. tls-crypt-v2 basically requires pull to actually work properly.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

Your config is still extremely questionable. tls-crypt-v2 basically requires pull to actually work properly.

Removed tls-crypt-v2, same result

Nov 20 20:23:03 openvpn 56039 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Nov 20 20:23:03 openvpn 56039 WARNING: using --pull/--client and --ifconfig together is probably not what you want
Nov 20 20:23:03 openvpn 56039 MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client8/sock
Nov 20 20:23:03 openvpn 55976 DCO version: FreeBSD 15.0-CURRENT #0 plus-RELENG_24_03-n256311-e71f834dd81: Fri Apr 19 00:28:14 UTC 2024 root@freebsd:/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/obj/amd64/Y4MAEJ2R/var/jenkins/workspace/pfSense-Plus-snapshots-24_03-main/sources/FreeBS

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

when i setup add mode server, ubuntu log show

us=458249 Note: NOT using '--topology subnet' disables data channel offload.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

when add topology subnet,ubuntu log show:

2024-11-20 20:40:32 us=695056 open_tun_dco: FRAvpn
2024-11-20 20:40:32 us=695066 net_iface_new: add FRAvpn type ovpn-dco
2024-11-20 20:40:32 us=695093 sitnl_send: checking for received messages
2024-11-20 20:40:32 us=695101 sitnl_send: rtnl: received 96 bytes
2024-11-20 20:40:32 us=695108 sitnl_send: rtnl: generic error (-17): File exists
2024-11-20 20:40:32 us=695116 Cannot create DCO interface FRAvpn: -17
2024-11-20 20:40:32 us=695121 DCO device FRAvpn already exists, won't be destroyed at shutdown
2024-11-20 20:40:32 us=695126 do_ifconfig, ipv4=1, ipv6=1
2024-11-20 20:40:32 us=695140 net_iface_mtu_set: mtu 1430 for FRAvpn
2024-11-20 20:40:32 us=695157 sitnl_send: checking for received messages
2024-11-20 20:40:32 us=695163 sitnl_send: rtnl: received 36 bytes
2024-11-20 20:40:32 us=695178 net_iface_up: set FRAvpn up
2024-11-20 20:40:32 us=695194 sitnl_send: checking for received messages
2024-11-20 20:40:32 us=695201 sitnl_send: rtnl: received 36 bytes
2024-11-20 20:40:32 us=695208 net_addr_v4_add: 10.11.3.1/-1 dev FRAvpn
2024-11-20 20:40:32 us=695227 sitnl_send: checking for received messages
2024-11-20 20:40:32 us=695232 sitnl_send: rtnl: received 52 bytes
2024-11-20 20:40:32 us=695236 sitnl_send: rtnl: generic error (-22): Invalid argument
2024-11-20 20:40:32 us=695242 Linux can't add IP to interface FRAvpn
2024-11-20 20:40:32 us=695251 Exiting due to fatal error

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

So now the question is how does DCO work in P2P mode?

@schwabe
Copy link
Contributor

schwabe commented Nov 20, 2024

@liuxyon it works fine in p2p mode but what you shown as configs and message you presented, suggests that you don't actually understand what p2p mode is. You want p2p mode but are saying messages like

when i setup add mode server, ubuntu log show

mode server is the opposite of mode p2p

Also your logs/config and what you are saying is all contradiction each other, so it really hard for us to make sense of what you are actually doing.

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

@liuxyon it works fine in p2p mode but what you shown as configs and message you presented, suggests that you don't actually understand what p2p mode is. You want p2p mode but are saying messages like

when i setup add mode server, ubuntu log show

mode server is the opposite of mode p2p

Also your logs/config and what you are saying is all contradiction each other, so it really hard for us to make sense of what you are actually doing.

I have tried all the settings you mentioned, and I have shown my configuration. That is to say, there is no known parameter configuration that can work p2p DCO mode

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

I have to use encrypted data packet headers, otherwise they will be recognized and blocked by GFW.
Error: TLS handshake failed

@schwabe
Copy link
Contributor

schwabe commented Nov 20, 2024

I have to use encrypted data packet headers, otherwise they will be recognized and blocked by GFW.
Error: TLS handshake failed

The concept of encrypted data packet headers does not exist in OpenVPN.

@ordex
Copy link
Member Author

ordex commented Nov 20, 2024

Nov 20 20:23:03 openvpn 56039 WARNING: using --pull/--client and --ifconfig together is probably not what you want

pfsense keeps showing this. it means that it is somehow enabling --pull again.
Maybe you should give up p2p and go with a normal clients/server setup.

However, please note that this is a bug tracker, not a support channel.
For support you can write to the user mailing list or in our IRC channel on libera.chat

@liuxyon
Copy link

liuxyon commented Nov 20, 2024

When DCO is not used, these configurations do not affect operation. Because I use it to establish BGP tunnel.It should be P2P mode.
pfsense says it's an openvpn code issue.
https://docs.netgate.com/pfsense/en/latest/vpn/openvpn/dco.html#openvpn-dco-limitations
Using a /30 or smaller tunnel network for peer-to-peer tunnels (one server with one client) is not compatible with DCO. There are problems with the code for this mode in OpenVPN which can lead to failed connections and instability.

where mailing list?

@ordex
Copy link
Member Author

ordex commented Nov 21, 2024

@ordex ordex closed this as completed Nov 21, 2024
@schwabe
Copy link
Contributor

schwabe commented Nov 21, 2024

@liuxyon we tested p2p mode with DCO and it works. However, we lost faith in your ability to setup it correctly, so we are no longer treating this issue as a bug report/issue but rather as a user configuration and as thus, it does not belong on this issue tracker.

@liuxyon
Copy link

liuxyon commented Nov 21, 2024

Give me your correct configuration, I didn't find anything about DCO p2p correct configuration on the Internet.

@ordex
Copy link
Member Author

ordex commented Nov 21, 2024

This issue is closed. Please write to the mailing list or join IRC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants