Hey everyone! I'm having trouble getting a wg interface on my NetBSD to work. So, I think I set everything up according to this Mail: https://mail-index.netbsd.org/current-users/2020/08/20/msg039393.html Compiled a -CURRENT kernel ( NetBSD myx.crabdance.com 9.99.93 NetBSD 9.99.93 (MYX) #1: Thu Dec 30 10:33:30 UTC 2021 sysbuild@myx.crabdance.com:/usr/src/sys/arch/amd64/compile/MYX amd64 ) including the wg pseudo-device. Then I set up the device like this, trying to get a tunnel up to a linux wireguard box: ifconfig wg0 create ifconfig wg0 inet 192.168.0.102 netmask 255.255.255.0 ifconfig wg0 inet6 fdeb:4a6:eeb6:1::102 prefixlen 64 ifconfig wg0 up wgconfig wg0 set private-key /root/wgkey wgconfig wg0 set listen-port 45340 wgconfig wg0 add peer linuxpeer ajvowg1234567890123456789012345678901234567= --allowed-ips=192.168.0.1/32,fdeb:4a6:eeb6:1::100/128 --endpoint=1.2.3.4:45340 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } Now I have my device like this: # ifconfig wg0 wg0: flags=0x8041<UP,RUNNING,MULTICAST> mtu 1420 status: active inet6 fe80::c838:34f1:ff28:f96d%wg0/64 flags 0 scopeid 0x6 inet6 fdeb:4a6:eeb6:1::102/64 flags 0 inet 192.168.0.102/24 flags 0 # wgconfig wg0 interface: wg0 private-key: (hidden) listen-port: 45340 peer: linuxpeer public-key: ajvowg1234567890123456789012345678901234567= endpoint: 1.2.3.4:45340 preshared-key: (hidden) allowed-ips: 192.168.0.1/32,fdeb:4a6:eeb6:1::100/128 latest-handshake: (never) # route -n show Routing tables Internet: Destination Gateway Flags Refs Use Mtu Interface ... 192.168.0/24 192.168.0.102 U - - - wg0 192.168.0.102 wg0 UHl - - - wg0 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } but I can ping neither the remote (just lists packet loss) nor myself: # ping -c 4 -v 192.168.0.102 PING 192.168.0.102 (192.168.0.102): 56 data bytes ping: sendto: No route to host ping: sendto: No route to host if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } tcpdump show echo requests on wg0, but nothing else. Did anyone get wg running? Am I missing something? Shouldn't pinging the local address work at least? I'm kinda out of ideas 😃 Appreciate any pointers, I can't find further documentation either.

pickles # ping -c 4 -v 192.168.0.102 PING 192.168.0.102 (192.168.0.102): 56 data bytes ping: sendto: No route to host ping: sendto: No route to host if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } Did you assign a route for this interface?

pickles Can you explicitly add a gateway route for 192.168.0/24, then? Something like: # route delete -net 192.168.0.0 # route add -net 192.168.0.0 -netmask 255.255.255.0 192.168.0.102 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } Or possibly a direct one using -interface : # route add -net 192.168.0.0 -netmask 255.255.255.0 -interface wg0 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); }

I dug a little into the supposed wireguard protocol traffic, and tcpdump'ed on the wire interface on both ends. So, what's supposed to happen, once I run traffic from the netbsd box (192.168.0.102) to the linux peer (192.168.0.1), is encrypted UDP traffic flowing to 1.2.3.4:45340, to initiate the handshake between both hosts. What I see is UDP packets going from netbsd to the linux box: # tcpdump -i vioif0 host 1.2.3.4 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vioif0, link-type EN10MB (Ethernet), capture size 262144 bytes 00:50:57.195727 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148 00:51:02.697201 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148 00:51:08.207179 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } and I also see them arriving: linux# tcpdump -i any host 85.235.67.103 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 00:52:08.812794 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148 00:52:14.322555 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148 if(window.hljsLoader && !document.currentScript.parentNode.hasAttribute('data-s9e-livepreview-onupdate')) { window.hljsLoader.highlightBlocks(document.currentScript.parentNode); } but nothing ever flows back. If I try to initiate the other way around and ping from linux to netbsd, I see UDP packets going the other way but still no answers. So my guess is the wireguard implementations are incompatible and either box just discards the other ones packets.

wg on NetBSD issue

pickles

rvp Isn't that done by the ifconfig and present in the routing table above? I'm fairly new to BSD routing, but UHl does read that the Host "lives" on wg0, right? (being a local host route)
I also see the ICMP requests on wg0, but no answers.

rvp

pickles What does netstat -nr show?

pickles

rvp same as the route output i guess, i abbreviated it a bit cause i thought the lo0 routes etc would be boring. here's the full (ipv4) output:

# netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use    Mtu Interface
default            85.235.64.1        UGS         -        -      -  vioif0
85.235.64/22       link#1             UC          -        -      -  vioif0
85.235.67.103      link#1             UHl         -        -      -  lo0
127/8              127.0.0.1          UGRS        -        -  33624  lo0
127.0.0.1          lo0                UHl         -        -  33624  lo0
192.168.0/24       192.168.0.102      U           -        -      -  wg0
192.168.0.102      wg0                UHl         -        -      -  wg0
85.235.64.3        2c:6b:f5:a0:77:c0  UHL         -        -      -  vioif0
85.235.64.1        00:00:5e:00:01:01  UHL         -        -      -  vioif0

rvp

pickles I haven't used WG, so I don't know much about it, and so I don't know what happens to ping packets to the local end of a WG tunnel (192.168.0.102): forwarded or not--no idea. Are packets to other hosts in that network (192.168.0/24) sent through?

pickles

no, the peer (192.168.0.1 in this case) is just 100% packet loss :/

rvp

pickles Can you explicitly add a gateway route for 192.168.0/24, then?
Something like:

# route delete -net 192.168.0.0
# route add -net 192.168.0.0 -netmask 255.255.255.0 192.168.0.102

Or possibly a direct one using -interface:

# route add -net 192.168.0.0 -netmask 255.255.255.0 -interface wg0

pickles

rvp tried that, now I get "no route to host" for 192.168.0.1, too 🙁

pickles

rvp Or possibly a direct one using -interface:

that doesn't even let me set the route route: wg0: bad value

pickles

I dug a little into the supposed wireguard protocol traffic, and tcpdump'ed on the wire interface on both ends.
So, what's supposed to happen, once I run traffic from the netbsd box (192.168.0.102) to the linux peer (192.168.0.1), is encrypted UDP traffic flowing to 1.2.3.4:45340, to initiate the handshake between both hosts.
What I see is UDP packets going from netbsd to the linux box:

# tcpdump -i vioif0 host 1.2.3.4
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vioif0, link-type EN10MB (Ethernet), capture size 262144 bytes
00:50:57.195727 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148
00:51:02.697201 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148
00:51:08.207179 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148

and I also see them arriving:

linux# tcpdump -i any host 85.235.67.103
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
00:52:08.812794 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148
00:52:14.322555 IP myx.crabdance.com.45340 > 1.2.3.4.45340: UDP, length 148

but nothing ever flows back.

If I try to initiate the other way around and ping from linux to netbsd, I see UDP packets going the other way but still no answers.
So my guess is the wireguard implementations are incompatible and either box just discards the other ones packets.

rvp

pickles that doesn't even let me set the route route: wg0: bad value

The syntax may be different on NetBSD--I'll check.

In this command:

wgconfig wg0 add peer linuxpeer ...  --endpoint=1.2.3.4:45340

you're not literally using 1.2.3.4, right? That's just the address of the peer in the example.

pickles

rvp yeah, no, i changed some IP addresses and mangled the keys in output here to protect the innocent 😉

add.: also UDP traffic does arrive at my peer. it just doesn't answer. and the other way around, too. wireguard is exceptionally bad at logging, unfortunatly.

rvp

pickles add.: also UDP traffic does arrive at my peer. it just doesn't answer. and the other way around, too.

I think the problem may simply be a too restrictive allowed-ips parameter. Change it to cover

a) both end-points, or
b) even the whole of that /24 subnet range:

... allowed-ips=10.0.1.1/32,10.0.1.0/32 ...

... allowed-ips=10.0.1.0/24 ...

You may have to do this on both ends (not sure how Linux handles this; it certainly is needed if the end-point hosts are both NetBSD)

pickles

rvp I think the problem may simply be a too restrictive allowed-ips parameter.

"allowed IPs" in wireguard terminology are for building the internal routing, so wireguard knows what traffic to direct to what node (wireguard is a lot of point-to-point links). if two nodes have the same network wireguard doesn't know where to direct traffic anymore.

I did try to change it to /24 nonetheless, but it didn't work out either.

rvp

pickles "allowed IPs" in wireguard terminology are for building the internal routing,...

Ah, I see...but, that was the thing that finally "worked" (ie. got bi-directional traffic flow) on my single-machine network test setup. However, running everything on localhost is not ideal and I might have messed up somewhere.

OK, send an email to me (same username) at SDF Unix and we can try with a more realistic setup. I'll do the NetBSD end, and you can do the Linux server end.

EDIT: One other thing I noticed (at least on my NetBSD localhost setup) was that you need to send one ping from the server host-end to VPN IP of the client end-point before 2-way traffic can flow. The other way around (client to server as shown in the wg(4) man-page) doesn't work.