This post will demonstrate how and when the iroute directive is used in OpenVPN.
The problem
In most cases iroute is not needed, and in fact many users probably have never used it (or are aware of it, for that matter). It usually comes into play when networks behind the VPN nodes need to communicate. Let's imagine a topology like this:
Let's suppose that you want communication between networks A and B, and between A and C, as indicated by the dotted arrows. The config files are something like this:
gwA # cat /etc/openvpn/server.conf # gwA local 172.20.0.1 port 1194 proto udp dev tun topology subnet mode server tls-server ifconfig 10.0.0.1 255.255.255.0 route 192.168.2.0 255.255.255.0 10.0.0.2 route 192.168.3.0 255.255.255.0 10.0.0.3 client-config-dir ccd # snip rest of config gwA # cat /etc/openvpn/ccd/gwB ifconfig-push 10.0.0.2 255.255.255.0 push "route 192.168.1.0 255.255.255.0 10.0.0.1" gwA # cat /etc/openvpn/ccd/gwC ifconfig-push 10.0.0.3 255.255.255.0 push "route 192.168.1.0 255.255.255.0 10.0.0.1"
gwB # cat /etc/openvpn/client.conf # gwB remote 172.20.0.1 1194 proto udp dev tun topology subnet # snip rest of config
gwC # cat /etc/openvpn/client.conf # gwC remote 172.20.0.1 1194 proto udp dev tun topology subnet # snip rest of config
You think that having all the necessary routes in place as per the above configs would be enough to allow the desired communication, right? Well, let's bring up the VPN and try pinging from a computer in network A to network C:
box-in-a # ping 192.168.3.1 PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data. ^C --- 192.168.3.1 ping statistics --- 8 packets transmitted, 0 received, 100% packet loss, time 7012ms
Let's try from network C to network A:
box-in-c # ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. ^C --- 192.168.1.1 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 4017ms
How come? Why isn't it working? Forwarding is enabled on all gateways, routes are in place:
gwA # ip route show 192.168.3.0/24 via 10.0.0.3 dev tun0 192.168.2.0/24 via 10.0.0.2 dev tun0 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.254 10.0.0.0/24 dev tun0 proto kernel scope link src 10.0.0.1 [snip] gwC # ip proute show 10.0.0.0/24 dev tun0 proto kernel scope link src 10.0.0.3 192.168.3.0/24 dev eth0 proto kernel scope link src 192.168.3.254 192.168.1.0/24 via 10.0.0.1 dev tun0 [snip]
In the C to A direction, packets appear to be correctly forwarded over the VPN at gwC. So let's have a look at the logs on gwA:
Sun Nov 15 13:25:35 2009 clientc/172.31.0.1:54788 MULTI: bad source address from client [192.168.3.1], packet dropped Sun Nov 15 13:25:36 2009 clientc/172.31.0.1:54788 MULTI: bad source address from client [192.168.3.1], packet dropped
So there is something wrong after all (172.31.0.1 is gwC's external public IP).
The theory
At this point, some background on how OpenVPN works internally is in order.
When OpenVPN receives a packet or frame on the tun/tap interface to forward, it encrypts it and encapsulates it into one or more UDP datagrams, which are then sent out to some remote (usually public) IP address where another VPN node will receive it on its public IP, decapsulate and decrypt them, and send them to the local tun/tap interface, where they will be finally seen by the OS. The process also works in the opposite direction of course.
If the IP addresses involved are only those belonging to the VPN, OpenVPN has no trouble to associate a certain VPN IP to the public IP address of a remote VPN peer (as long as the addresses were pushed by the server to the clients and not statically assigned).
However, when non-VPN packets are involved, OpenVPN needs more information. In our A-to-C example, when gwA receives the packet with src=192.168.1.1 and dst=192.168.3.1, the routing table sends it to the tun0 interface, and thus to OpenVPN. Now, how does OpenVPN know behind which remote peer that destination IP is? That is a necessary piece of information it needs in order to know the destination IP to use for the encapsulating UDP packets.
Similarly, in the C-to-A direction, when gwA's OpenVPN sees a packet with src=192.168.3.1 and dst=192.168.1.1, it needs to make sure that it knows how to reach the source address, in order to send replies later. So it's just a different aspect of the same problem.
Since there can be (and there actually is) more than one peer, OpenVPN needs to know which network is behind each peer. You might think (as I naively did) that it should be able to somehow infer that information from the routes in the routing table, for example since gwA has this route in the routing table
192.168.3.0/24 via 10.0.0.3 dev tun0
it could assume that 192.168.3.0/24 is behind 10.0.0.3, and thus use gwC's public IP to send encapsulated traffic destined to 192.168.3.0/24. Well, it seems that it doesn't work that way. You have to explicitly tell OpenVPN which network is behind each client. This is where our iroute directive comes into play.
iroute
What iroute does, essentially, is to tell OpenVPN to create an "internal" OpenVPN route to that network via a specific peer. Of course this is a per-client configuration fragment (because each client can have different networks behind it), so the right place to insert this information on the server is in the client config directory. Let's update our config on gwA:
gwA # cat /etc/openvpn/ccd/gwB ifconfig-push 10.0.0.2 255.255.255.0 push "route 192.168.1.0 255.255.255.0 10.0.0.1" iroute 192.168.2.0 255.255.255.0 gwA # cat /etc/openvpn/ccd/gwC ifconfig-push 10.0.0.3 255.255.255.0 push "route 192.168.1.0 255.255.255.0 10.0.0.1" iroute 192.168.3.0 255.255.255.0
There's no explicit mention of a gateway, but for example having the directive
iroute 192.168.3.0 255.255.255.0
in gwC's client config file already implies that 192.168.3.0/24 is reachable through gwC (thus 10.0.0.3 and associated public IP). The same for gwB. With this final piece of information, OpenVPN is finally able to route traffic for those remote networks. Let's have a look at gwA's log when the clients connect:
Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: Learn: 10.0.0.3 -> gwC/172.31.0.1:38107 Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: primary virtual IP for gwC/172.31.0.1:38107: 10.0.0.3 Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: internal route 192.168.3.0/24 -> gwC/172.31.0.1:38107 Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: Learn: 192.168.3.0/24 -> gwC/172.31.0.1:38107 .... Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: Learn: 10.0.0.2 -> gwB/172.17.0.1:58645 Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: primary virtual IP for gwB/172.17.0.1:58645: 10.0.0.2 Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: internal route 192.168.2.0/24 -> gwB/172.17.0.1:58645 Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: Learn: 192.168.2.0/24 -> gwB/172.17.0.1:58645
Let's verify that communication is indeed working:
box-in-c # ping -c 4 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. 64 bytes from 192.168.1.1: icmp_seq=1 ttl=62 time=21.0 ms 64 bytes from 192.168.1.1: icmp_seq=2 ttl=62 time=4.83 ms 64 bytes from 192.168.1.1: icmp_seq=3 ttl=62 time=14.9 ms 64 bytes from 192.168.1.1: icmp_seq=4 ttl=62 time=2.29 ms --- 192.168.1.1 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3044ms rtt min/avg/max/mdev = 2.298/10.790/21.081/7.597 ms
Gotchas
After discussing iroute, it's worth mentioning two important gotchas.
NAT
You could avoid using iroute altogether by SNATing traffic at the client. In our example, gwC would SNAT traffic coming from 192.168.3.0/24 that needs to be forwarded via the VPN, using something like
gwC # iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o tun0 -j MASQUERADE
A similar command would be used on gwB.
It's up to you to decide whether to do this or to use iroute instead. Generally speaking, using NAT has some configuration overhead at the client, whereas iroute can be entirely controlled at the server. (some clients might not be accessible for configuration, or might not even support NAT)
Explicit remote
Don't assume that if there is only one client you don't need iroute. You still need it. However, if you are sure that there will always be only one client, and the setup is not going to change, you can work around the need to use iroute by explicitly using local and remote on both peers, and a mostly static configuration. Something like this:
# server tls-server local x.x.x.x remote y.y.y.y port 1194 ifconfig 10.0.0.1 255.255.255.0 push "ifconfig 10.0.0.2 255.255.255.0" ....
# client client local y.y.y.y remote x.x.x.x port 1194 ...
This way, there is no ambiguity as OpenVPN is forced to use the address in remote for all traffic, so in this setup you can just set external (ie, routing table) routes and traffic will be forwarded without the need for iroute.
is there any way to hot change iroute? Let's say behind client C is client D with it's custom subnet. I unplug it from C and plug to B. Is there any way to modify internal route of OpenVPN server without restarting vpn connections?
Not that I know of, but I'm not 100% up to date with the last changes. You may have a better answer if you ask on the openvpn-users mailing list.
Ok, thanks for response. Do you mean any of those - https://openvpn.net/community-resources/mailing-lists/# ? it looks like they are not working anymore
They are, see eg https://sourceforge.net/p/openvpn/mailman/openvpn-users/, there are already several messages for april 2019.
my site-to-site works ! many thanks!
Hi
Nice article. But i have a question.
Considering the communication from C->A (Example: src: 192.168.3.1 , dest: 192.168.1.1)without having used iroute. The article
states that the GwA does not know how to respond to the source and hence we have a problem. But i donot understand how the GwC was able to know that it has to send to the (public ip of) GwA.
Could you shed some light to it?
Because GwA is pushing a route (not iroute) to GwC for the 192.168.1.0/24 network:
Very nice explanation indeed.
I was thinking to make things (maintenance) easier with OSPF (Quagga). Didn't research on it yet, but it seems, it won't work. Unless I modify the OpenVPN code to get iroute from routing table or routing software
My net topology is:
LAN A pfSense_ovpn_client_A [VPS + OVPN Server] pfSense_ovpn_client_B LAN B
It works very nice, since neither client public IP, nothing is listening on them. Only one public IP with domain name is needed.
Having Unbound DNS in the middle, it makes full fledged intranet with different servers, SIP etc residing mostly on VPN net
Though, I have more than one [VPS + OpenVPN] and more than two clients like that. And see some issues:
1) Cumbersome config. iroute, push route, etc.
2) No dynamic routing, single point of failure ...
I'm quite new to anything above static routing. Do you think, it makes sense to contribute to OSPF learning?
While I personally don't run OSPF over OpenVPN, you can find quite a few people in forums and mailing lists who do, and it seems to work well.
hi,
I need help with making aslightly diffrent topology of the OpenVPN network,
I have PC with 2 nics, the nics are configured as router ( windows configuration -regedit) on that router pc i have installed OPENVPN CLIENT.
one nic ( ip 192.168.1.5) is connected to a pc/server which has VPNSERVER on it( local IP 192.168.10 GW 192.168.5)
I have established VPN tunnel connection between them- and its working ( ping and every thing)( server 10.8.0.1 Client 10.8.0.6)
on the second NIC of the PC /router (ip 172.168.1.11) i connected Laptop (ip 172.168.1.5 gw 172.168.1.11)
i try to gain connection to 10.8.0.1 but with no success..
I hope some one can help me.
thanks,
ziv
This has nothing to do with iroute, anyway you probably need a route to the 172.168.1 network on the VPN server machine.
If not, use tcpdump on all interfaces in the path to see where the packets stop.
hi,
You are right about the topology when i conect the machines behind the serverVPN,
but my machines (172.168.X.X) are behind the VPNclient ( which has IProute =1 in the registry thus router)
so as i under stand it i need to use the iroute
If it's an iroute-related problem, you should see corresponding errors in the OpenVPN log. If not, again, use tcpdump to debug where the traffic stops.
Thanks a lot.
This is the best "iroute" explanation I have come across so far.
Thanks so much for this excellent explanation! - five years later I was struggling to get site-to-site routing working between a pfSense and a Sophos UTM (Astaro) and the iroute configuration trick was the missing piece I needed.
I have an OpenVPN server running on Linux on a public address.
I have an OpenVPN remote client running on Linux on a LAN behind a router/firewall.
I'm using tun interfaces so am using the "routed" mode.
The remote client is 192.168.1.225.
The tunnel comes up just fine and from the server I can ping any host on the client's LAN.
Also, from the client, I can ping any host on the server's LAN.
However, there are two LAN's behind the client:
192.168.1.0/24 and
192.168.2.0/24
From the client, I can ping 192.168.2.1 just fine.
However, from the server I cannot ping 192.168.2.1, it just hangs indefinitely.
In the client's ccd file on the server I have the following:
iroute 192.168.1.0 255.255.255.0
iroute 192.168.2.0 255.255.255.0
In the server.conf file I have:
route 192.168.1.0 255.255.255.0
route 192.168.2.0 255.255.255.0
push "route 192.168.1.0 255.255.255.0"
If I add push "route 192.168.2.0 255.255.255.0" to the server.conf file, I can no longer ssh to the client, plus I still am unable to ping the 192.168.2.1 host.
I've tried so many different combinations but so far have not had any luck.
I could send all of config files but thought that might be too much clutter for just a comment type post. I would be happy to provide any additional information that may be useful.
I actually have other /30 type clients connected to this same server and all works fine. But, each of those clients only have one LAN behind them.
Thanks for any hints you can provide.
I think you don't have to push routes to the client, for networks that are behind it and that it should already know how to reach. Otherwise the client will start to try to reach those networks over the VPN, which of course isn't what is wanted.
So try removing the
from the server config (you do need the "route" and "iroute" directives though). What you *may* want to push to the client are routes to networks *behind the OpenVPN server*, if any; but certainly not routes for networks that the client already knows how to reach.
You might also want to assign a fixed VPN IP address to that client, and point the two routes on the server to that IP, for example (assuming you assign the client 10.0.0.10):
Thanks for your help Waldner. Removing the push statements did not help. I will continue to experiment by using Wireshark or tcpdump to see where the packets are getting stopped.
Also, how did you do the "code" sections in your response? I tried to find a way to do that when I responded earlier but couldn't.
Thanks again.
It's a normal <pre> tag.
This is a great tutorial, thank you!
In my setup, I have this one big problem that I can't figure out. I have 2 local networks behind my OpenVPN client and I can reach computers that are on the client's network, but I can't reach the other network. The other network IS accessible from the client, but I can't reach it from the OpenVPN server.
I've not been able to find a tutorial that shows multiple LAN's behind a client. Do you know of one?
Without more information it's a bit of a guessing game...if I understand correctly, you have to add two "iroute" statements (one for each network that is behind your client) in that client's CCD file. That way the OpenVPN server will know that those two networks are reachable through that client.
On the client itself, of course, you will also have to set up routing/firewalling/NAT/whatever correctly for the traffic from/to the OpenVPN server to be able to go to/from the two networks in question.
What if two or more networks having overlapping internal IP ranges (shorter prefix length if network A for example) behind their tun interfaces? Or to put in another way, does iroute maintain and process an internal routing take just like kernel does with a normal routing table, using prefix length and metric to decide priorities?
AFAICT, the "iroute" directive accepts a netmask specification.
thaks very usefull and clear.
don't work your howto some inconsitences in reference to http://openvpn.net/index.php/open-source/documentation/howto.html#policy because openvpn "ifconfig-push ip mask" is a mistake in ccd need two ip to end point vpn network as "ifconfig-push ip_peer ip_end_point"
With "topology subnet" (available from OpenVPN 2.1), which is what the example uses, the second parameter must be the subnet mask. The document you're referencing is ancient and assumes that the old "net30" topology is being used.
This is because the client uses the information it receives to run this command:
/sbin/ifconfig tun0 10.0.0.2 netmask 255.255.255.0 mtu 1500 broadcast 10.0.0.255(or iproute if configured to do so). If you change the ifconfig-push directive to read
ifconfig-push 10.0.0.2 10.0.0.1this is what happens on the client:
Mon Nov 28 18:05:59 2011 /sbin/ifconfig tun0 10.0.0.2 netmask 10.0.0.1 mtu 1500 broadcast 255.255.255.254SIOCSIFNETMASK: Invalid argument
Mon Nov 28 18:05:59 2011 Linux ifconfig failed: external program exited with error status: 1
Mon Nov 28 18:05:59 2011 Exiting
Awesome! Many thanks!