Skip to content
 

OpenVPN and iroute

This post will demonstrate how and when the iroute directive is used in OpenVPN.

The problem

In most cases iroute is not needed, and in fact many users probably have never used it (or are aware of it, for that matter). It usually comes into play when networks behind the VPN nodes need to communicate. Let's imagine a topology like this:

iroute

Let's suppose that you want communication between networks A and B, and between A and C, as indicated by the dotted arrows. The config files are something like this:

gwA # cat /etc/openvpn/server.conf
# gwA
local 172.20.0.1
port 1194
proto udp
dev tun
topology subnet
mode server
tls-server
ifconfig 10.0.0.1 255.255.255.0
route 192.168.2.0 255.255.255.0 10.0.0.2
route 192.168.3.0 255.255.255.0 10.0.0.3
client-config-dir ccd
# snip rest of config

gwA # cat /etc/openvpn/ccd/gwB
ifconfig-push 10.0.0.2 255.255.255.0
push "route 192.168.1.0 255.255.255.0 10.0.0.1"
gwA # cat /etc/openvpn/ccd/gwC
ifconfig-push 10.0.0.3 255.255.255.0
push "route 192.168.1.0 255.255.255.0 10.0.0.1"
gwB # cat /etc/openvpn/client.conf
# gwB
remote 172.20.0.1 1194
proto udp
dev tun
topology subnet
# snip rest of config
gwC # cat /etc/openvpn/client.conf
# gwC
remote 172.20.0.1 1194
proto udp
dev tun
topology subnet
# snip rest of config

You think that having all the necessary routes in place as per the above configs would be enough to allow the desired communication, right? Well, let's bring up the VPN and try pinging from a computer in network A to network C:

box-in-a # ping 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
^C
--- 192.168.3.1 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 7012ms

Let's try from network C to network A:

box-in-c # ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
^C
--- 192.168.1.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 4017ms

How come? Why isn't it working? Forwarding is enabled on all gateways, routes are in place:

gwA # ip route show
192.168.3.0/24 via 10.0.0.3 dev tun0 
192.168.2.0/24 via 10.0.0.2 dev tun0 
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.254
10.0.0.0/24 dev tun0  proto kernel  scope link  src 10.0.0.1 
[snip]

gwC # ip proute show
10.0.0.0/24 dev tun0  proto kernel  scope link  src 10.0.0.3 
192.168.3.0/24 dev eth0  proto kernel  scope link  src 192.168.3.254 
192.168.1.0/24 via 10.0.0.1 dev tun0 
[snip]

In the C to A direction, packets appear to be correctly forwarded over the VPN at gwC. So let's have a look at the logs on gwA:

Sun Nov 15 13:25:35 2009 clientc/172.31.0.1:54788 MULTI: bad source address from client [192.168.3.1], packet dropped
Sun Nov 15 13:25:36 2009 clientc/172.31.0.1:54788 MULTI: bad source address from client [192.168.3.1], packet dropped

So there is something wrong after all (172.31.0.1 is gwC's external public IP).

The theory

At this point, some background on how OpenVPN works internally is in order.
When OpenVPN receives a packet or frame on the tun/tap interface to forward, it encrypts it and encapsulates it into one or more UDP datagrams, which are then sent out to some remote (usually public) IP address where another VPN node will receive it on its public IP, decapsulate and decrypt them, and send them to the local tun/tap interface, where they will be finally seen by the OS. The process also works in the opposite direction of course.
If the IP addresses involved are only those belonging to the VPN, OpenVPN has no trouble to associate a certain VPN IP to the public IP address of a remote VPN peer (as long as the addresses were pushed by the server to the clients and not statically assigned).

However, when non-VPN packets are involved, OpenVPN needs more information. In our A-to-C example, when gwA receives the packet with src=192.168.1.1 and dst=192.168.3.1, the routing table sends it to the tun0 interface, and thus to OpenVPN. Now, how does OpenVPN know behind which remote peer that destination IP is? That is a necessary piece of information it needs in order to know the destination IP to use for the encapsulating UDP packets.
Similarly, in the C-to-A direction, when gwA's OpenVPN sees a packet with src=192.168.3.1 and dst=192.168.1.1, it needs to make sure that it knows how to reach the source address, in order to send replies later. So it's just a different aspect of the same problem.

Since there can be (and there actually is) more than one peer, OpenVPN needs to know which network is behind each peer. You might think (as I naively did) that it should be able to somehow infer that information from the routes in the routing table, for example since gwA has this route in the routing table

192.168.3.0/24 via 10.0.0.3 dev tun0

it could assume that 192.168.3.0/24 is behind 10.0.0.3, and thus use gwC's public IP to send encapsulated traffic destined to 192.168.3.0/24. Well, it seems that it doesn't work that way. You have to explicitly tell OpenVPN which network is behind each client. This is where our iroute directive comes into play.

iroute

What iroute does, essentially, is to tell OpenVPN to create an "internal" OpenVPN route to that network via a specific peer. Of course this is a per-client configuration fragment (because each client can have different networks behind it), so the right place to insert this information on the server is in the client config directory. Let's update our config on gwA:

gwA # cat /etc/openvpn/ccd/gwB
ifconfig-push 10.0.0.2 255.255.255.0
push "route 192.168.1.0 255.255.255.0 10.0.0.1"
iroute 192.168.2.0 255.255.255.0
gwA # cat /etc/openvpn/ccd/gwC
ifconfig-push 10.0.0.3 255.255.255.0
push "route 192.168.1.0 255.255.255.0 10.0.0.1"
iroute 192.168.3.0 255.255.255.0

There's no explicit mention of a gateway, but for example having the directive

iroute 192.168.3.0 255.255.255.0

in gwC's client config file already implies that 192.168.3.0/24 is reachable through gwC (thus 10.0.0.3 and associated public IP). The same for gwB. With this final piece of information, OpenVPN is finally able to route traffic for those remote networks. Let's have a look at gwA's log when the clients connect:

Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: Learn: 10.0.0.3 -> gwC/172.31.0.1:38107
Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: primary virtual IP for gwC/172.31.0.1:38107: 10.0.0.3
Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: internal route 192.168.3.0/24 -> gwC/172.31.0.1:38107
Sun Nov 15 16:30:28 2009 gwC/172.31.0.1:38107 MULTI: Learn: 192.168.3.0/24 -> gwC/172.31.0.1:38107
....
Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: Learn: 10.0.0.2 -> gwB/172.17.0.1:58645
Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: primary virtual IP for gwB/172.17.0.1:58645: 10.0.0.2
Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: internal route 192.168.2.0/24 -> gwB/172.17.0.1:58645
Sun Nov 15 16:30:41 2009 gwB/172.17.0.1:58645 MULTI: Learn: 192.168.2.0/24 -> gwB/172.17.0.1:58645

Let's verify that communication is indeed working:

box-in-c # ping -c 4 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=62 time=21.0 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=62 time=4.83 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=62 time=14.9 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=62 time=2.29 ms

--- 192.168.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3044ms
rtt min/avg/max/mdev = 2.298/10.790/21.081/7.597 ms

Gotchas

After discussing iroute, it's worth mentioning two important gotchas.

NAT

You could avoid using iroute altogether by SNATing traffic at the client. In our example, gwC would SNAT traffic coming from 192.168.3.0/24 that needs to be forwarded via the VPN, using something like

gwC # iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o tun0 -j MASQUERADE

A similar command would be used on gwB.

It's up to you to decide whether to do this or to use iroute instead. Generally speaking, using NAT has some configuration overhead at the client, whereas iroute can be entirely controlled at the server. (some clients might not be accessible for configuration, or might not even support NAT)

Explicit remote

Don't assume that if there is only one client you don't need iroute. You still need it. However, if you are sure that there will always be only one client, and the setup is not going to change, you can work around the need to use iroute by explicitly using local and remote on both peers, and a mostly static configuration. Something like this:

# server
tls-server
local x.x.x.x
remote y.y.y.y
port 1194
ifconfig 10.0.0.1 255.255.255.0
push "ifconfig 10.0.0.2 255.255.255.0"
....
# client
client
local y.y.y.y
remote x.x.x.x
port 1194
...

This way, there is no ambiguity as OpenVPN is forced to use the address in remote for all traffic, so in this setup you can just set external (ie, routing table) routes and traffic will be forwarded without the need for iroute.

27 Comments

  1. Peter says:

    is there any way to hot change iroute? Let's say behind client C is client D with it's custom subnet. I unplug it from C and plug to B. Is there any way to modify internal route of OpenVPN server without restarting vpn connections?

  2. anotherone says:

    my site-to-site works ! many thanks!

  3. srik says:

    Hi

    Nice article. But i have a question.
    Considering the communication from C->A (Example: src: 192.168.3.1 , dest: 192.168.1.1)without having used iroute. The article
    states that the GwA does not know how to respond to the source and hence we have a problem. But i donot understand how the GwC was able to know that it has to send to the (public ip of) GwA.

    Could you shed some light to it?

    • waldner says:

      Because GwA is pushing a route (not iroute) to GwC for the 192.168.1.0/24 network:

      gwA # cat /etc/openvpn/ccd/gwC
      ifconfig-push 10.0.0.3 255.255.255.0
      push "route 192.168.1.0 255.255.255.0 10.0.0.1"
      
  4. Vitali says:

    Very nice explanation indeed.

    I was thinking to make things (maintenance) easier with OSPF (Quagga). Didn't research on it yet, but it seems, it won't work. Unless I modify the OpenVPN code to get iroute from routing table or routing software

    My net topology is:

    LAN A pfSense_ovpn_client_A [VPS + OVPN Server] pfSense_ovpn_client_B LAN B

    It works very nice, since neither client public IP, nothing is listening on them. Only one public IP with domain name is needed.
    Having Unbound DNS in the middle, it makes full fledged intranet with different servers, SIP etc residing mostly on VPN net

    Though, I have more than one [VPS + OpenVPN] and more than two clients like that. And see some issues:

    1) Cumbersome config. iroute, push route, etc.
    2) No dynamic routing, single point of failure ...

    I'm quite new to anything above static routing. Do you think, it makes sense to contribute to OSPF learning?

    • waldner says:

      While I personally don't run OSPF over OpenVPN, you can find quite a few people in forums and mailing lists who do, and it seems to work well.

  5. Ziv says:

    hi,

    I need help with making aslightly diffrent topology of the OpenVPN network,

    I have PC with 2 nics, the nics are configured as router ( windows configuration -regedit) on that router pc i have installed OPENVPN CLIENT.
    one nic ( ip 192.168.1.5) is connected to a pc/server which has VPNSERVER on it( local IP 192.168.10 GW 192.168.5)
    I have established VPN tunnel connection between them- and its working ( ping and every thing)( server 10.8.0.1 Client 10.8.0.6)

    on the second NIC of the PC /router (ip 172.168.1.11) i connected Laptop (ip 172.168.1.5 gw 172.168.1.11)
    i try to gain connection to 10.8.0.1 but with no success..

    I hope some one can help me.

    thanks,

    ziv

    • waldner says:

      This has nothing to do with iroute, anyway you probably need a route to the 172.168.1 network on the VPN server machine.
      If not, use tcpdump on all interfaces in the path to see where the packets stop.

      • Ziv says:

        hi,
        You are right about the topology when i conect the machines behind the serverVPN,

        but my machines (172.168.X.X) are behind the VPNclient ( which has IProute =1 in the registry thus router)

        so as i under stand it i need to use the iroute

        • waldner says:

          If it's an iroute-related problem, you should see corresponding errors in the OpenVPN log. If not, again, use tcpdump to debug where the traffic stops.

  6. Bertalan says:

    Thanks a lot.
    This is the best "iroute" explanation I have come across so far.

  7. Alexander Dupuy says:

    Thanks so much for this excellent explanation! - five years later I was struggling to get site-to-site routing working between a pfSense and a Sophos UTM (Astaro) and the iroute configuration trick was the missing piece I needed.

  8. Mike says:

    I have an OpenVPN server running on Linux on a public address.
    I have an OpenVPN remote client running on Linux on a LAN behind a router/firewall.
    I'm using tun interfaces so am using the "routed" mode.
    The remote client is 192.168.1.225.
    The tunnel comes up just fine and from the server I can ping any host on the client's LAN.
    Also, from the client, I can ping any host on the server's LAN.
    However, there are two LAN's behind the client:
    192.168.1.0/24 and
    192.168.2.0/24
    From the client, I can ping 192.168.2.1 just fine.
    However, from the server I cannot ping 192.168.2.1, it just hangs indefinitely.

    In the client's ccd file on the server I have the following:
    iroute 192.168.1.0 255.255.255.0
    iroute 192.168.2.0 255.255.255.0

    In the server.conf file I have:
    route 192.168.1.0 255.255.255.0
    route 192.168.2.0 255.255.255.0
    push "route 192.168.1.0 255.255.255.0"

    If I add push "route 192.168.2.0 255.255.255.0" to the server.conf file, I can no longer ssh to the client, plus I still am unable to ping the 192.168.2.1 host.

    I've tried so many different combinations but so far have not had any luck.
    I could send all of config files but thought that might be too much clutter for just a comment type post. I would be happy to provide any additional information that may be useful.

    I actually have other /30 type clients connected to this same server and all works fine. But, each of those clients only have one LAN behind them.

    Thanks for any hints you can provide.

    • waldner says:

      I think you don't have to push routes to the client, for networks that are behind it and that it should already know how to reach. Otherwise the client will start to try to reach those networks over the VPN, which of course isn't what is wanted.

      So try removing the

      push "route 192.168.1.0 255.255.255.0"
      push "route 192.168.2.0 255.255.255.0"
      

      from the server config (you do need the "route" and "iroute" directives though). What you *may* want to push to the client are routes to networks *behind the OpenVPN server*, if any; but certainly not routes for networks that the client already knows how to reach.

      You might also want to assign a fixed VPN IP address to that client, and point the two routes on the server to that IP, for example (assuming you assign the client 10.0.0.10):

      route 192.168.1.0 255.255.255.0 10.0.0.10
      route 192.168.2.0 255.255.255.0 10.0.0.10
      
      • Mike says:

        Thanks for your help Waldner. Removing the push statements did not help. I will continue to experiment by using Wireshark or tcpdump to see where the packets are getting stopped.

        Also, how did you do the "code" sections in your response? I tried to find a way to do that when I responded earlier but couldn't.

        Thanks again.

  9. Mike says:

    This is a great tutorial, thank you!

    In my setup, I have this one big problem that I can't figure out. I have 2 local networks behind my OpenVPN client and I can reach computers that are on the client's network, but I can't reach the other network. The other network IS accessible from the client, but I can't reach it from the OpenVPN server.

    I've not been able to find a tutorial that shows multiple LAN's behind a client. Do you know of one?

    • waldner says:

      Without more information it's a bit of a guessing game...if I understand correctly, you have to add two "iroute" statements (one for each network that is behind your client) in that client's CCD file. That way the OpenVPN server will know that those two networks are reachable through that client.
      On the client itself, of course, you will also have to set up routing/firewalling/NAT/whatever correctly for the traffic from/to the OpenVPN server to be able to go to/from the two networks in question.

  10. He Wei says:

    What if two or more networks having overlapping internal IP ranges (shorter prefix length if network A for example) behind their tun interfaces? Or to put in another way, does iroute maintain and process an internal routing take just like kernel does with a normal routing table, using prefix length and metric to decide priorities?

  11. einom says:

    thaks very usefull and clear.

  12. Jorge says:

    don't work your howto some inconsitences in reference to http://openvpn.net/index.php/open-source/documentation/howto.html#policy because openvpn "ifconfig-push ip mask" is a mistake in ccd need two ip to end point vpn network as "ifconfig-push ip_peer ip_end_point"

    • waldner says:

      With "topology subnet" (available from OpenVPN 2.1), which is what the example uses, the second parameter must be the subnet mask. The document you're referencing is ancient and assumes that the old "net30" topology is being used.

      This is because the client uses the information it receives to run this command:

      /sbin/ifconfig tun0 10.0.0.2 netmask 255.255.255.0 mtu 1500 broadcast 10.0.0.255

      (or iproute if configured to do so). If you change the ifconfig-push directive to read

      ifconfig-push 10.0.0.2 10.0.0.1

      this is what happens on the client:

      Mon Nov 28 18:05:59 2011 /sbin/ifconfig tun0 10.0.0.2 netmask 10.0.0.1 mtu 1500 broadcast 255.255.255.254
      SIOCSIFNETMASK: Invalid argument
      Mon Nov 28 18:05:59 2011 Linux ifconfig failed: external program exited with error status: 1
      Mon Nov 28 18:05:59 2011 Exiting

  13. David says:

    Awesome! Many thanks!