Life is unfair and shit happens, so let's consider the situation in the diagram:
Site A and site B both use the same IP range (192.168.10.0/24), and now for whatever reason the two sites need to talk to each other (for example through a VPN). Connections can be in either direction, and potentially any host at any site should be able to talk to any host or hosts at the other site.
Of course, renumbering is out of the question, so the poor sysadmin has to come up with some klu^Wbrilliant solution to solve the routing problem and save the day.
One may think of doing some sort of 1:1 NAT, also known as full cone NAT (here even without port number translation). The idea is: to each site, the other site will appear as if it is using some other, fake, IP range. So if our host 192.168.10.44 at site A wants to talk to host 192.168.10.211 at site B, it will pretend it wants to talk to host 192.168.200.211 instead. More generally, a host at site A wanting to talk to host 192.168.10.x at site B, will instead use a destination address of 192.168.200.x (where x is the same in both addresses).
Assuming we're somehow able to deliver those packets to site B (see later), something has to happen in between so the host at site B thinks the packets are destined to it, and they should look like they're coming from an IP range different from its own (the fake range assigned to site A, 192.168.100.0/24). More generally, packets coming from site A's host 192.168.10.y will appear at site B as if they're coming from host 192.168.100.y (where y is the same in both addresses).
The diagram below illustrates what needs to be done.
In the upper part of the diagram, host 192.168.10.44 sends its packet with source 192.168.10.44 and destination 192.168.200.211. Gateway gwa, before forwarding the packet to the VPN link, rewrites the source address so it looks like the packet is coming from 192.168.100.44 instead (192.168.100.0/24 is the fake IP range assigned to site A). When gateway gwb receives it, it has to change the destination address to the real one of the receiving host, that is, 192.168.10.211. Once that is done, the target host receives the packet and thinks it's from 192.168.100.44.
In the other direction (lower part of the diagram), the same tricks are applied in reverse so at the end the host at site A thinks it's really talking to 192.168.200.211, and the host at site B thinks it's really talking to 192.168.100.44.
All the logic is implemented in the gateways. At site A, the gateway rewrites source addresses of the form 192.168.10.x to 102.168.100.x for outgoing traffic, and does the reverse translation on destination addresses (from .100.x to .10.x) on incoming traffic. At site B, something similar happens, except that the translation is from source .10.x to .200.x on outgoing traffic, and from destination .200.x to .10.x on incoming traffic.
That way, everyone is fooled and happy, and things (mostly) work.
Since the NAT is 1:1, new machines can be added at each site and the gateways need no change in configuration.
All this machinery is made possible by a neat iptables target called NETMAP, which can automagically perform the translations described above. NETMAP only works on the nat table, and depending on whether it's used in the POSTROUTING or the PREROUTING chain, it does the right thing and rewrites the right address (source or destination).
This target allows you to statically map a whole network of addresses onto another network of addresses. It can only be used from rules in the nat table.
Network address to map to. The resulting address will be constructed in the following way: All 'one' bits in the mask are filled in from the new 'address'. All bits that are zero in the mask are filled in from the original address.
Obviously, given the way it works, it is important that the range being used for rewriting have the same
netmask as the range being rewritten.
So here are the steps to follow. Each gateway needs to have a route to the other site's fake IP range pointing to the VPN peer:
gwa# ip route 10.0.0.0/24 dev tun0 proto kernel scope link src 10.0.0.2 192.168.200.0/24 via 10.0.0.1 dev tun0 192.168.10.0/24 dev eth0 proto kernel scope link src 192.168.10.254 ...
gwb# ip route 192.168.100.0/24 via 10.0.0.2 dev tun0 10.0.0.0/24 dev tun0 proto kernel scope link src 10.0.0.1 192.168.10.0/24 dev eth0 proto kernel scope link src 192.168.10.254 ...
If using OpenVPN for the VPN, the routes can easily be pushed/pulled through OpenVPN's configuration; alternatively they can be manually created upon connection establishment.
Then, suitable iptables rules need to be created (these can be created before or after the VPN is established, although the VPN needs to be up for them to work):
# rewrite destination address on incoming traffic gwa# iptables -t nat -A PREROUTING -s 192.168.200.0/24 -d 192.168.100.0/24 -i tun0 -j NETMAP --to 192.168.10.0/24 # rewrite source address on outgoing traffic gwa# iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -d 192.168.200.0/24 -o tun0 -j NETMAP --to 192.168.100.0/24
# same things, on gwb gwb# iptables -t nat -A PREROUTING -s 192.168.100.0/24 -d 192.168.200.0/24 -i tun0 -j NETMAP --to 192.168.10.0/24 gwb# iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -d 192.168.100.0/24 -o tun0 -j NETMAP --to 192.168.200.0/24
Let's check that it works:
anyhost-siteA$ ping 192.168.200.211 PING 192.168.200.211 (192.168.200.211) 56(84) bytes of data. 64 bytes from 192.168.200.211: icmp_req=1 ttl=64 time=20.7 ms 64 bytes from 192.168.200.211: icmp_req=2 ttl=64 time=20.4 ms 64 bytes from 192.168.200.211: icmp_req=3 ttl=64 time=21.1 ms 64 bytes from 192.168.200.211: icmp_req=4 ttl=64 time=20.5 ms
Tools like tcpdump or wireshark can be used at each interface along the path to verify that addresses are being rewritten as they should.
Of course, if using names to communicate, appropriate DNS entries pointing to the other site's fake IPs have to
be configured at each site.
This is a quick and dirty setup (to say the least). As such, in the grand tradition of how things usually work in this field, once it's in place it will never be touched anymore (that is, as long as it works), and the poor sysadmin who wanted to renumber one of the sites will never be able to do so. That's life.
One downside of this setup (besides being a stinking kludge, which should be enough to keep people from using it) is that the gateways cannot talk to the hosts at the other site; if that is really needed, it can certainly be done, thus making the whole thing even more kludgy. To accomplish it, all that is needed is that each gateway be configured to use its internal IP as source address when talking to the other site's fake IPs. Implementing this is left as an exercise for the reader. (Hint: it's as easy as using the "src" argument to "ip route").