Acquiring multiple non-residential IPs

Self-hosting

Background

When hosting services, it can be useful to have multiple Internet Protocol (IP) addresses. For Internet Protocol version 6 (IPv6) this is not a problem—assuming your Internet service provider (ISP) is not completely incompetent—but you likely only have one globally routable Internet Protocol version 4 (IPv4) address. Additionally, using IP addresses categorized as “residential” may cause issues (e.g., message transfer agents (MTAs) will often increase the spam score of an e-mail sent from another MTA with a residential IP). Finally, hosting is easier with static IP addresses; but most residential IPs are dynamic.

Virtual private server (VPS) provider

It can be challenging to find a high-quality VPS provider that does routing properly. Fortunately, I was able to find one in ARP Networks. They are a no-nonsense provider that charge reasonable prices at the cost of a more rudimentary graphical user interface (GUI) and no hand-holding: if you know what you are doing, then they are great. While I could probably make do with their cheapest offering, I “splurge” on their $20 plan. For an additional $5, they provide a /29 IPv4 network instead of a single IPv4 address. They also provide a /48 IPv6 network. It is sad that that is such a big deal as IPv6 is designed to not only provide globally routable addresses to any device but also the ability to have many subnets. What is extra remarkable about ARP Networks is that they route the IPv6 block via a link-local address giving you the entire block. They even accommodated me when I requested a similar setup for the IPv4 network by routing it via a private address.

The network configuration is pretty simple to set up on the VPS since we are dealing with static IPs; but for the sake of completeness, here you go—note I run OpenBSD on both my VPS and router, so the configuration will be for it.

vps$ cat /etc/hostname.vio0
inet6 2607:f2f8:a600::1 128
inet6 fe80::2%vio0 64
inet 10.0.121.2 255.255.255.252 NONE
vps$ cat /etc/mygate
fe80::1%vio0
10.0.121.1

Obviously you have to make sure your pf.conf(5) rules are set up correctly. We want for the VPS itself to have access to the Internet for things like security updates, but we don’t want to waste any of our precious IPv4 addresses. As a result, we only assign an IPv6 address from the first /56 routed to us.

Routing the acquired networks to my house

With the IPs routed to my VPS, the next thing is to route them to my house. This can easily be achieved via a virtual private network (VPN) tunnel. In my case I use WireGuard.

vps# cat /etc/hostname.wg0
wgkey <redacted>
wgport 58120
wgpeer Dpx8UOJJFxb4sj4NDR5qC77IV42XfTwe1QFtR3RkPko= wgpsk <redacted> wgendpoint 2001:558:6040:83:a99d:b51a:6dd1:632c 58120 wgaip fe80::2/128 wgaip 2607:f2f8:a600:100::/56 wgaip 10.0.0.2/32 wgaip 174.136.97.8/29
inet6 fe80::1%wg0 64
inet 10.0.0.1 255.255.255.0 NONE
!route -qn add -inet6 -net 2607:f2f8:a600:100::/56 -static fe80::2%wg0
!route -qn add -inet -net 174.136.97.8/29 -static 10.0.0.2

Routing traffic through WireGuard at home

For the router we will use two separate rdomain(4)s: rdomain 0 will have a routing table whose default route is my ISP while rdomain 1 will have a routing table with default route being the VPS. We first create a loopback device for rdomain 1 adding the normal reject routes. After that we create the wg(4) pseudo-device containing the VPS as a peer. Within the hostname.if(5), we add the default route to be the VPS.

router$ cat /etc/hostname.lo1
rdomain 1
inet6 ::1 128
inet 127.0.0.1 255.0.0.0 NONE
!route -qn -T1 add -inet6 -net ::/96 -static ::1 -reject
!route -qn -T1 add -inet6 -net ::ffff:0.0.0.0/96 -static ::1 -reject
!route -qn -T1 add -inet6 -net 2002::/24 -static ::1 -reject
!route -qn -T1 add -inet6 -net 2002:7f00::/24 -static ::1 -reject
!route -qn -T1 add -inet6 -net 2002:e000::/20 -static ::1 -reject
!route -qn -T1 add -inet6 -net 2002:ff00::/24 -static ::1 -reject
!route -qn -T1 add -inet6 -net fe80::/10 -static ::1 -reject
!route -qn -T1 add -inet6 -net fec0::/10 -static ::1 -reject
!route -qn -T1 add -inet6 -net ff01::/16 -static ::1 -reject
!route -qn -T1 add -inet6 -net ff02::/16 -static ::1 -reject
!route -qn -T1 add -inet -net 224.0.0.0/4 -iface -static 127.0.0.1 -reject
!route -qn -T1 add -inet -net 127.0.0.0/8 -static 127.0.0.1 -reject
router# cat /etc/hostname.wg0
rdomain 1
wgkey <redacted>
wgport 58120
wgpeer C0U1GJJQ5vCQwes7Ybb+/MbeVeoTbnEFRRZE0KwzgUI= wgrtable 0 wgpsk <redacted> wgendpoint 2607:f2f8:a600::1 58120 wgaip ::/0 wgaip 0.0.0.0/0
inet6 fe80::2%wg0 64
inet 10.0.0.2 255.255.255.0 NONE
!route -qn -T1 add -inet6 -net default -link -iface -static wg0
!route -qn -T1 add -inet -net default -link -iface -static wg0

One thing to be mindful of is the maximum transmission unit (MTU) of devices that will be routed through the VPN tunnel. Since the MTU of wg0 is 1420, these other interfaces must be less than or equal to that. For example:

router$ cat /etc/hostname.vlan6
rdomain 1
mtu 1420
parent ixl0
vnetid 50
inet6 2607:f2f8:a600:100::1 128
inet 174.136.97.8 255.255.255.255 NONE

Voilà: entire globally-routable IPv6 and IPv4 networks at my house. We didn’t have to rely on nasty hacks like network address translation (NAT) or Neighbor Discovery Protocol (NDP) proxying either. You may want communication to occur across rdomains. For example my e-mail server is in rdomain 1, but I have devices in rdomain 0 that send e-mails for security updates and the like. Instead of such a device sending an e-mail across several hops until it reaches my VPS in Los Angeles just for it to come back via WireGuard, it makes a lot more sense if the e-mail went directly to my e-mail server. Fortunately, pf(4) can do that rather easily. Here is a rule that redirects Internet Control Message Protocol version 6 (ICMPv6) traffic coming into the router on rdomain 0 to servers located in rdomain 1:

pass in quick on rdomain 0 inet6 proto icmp6 to $servers allow-opts rtable 1