OpenSwan – Connecting two VPC’s of different Regions in Amazon AWS

As of now today, Amazon AWS doesn’t have any in-built solution to enable the connectivity between VPC’s of two different regions unlike VPC peering between VPC’s of the same region. So we need to build a custom VPN solution to establish the connectivity between the region VPC’s. OpenSwan IPSec VPN is one of the best solution to achieve this.

I looked at many blog posts which explains how to connect the VPC’s of two regions using OpenSwan VPN. The posts often cofuses about the steps, route tables, security groups and etc. So I thought of writing a detailed post with every minute step which I have gone through recently for one of our esteemed client. Hope this helps you.

Here is the simple High-Level diagram to connect multiple VPC’s using OpenSwan between regions.

Considerations

  • The Chosen VPC’s shouldn’t have the same CIDR range
  • The IPSec connections require each VPN instance to live in a public subnet and have an EIP address.
  • VPC Configurations
    VPC#1 – US-EAST-1 | 10.10.0.0/21 | EIP1
    VPC#2 – US-WEST-1 | 10.20.0.0/21 | EIP2

Configuration

  1. Launch two RHEL instances from pre-approved AMI’s, one in each VPC public subnet, with the following characteristics:
    • Allocate two VPC EIPs (or one VPC EIP in each region) and associate an EIP to each VPN instance. In this example we will use EIP1 and EIP2 to represent the EIPs for VPC 1 and VPC 2 respectively.
    • Ensure the Security Groups associated with each instance allow both Inbound and outbound UDP 500, UDP 4500, IP 50, and IP 51 ports between each other’s EIP
      Inbound

      Outbound
  2. Disable Source/Dest checking on both instances by right-clicking on the instances and selecting Change Source/Dest. Check.

  3. Configure all your Route Tables in both VPCs to send traffic to the “other” VPC through the VPC EC2 instances of other VPC CIDR’s.
    VPC#1 Route Table:

    VPC#2 Route Table

Configure IPSec OpenSwan on both EC2 Instances

  1. Perform the following actions on both machines in both the regions
    • Disable the SELINUX and both the machines
    • Configure the Linux instances to route traffic by editing /etc/sysctl.confand change thenet.ipv4.ip_forward, net.ipv4.conf.all.accept_redirects, and net.ipv4.conf.all.send_redirectsvariables.

      ipv4.ip_forward = 1
      net.ipv4.conf.all.accept_redirects = 0
      net.ipv4.conf.all.send_redirects = 0

    • Disable the Firewall Service during startup using chkconfig command
    • Reboot the machine to take these settings effective across reboots
    • Install the OpenSwan package using Yum command : yum install openswan
  2. Edit the /etc/ipsec.conf file (as root) to include files in /etc/ipsec.d/*.conf
    • Uncomment the last line by removing the ‘#’ on the first character of the last line so it looks like the following:
      include /etc/ipsec.d/*.conf
    • Add the plutostderrlog=/tmp/pluto.log entry in the same file for the logging and troubleshooting
  3. Edit the /etc/ipsec.secrets file(as root) and the following line :
    include /etc/ipsec.d/*.secrets
  4. Create the IPSec tunnel configuration in both the EC2 Instances to establish the connectivity
    • VPC1/EIP1 OpenSwan Instance 
      • Create a file vi /etc/ipsec.d/vpc1-to-vpc2.conf and add the following content to it.

        conn vpc1-to-vpc2
        type=tunnel
        authby=secret
        left=%defaultroute
        leftid=<EIP1>
        leftnexthop=%defaultroute
        leftsubnet=<VPC1 CIDR>
        right=<EIP2>
        rightsubnet=<VPC2 CIDR>
        pfs=no
        auto=start

    • Create a secrets file for Key Exchange between the servers

      vi /etc/ipsec.d/vpc1-to-vpc2.secrets
      <EIP1> <EIP2>: PSK “Put a Preshared Key here!!”
      chmod 600 /etc/ipsec.d/vpc1-to-vpc2.secrets

    • VPC2/EIP2 OpenSwan Instance
      • Create a file vi /etc/ipsec.d/vpc2-to-vpc1.conf and add the following content to it.

        conn vpc2-to-vpc1
        type=tunnel
        authby=secret
        left=%defaultroute
        leftid=<EIP2>
        leftnexthop=%defaultroute
        leftsubnet=<VPC2 CIDR>
        right=<EIP1>
        rightsubnet=<VPC1 CIDR>
        pfs=no
        auto=start

    • Create a secrets file for Key Exchange between the servers

      vi /etc/ipsec.d/vpc2-to-vpc1.secrets
      <EIP1> <EIP2>: PSK “Put a Preshared Key here!!”
      chmod 600 /etc/ipsec.d/vpc2-to-vpc1.secrets

  5. On both instances, perform the following steps:
    • Start IPSec/Openswan : service ipsec start
    • Configure IPSec/Openswan to always start on boot : chkconfig ipsec on

Checking VPN Status

  1. Check whether the pluto service is running or not?
    ps –ef | gre pluto

  2. Check whether the 500 and 4500 ports are listening or not against pluto service?
    netstat -tunlp | grep pluto

  3. Check whether the Peering is established or not? In the pluto logs.
    tail –100 /tmp/pluto.log
  4. Ping each machine using their Private IP Addresses to cross verify whether the communication is established or not? ( Allow both SG’s private IP Address outbound/Inbound security groups ICMP to communicate with each other).
  5. Verify the route Policies for Tunneling
    ip xfrm policy

Traffic Allow between VPC’s

After the tunnels are up, we can use the private IP Addresses of each side VPC’s to establish the communication. But the firewall rules are little tricky in VPC because of the Inbound/Outbound restrictions.

Example: Allow the DR Linux EC2 Instance to communicate with PROD Linux EC2 instance over 22 port.

  • Add an outbound rule to DR EC2 Associated instance SG – 22 port / Prod EC2 Instance IP Address
  • Add an Inbound rule to DR OpenSwan VPN Instance associated SG – 22 port / DR EC2 Instance associated SG reference
  • Add an Outbound rule to Prod OpenSwan VPN Instance associated SG – 22 port / Prod EC2 Instance associated SG reference
  • Add an Inbound rule to Prod EC2 Instance associated SG – 22 port / DR EC2 Instance IP Address

OpenSwan VPN Monitoring

The OpenSwan EC2 Instances availability and Tunnels status can be monitored with our standard monitoring tool called Nagios.

Backup & Recovery

The OpenSwan configurations can be backed up by using the AMI Creation process. Whenever we find the outage on our server and can’t be fixed with the current instance.

  • Launch an EC2 Instance with an Latest backup AMI
  • Re-associate the same old EIP with the new instance
  • Update the route tables with the new Instance/ENI ID in all the route tables.

Troubleshooting

To troubleshoot any of the tunnel down issues, please use the following two log files to understand the root cause of the error.

/var/log/messages
/tmp/pluto.log

Apart from that, always cross check the following items also.

  • Source/Dest check disable status
  • Security groups Inbound/Outbound rules
  • Verify the sysctl config parameters for traffic forwarding
  • Firewall rules and status
  • Route table routes and associated subnets

High Availability

The High Availability of the Connectivity between regions in multiple VPC’s can be achieved by deploying more than one VPN Instance in each zone and establish the connectivity in other region with more than one VPN Instance.

See the below diagram for reference:

4 Comments

 Add your comment
  1. Why do you need the CIDR to be the same? What if they are NOT and you meed to VPN connect them!

  2. it is necessary to check firewall rule everytime while troubleshooting?

  3. Great article! Thanks!

    On /etc/sysctl.conf, I also had to add those 2 lines to avoid ERROR when validating with “sudo ipsec verify”:
    net.ipv4.conf.default.accept_redirects = 0
    net.ipv4.conf.default.send_redirects = 0

    Can you explain more about the HA scenario? How should I configure Route Table on subnets if I’m using more than 1 instance on each region for VPN?

Leave a Comment

Your email address will not be published.