Building a Multisourced Infrastructure Using OpenVPN

How to use OpenVPN to take your hosting to the next level.

To address this issue, we developed a dynamic routing dæmon called cube-routed (download it from www.cohesiveft.com/multisourced-infra). It shares routing information between vpnsrvA and vpnsrvB and adjusts routing tables depending on which client connects to which server in near real time. Its internal structure is not very complex. One thread connects to a local OpenVPN dæmon process via its management interface (see the management option in the OpenVPN configuration file) and regularly runs the status command to update the list of clients connected locally. Another thread publishes this information for the remote instance of cube-routed. The third thread regularly reads a list of connected clients from the remote instance of cube-routed. Finally, the fourth thread adjusts the local routing table based on the following two rules: 1) adds a host route for every host connected to the remote OpenVPN server and 2) deletes the host route for every host connected to the local OpenVPN server.

cube-routed instances will exchange information over the management subnet we selected earlier. Create a second tunnel tun1 between vpnsrvA and vpnsrvB. vpnsrvA can be a server with IP 10.200.200.1, and vpnsrvB is its client with IP 10.200.200.5. You can use the configuration files from Listings 1 and 2 as a basis, but remember to adjust the IP addresses and select a different port—for example, you could add port 11940 to both the server and client. Start both OpenVPN dæmons, and use ping 10.200.200.1 and ping 10.200.200.5 to verify connectivity between them.

Now, create configuration files for cube-routed on both vpnsrvA and vpnsrvB, as shown in Listings 4a and 4b, and start both instances as root with the path to the configuration file as the only parameter (note that OpenVPN must already be running, and the tun0/tun1 interfaces on both vpnsrvA and vpnsrvB must be up).

Once you start everything and after several minutes of initial convergence time, host1 from the example above will be able to communicate with host2, even though they connected to different OpenVPN servers. Thus, you've achieved a fully fault-tolerant virtual LAN connectivity, with data traffic encryption as an added bonus.

Conclusion

This implementation is not without its limitations. First, applications that use broadcast or multicast will not work with OpenVPN's tun device. You can use the same network layout as described here, but instead of tun, experiment with OpenVPN's tap device to work around this. Second, latency of network links over the public Internet is significantly higher than that of Ethernet. If this is an inherent requirement for your application, you probably should leave this part of your infrastructure single-sourced. Third, because we use UDP-based tunnels, OpenVPN links will tend to go up and down more often than Ethernet, especially during times of network congestion. You can implement data caches, avoid long-lived TCP connections, focus on network exception-handling logic and experiment with TCP tunnels to reduce negative impact. Finally, there are exactly two OpenVPN servers in this setup. This generally should be sufficient, as it doesn't affect the number of actual hosts that you have connected to your multisourced infrastructure. If for some reason you need more than two, it becomes much more difficult to implement route sharing among cube-routed instances. In that case, you might want to consider a messaging system instead of raw sockets (for example, RabbitMQ). All in all, in our case, we found that the overall benefits of a multisourced infrastructure far outweighed the problems caused by these limitations, particularly if you design your architecture with these limitations in mind.

Multisourced infrastructure is a logical extension of its single-sourced predecessor, similar to the distributed service-oriented architecture, which came after monolithic applications and enabled greater flexibility, a faster development cycle and higher availability. It can help you design a smarter architecture and avoid a lock-in to a single hosting provider, on top of a standard time-tested open-source OpenVPN.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Cube-routed on GitHub

Ben Standefer's picture

Dmitry has been very responsive and helpful, and has made cube-routed available for download at http://cohesiveft.com/dnld/cube-routed-0.1.tar.gz

I added it to my GitHub account here: http://github.com/aguynamedben/cube-routed Hope this helps readers of this article!

-Ben

Cube-routed on GitHub

Ben Standefer's picture

Dmitry has been very responsive and helpful, and has made cube-routed available for download at http://cohesiveft.com/dnld/cube-routed-0.1.tar.gz

I added it to my GitHub account here: http://github.com/aguynamedben/cube-routed Hope this helps readers of this article!

-Ben

cube-routed de-open-sourced?

Ben Standefer's picture

I followed along through half this article and it looks like cube-routed has been de-open-sourced by Dmitry/CohesiveFT! I can't find it anywhere, Dmitry's GitHub, cohesiveft.com, or anywhere. Looks like they pulled a really non-FOSS maneuver, please tell me I'm wrong. =( There should at least be a disclaimer at the beginning of the article saying it's impossible to complete.

Ben Standefer

Good Solution on Multisourced Infrastructure

Ahamed Bauani's picture

Hello Dmitriy Samovskiy

Your article is the thing which I was looking for a long time. I have different service on several data center and your article help me to communicate each other in secure way. - Thanks Man!

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix