Building a Multisourced Infrastructure Using OpenVPN

How to use OpenVPN to take your hosting to the next level.

To address this issue, we developed a dynamic routing dæmon called cube-routed (download it from It shares routing information between vpnsrvA and vpnsrvB and adjusts routing tables depending on which client connects to which server in near real time. Its internal structure is not very complex. One thread connects to a local OpenVPN dæmon process via its management interface (see the management option in the OpenVPN configuration file) and regularly runs the status command to update the list of clients connected locally. Another thread publishes this information for the remote instance of cube-routed. The third thread regularly reads a list of connected clients from the remote instance of cube-routed. Finally, the fourth thread adjusts the local routing table based on the following two rules: 1) adds a host route for every host connected to the remote OpenVPN server and 2) deletes the host route for every host connected to the local OpenVPN server.

cube-routed instances will exchange information over the management subnet we selected earlier. Create a second tunnel tun1 between vpnsrvA and vpnsrvB. vpnsrvA can be a server with IP, and vpnsrvB is its client with IP You can use the configuration files from Listings 1 and 2 as a basis, but remember to adjust the IP addresses and select a different port—for example, you could add port 11940 to both the server and client. Start both OpenVPN dæmons, and use ping and ping to verify connectivity between them.

Now, create configuration files for cube-routed on both vpnsrvA and vpnsrvB, as shown in Listings 4a and 4b, and start both instances as root with the path to the configuration file as the only parameter (note that OpenVPN must already be running, and the tun0/tun1 interfaces on both vpnsrvA and vpnsrvB must be up).

Once you start everything and after several minutes of initial convergence time, host1 from the example above will be able to communicate with host2, even though they connected to different OpenVPN servers. Thus, you've achieved a fully fault-tolerant virtual LAN connectivity, with data traffic encryption as an added bonus.


This implementation is not without its limitations. First, applications that use broadcast or multicast will not work with OpenVPN's tun device. You can use the same network layout as described here, but instead of tun, experiment with OpenVPN's tap device to work around this. Second, latency of network links over the public Internet is significantly higher than that of Ethernet. If this is an inherent requirement for your application, you probably should leave this part of your infrastructure single-sourced. Third, because we use UDP-based tunnels, OpenVPN links will tend to go up and down more often than Ethernet, especially during times of network congestion. You can implement data caches, avoid long-lived TCP connections, focus on network exception-handling logic and experiment with TCP tunnels to reduce negative impact. Finally, there are exactly two OpenVPN servers in this setup. This generally should be sufficient, as it doesn't affect the number of actual hosts that you have connected to your multisourced infrastructure. If for some reason you need more than two, it becomes much more difficult to implement route sharing among cube-routed instances. In that case, you might want to consider a messaging system instead of raw sockets (for example, RabbitMQ). All in all, in our case, we found that the overall benefits of a multisourced infrastructure far outweighed the problems caused by these limitations, particularly if you design your architecture with these limitations in mind.

Multisourced infrastructure is a logical extension of its single-sourced predecessor, similar to the distributed service-oriented architecture, which came after monolithic applications and enabled greater flexibility, a faster development cycle and higher availability. It can help you design a smarter architecture and avoid a lock-in to a single hosting provider, on top of a standard time-tested open-source OpenVPN.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Cube-routed on GitHub

Ben Standefer's picture

Dmitry has been very responsive and helpful, and has made cube-routed available for download at

I added it to my GitHub account here: Hope this helps readers of this article!


Cube-routed on GitHub

Ben Standefer's picture

Dmitry has been very responsive and helpful, and has made cube-routed available for download at

I added it to my GitHub account here: Hope this helps readers of this article!


cube-routed de-open-sourced?

Ben Standefer's picture

I followed along through half this article and it looks like cube-routed has been de-open-sourced by Dmitry/CohesiveFT! I can't find it anywhere, Dmitry's GitHub,, or anywhere. Looks like they pulled a really non-FOSS maneuver, please tell me I'm wrong. =( There should at least be a disclaimer at the beginning of the article saying it's impossible to complete.

Ben Standefer

Good Solution on Multisourced Infrastructure

Ahamed Bauani's picture

Hello Dmitriy Samovskiy

Your article is the thing which I was looking for a long time. I have different service on several data center and your article help me to communicate each other in secure way. - Thanks Man!

Geek Guide
The DevOps Toolbox

Tools and Technologies for Scale and Reliability
by Linux Journal Editor Bill Childers

Get your free copy today

Sponsored by IBM

8 Signs You're Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
On Demand
Moderated by Linux Journal Contributor Mike Diehl

Sign up and watch now

Sponsored by Skybot