Use SSH to Cross a Suspect Host Securely

SSH tunnels can be used to connect securely to a server behind a firewall in such a way that the firewall can't observe the data. This is also true and useful if you know an attacker has gained control of the firewall.

Figure 1 displays the example graphically. The initial ssh command builds the SSH connection for the tunnel, the tunneled connection and the forwarded connection. The second ssh command initiates a new connection through the tunnel and forwards the connection directly to the machine you are trying to reach. The forwarded connection is in red to show that it is unencrypted, but the blue SSH connection going through it is secure.

Figure 1. Tunneling across a Compromised Host (Brian Cluff, LJ2009@Macrosift.com, created the image for this article.)

This tunnel could work for any TCP protocol, but the packets from Gateway to Server and back to Gateway are unencrypted. That means unencrypted services would not be secure between Gateway and Server, even though you're using an SSH tunnel. You may want to review the -L option section of the SSH man page and consider this a bit to convince yourself that the Server/Gateway connection (the red part in Figure 1) is unencrypted. Anything that isn't secure or that gets decrypted on Gateway can be read by the attacker if the attacker has root access on Gateway. The attacker also can read much of that simply by having access to your account on Gateway even without root access on Gateway.

Before connecting to the tunnel, you need to make sure you have Server's host public key registered as a key for localhost. If you tried to pull the public key out of Gateway's known_hosts file, the attacker can give you a bogus key, so you need to get the key another way.

Therefore, it's best to have Server's public key already. If you don't already have the public key, make sure to acquire it or the fingerprint for the key from a secure, trusted source. For instance, you could list the SSH key fingerprints for all of your servers on a secure Web page.

I suggest registering Server's public key in your known_hosts file under the real server name as well as under localhost. By registering under the real server name, you can figure out to which server a key belongs with the ssh-keygen command.

The following command checks your $HOME/.ssh/known_hosts file for a public key for Server. The command also reports the line number for the entry:

ssh-keygen -F Server

The entry then can be copied. By replacing the hostname portion of the copied entry with localhost, you create an entry for that key on localhost. See the Obtaining the Public Key sidebar for information on how to obtain a server's public key securely.

If your known_hosts file has hostnames that look like the keys, you have hashed hostnames. Use the -H option to ssh-keygen to hash your new localhost entry. The following command translates plain-text hostnames in your known_hosts file to hashed hostname entries:

ssh-keygen -H

Now with the tunnel set up, you can connect to SSH on Server, and because it's an SSH connection, it's encrypted:

ssh -p 3333 localhost

Again, localhost here is from Netbook's perspective, so that command connects to port 3333 on Netbook. The last SSH “tunnel” command created a tunnel from that port over to the SSH port on Server via Gateway—meaning that this command uses the tunnel to connect to Server's SSH server via port 3333 on Netbook.

Even though the tunnel passes through Gateway, it is unreadable by Gateway. Just as SSH connections normally travel across untrustable routers, yet are secure, the connection through Gateway to Server is secure. The SSH connection via the tunnel is never unencrypted on Gateway, so the attacker can't compromise it. But again, remember to have verified the host key for Server already, or the attacker can compromise the connection with a man-in-the-middle attack.

With the connection to Server, you now could set up a SOCKS proxy to allow applications running on Netbook to be effectively on your work's network. See the SOCKS Proxy sidebar for more information.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

"you can ssh safely across a

Anonymous's picture

"you can ssh safely across a compromised host"

You keep using that word. I do not think it means what you think it means.

Not so troubling

lufthans's picture

WARNING: Anything communicated to authenticate to a compromized host could potentially be captured by the attacker and used against other hosts.

As the author, I do not mean for my article to dismiss the importance of shutting down a compromised host. The article explains that you might need to temporarily hop through the compromized host and describes how to do so safely.

I agree in pulling a machine off the air and even specify doing so in the 3rd paragraph. As a sysadmin I realize there are often steps to take before doing so. This article describes a way to safely connect to the internal network if needed, even if it's not the best position to be in.

The authentication credentials for the bastion host and the chewy center of your network should be different :).

I warn against forwarding agent connections through the compromized host. I also cover safely verifying the internal server's host key.

I see that I did leave out specific warnings about authentication credential ( such as personal key and password ) reuse. You are correct, if they are the same on both Gateway and Server the attacker could gain the credentials when you connect with Gateway and then use them on Server. I should have covered that in the article.

WARNING: Anything communicated to authenticate to the Gateway could potentially be captured by the attacker and used against other hosts.

Better?

The article does not pretend there are no security risks at all and specifically warns against a couple.

I generally believe bastion hosts on the edge of the network should be treated as if they could be compromized any minute. The methods in this article allow access to network resources without giving the bastion host access to those resources even if it's the gateway to reach them.

Very Troubling Article

Anonymous's picture

This article does a fine job of explaining ssh tunneling. However, I am troubled by two issues with the technique suggested by the author.

First, that the author simply dismisses the importance of immediately removing a suspect host from the network. This is a red flag to be highly skeptical of any security advice to follow. Best practices are ignored at one's own peril.

Second, the omission of any discussion of the risks of establishing an ssh connection to the compromised host.

Yes, the tunneled ssh connection is not much different than an ssh session crossing unknown internet hosts. However in order to create that tunnel, one must first authenticate to the compromised host. This risks of this cannot be understated.

Well known are many rootkits which alter or replace sshd in very nasty ways. At a minimum, you are handing over your authentication credentials. By establishing a tunnel to an internal host you are potentially alerting your attacker to another target of interest which may accept those credentials.

The entire tone of the article presents the author's tunneling technique as if it involved no security risks at all. This is simply not true. The security best practice in this case is well established: Never open a network login to a compromised host.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix