OpenSSH Key Management, Part 3

Agent Forwarding

In this third article in a series, Daniel Robbins shows you how to take advantage of OpenSSH agent connection forwarding to enhance security. He also shares recent improvements to the keychain shell script.

   Support Funtoo!

Get an awesome Funtoo container and support Funtoo! See Funtoo Containers for more information.

Many of us use the excellent OpenSSH as a secure, encrypted replacement for the venerable telnet and rsh commands. One of OpenSSH's more intriguing features is its ability to authenticate users using the RSA and DSA authentication protocols, which are based on a pair of complementary numerical "keys." One of the main appeals of RSA and DSA authentication is the promise of being able to establish connections to remote systems without supplying a password. For more background, see the previous installments of this series on OpenSSH key management, which cover RSA/DSA authentication (Part 1) and ssh-agent and keychain (Part 2), respectively.

Since Part 2 was published on developerWorks in September 2001, and later referenced on Slashdot and Freshmeat (see Resources later in this article for links to these sites), a lot of people have started using keychain, and it's undergone a lot of changes. I've received approximately 20 or so high-quality patches from developers around the world. I've incorporated many of these patches into the keychain source, which is now at version 1.8 (see Resources). I send my sincere thanks to all those who submitted patches, bug reports, feature requests, and notes of appreciation.

Tightening ssh security

In my last article, I've spent some time discussing the security benefits and tradeoffs of running ssh-agent. A few days after the second article appeared on developerWorks, I received an e-mail from Charles Karney of Sarnoff Corporation, who politely informed me of OpenSSH's new authentication agent forwarding abilities, which we'll take a look at in a bit. In addition, Charles emphasized that running ssh-agent on untrusted machines is quite dangerous: if someone manages to get root access on the system, then your decrypted keys can be extracted from ssh-agent. Even though extracting the keys would be somewhat difficult, it is within the skill of professional crackers. And the mere fact that private key theft is possible means that we should take steps to guard against it happening in the first place.

To formulate a strategy to protect our private keys, we must first put the machines we access into one of two categories. If a particular host is well-secured or isolated -- making successful root exploit against it quite unlikely -- then that machine should be considered a trusted host. If, however, a machine is used by many other people or you have some doubts about the security of the system, then the machine should be considered an untrusted host. To guard your private keys against extraction, ssh-agent (and thus keychain) should never be run on an untrusted host. That way, even if the system's security is compromised, there will be no ssh-agent around for the intruder to extract keys from in the first place.

However, this creates a problem. If you can't run ssh-agent on untrusted hosts, then how do you establish secure, passwordless ssh connections from these systems? The answer is to only use ssh-agent and keychain on trusted hosts, and to use OpenSSH's new authentication forwarding abilities to extend passwordless authentication to any untrusted hosts. In a nutshell, authentication forwarding works by allowing remote ssh sessions to contact an ssh-agent running on a trusted system.

Authentication agent forwarding

To get an idea of how authentication forwarding works, let's first take a look at a hypothetical situation where user drobbins has a trusted laptop called lappy, a trusted server called trustbox, and two other untrusted systems that he must access, called notrust1 and notrust2, respectively. Currently, he uses ssh-agent along with keychain on all four machines, as follows:

ssh-agent running on trusted and untrusted machines

The problem with this approach is that if someone gains root access on notrust1 or notrust2, then it is of course possible for this person to extract keys from the now vulnerable ssh-agent process. To fix this, drobbins stops running ssh-agent and keychain on untrusted hosts notrust1 and notrust2. In fact, to be even more careful, drobbins decides to only use ssh-agent and keychain on lappy. This limits exposure of his decrypted private keys, protecting him against private key theft:

ssh-agent running only on lappy; a more secure configuration

Of course, the problem with this approach is that drobbins can now only establish passwordless connections from lappy. Let's see how to enable authentication forwarding and get around this problem.

Assuming that all machines are running recent versions of OpenSSH, we can get around this problem by using authentication forwarding. Authentication forwarding allows remote ssh processes to contact the ssh-agent that is running on your local trusted machine -- rather than requiring a version of ssh-agent to be running on the same machine that you are sshing out from. This usually allows you to run ssh-agent (and keychain) on a single machine, and means that all ssh connections that originate (either directly or indirectly) from this machine will use your local ssh-agent.

To enable authentication forwarding, we add the following line to lappy and trustbox's /etc/ssh/ssh_config. Note that this is the config file for ssh (ssh_config), not the ssh daemon sshd (sshd_config):

   ssh_config

ForwardAgent Yes

Now, to take advantage of authentication forwarding, drobbins can connect from lappy to trustbox, and then from trustbox to notrust1 without supplying passphrases for any of the connections. Both ssh processes "tap in" to the ssh-agent running on lappy:

user $ ssh drobbins@trustbox
Last login: Wed Sep 26 13:42:08 2001 from lappy

Welcome to trustbox!
user $ ssh drobbins@notrust1
Last login: Tue Sep 25 12:03:40 2001 from trustbox

Welcome to notrust1!
user $

If you try a similar configuration and find that agent forwarding isn't working, try using ssh -A instead of plain old ssh to explicitly enable authentication forwarding. Here's a diagram of what went on behind the scenes when we logged in to trustbox and notrust1 using authentication forwarding, above:

Agent forwarding in action

As you can see, when ssh connected to trustbox, it maintained a connection to the ssh-agent running on lappy. When an ssh connection was made from trustbox to notrust1, this new ssh process maintained the authentication connection to the previous ssh, effectively extending the chain. Whether this authentication chain can be extended beyond notrust1 to other hosts depends on how notrust1's /etc/ssh/ssh_config is configured. As long as agent forwarding is enabled, all parts of the chain will be able to authenticate using the ssh-agent running on the trusted lappy.

Advantages of agent connection forwarding

Authentication forwarding offers a number of security advantages not touched on here. To convince me of the importance of agent connection forwarding, Charles Karney shared with me these three security advantages:

The private key is stored only on the trusted machine. This prevents malicious users from grabbing your encrypted key from disk and attempting to crack the encryption. ssh-agent runs only on the trusted machine. This prevents an intruder from doing a memory dump of a remote ssh-agent process and then extracting your decrypted private keys from the dump.

Since you only need to type in the passphrase on your trusted machine, you prevent any keystroke loggers from stealthily grabbing your passphrase as it is entered. The one drawback to relying on authentication agent connection forwarding is that it doesn't solve the problem of allowing cron jobs to take advantage of RSA/DSA authentication. One solution to this problem is to set up all cron jobs that need RSA/DSA authentication so that they execute from a trusted machine on your LAN. If necessary, these cron jobs can use ssh to connect to remote systems to automate backups, synchronize files, and so on.

Now that we've looked at authentication agent connection forwarding, let's turn to recent improvements made to the keychain script itself.

Keychain functionality improvements

Since the time this article was originally written in 2001, Keychain has become a successful Open Source project, and now supports nearly every version of UNIX (including Linux, BSD, Solaris, IRIX, and AIX as well as other UNIX platforms,) and has lots of advanced features. These features include support for gpg-agent, as well as a number of new command-line options, which you can learn about by typing keychain --help or reading the keychain man page (man keychain).

The official home for Keychain is on the Keychain page on the Funtoo wiki. Check there for updates and more information on this useful tool.

Conclusion

This column concludes my coverage of OpenSSH. Hopefully, you've learned enough about it to start using OpenSSH in an effective way to secure your systems.

Resources

Learn more about Keychain, a tool for managing SSH and GPG keys, on the official Keychain project page.

   Note

Browse all our available articles below. Use the search field to search for topics and keywords in real-time.

	Article	Subtitle
	Article	Subtitle
	Awk by Example, Part 1	An intro to the great language with the strange name
	Awk by Example, Part 2	Records, loops, and arrays
	Awk by Example, Part 3	String functions and ... checkbooks?
	Bash by Example, Part 1	Fundamental programming in the Bourne again shell (bash)
	Bash by Example, Part 2	More bash programming fundamentals
	Bash by Example, Part 3	Exploring the ebuild system
	BTRFS Fun
	Funtoo Filesystem Guide, Part 1	Journaling and ReiserFS
	Funtoo Filesystem Guide, Part 2	Using ReiserFS and Linux
	Funtoo Filesystem Guide, Part 3	Tmpfs and Bind Mounts
	Funtoo Filesystem Guide, Part 4	Introducing Ext3
	Funtoo Filesystem Guide, Part 5	Ext3 in Action
	GUID Booting Guide
	Learning Linux LVM, Part 1	Storage management magic with Logical Volume Management
	Learning Linux LVM, Part 2	The cvs.gentoo.org upgrade
	Libvirt
	Linux Fundamentals, Part 1
	Linux Fundamentals, Part 2
	Linux Fundamentals, Part 3
	Linux Fundamentals, Part 4
	LVM Fun
	Making the Distribution, Part 1
	Making the Distribution, Part 2
	Making the Distribution, Part 3
	Maximum Swappage	Getting the most out of swap
	On screen annotation	Write on top of apps on your screen
	OpenSSH Key Management, Part 1	Understanding RSA/DSA Authentication
	OpenSSH Key Management, Part 2	Introducing ssh-agent and keychain
	OpenSSH Key Management, Part 3	Agent Forwarding
	Partition Planning Tips	Keeping things organized on disk
	Partitioning in Action, Part 1	Moving /home
	Partitioning in Action, Part 2	Consolidating data
	POSIX Threads Explained, Part 1	A simple and nimble tool for memory sharing
	POSIX Threads Explained, Part 2
	POSIX Threads Explained, Part 3	Improve efficiency with condition variables
	Sed by Example, Part 1
	Sed by Example, Part 2
	Sed by Example, Part 3
	Successful booting with UUID	Guide to use UUID for consistent booting.
	The Gentoo.org Redesign, Part 1	A site reborn
	The Gentoo.org Redesign, Part 2	The Documentation System
	The Gentoo.org Redesign, Part 3	The New Main Pages
	The Gentoo.org Redesign, Part 4	The Final Touch of XML
	Traffic Control
	Windows 10 Virtualization with KVM