1. Ssh Forward Agent Log
2. Ssh Forward Agent Account
3. Ssh Config Agent Forwarding
4. Ssh Forward Agent Software
5. Ssh Forward Agent Registration

SSH agent forwarding can be used to make deploying to a server simple. It allows you to use your local SSH keys instead of leaving keys (without passphrases!) sitting on your server. Let's configure and test SSH forwarding using github as remote service to pull our code into the host. Setting up SSH agent forwarding. SSH Agent Forwarding Furthermore, the SSH protocol implements agent forwarding, a mechanism whereby an SSH client allows an SSH server to use the local ssh-agent on the server the user logs into, as if it was local there. Run ssh-agent and add your keys ssh-agent bash ssh-add /.ssh/mykeyidrsa Troubleshooting. If SSH agent forwarding doesn't seem to be working, you can try the following: Make sure there are keys loaded in ssh-agent by typing in the ssh-add -L command. If you rebuilt the server, the knownhosts verification might be failing. With our key agent in place, it's time to enable the final piece of our puzzle: agent forwarding. In short, this allows a chain of SSH connections to forward key challenges back to the original agent, obviating the need for passwords or private keys on any of the intermediate machines.

-->

Most authentication in Windows environments is done with a username-password pair.This works well for systems that share a common domain.When working across domains, such as between on-premise and cloud-hosted systems, it becomes vulnerable to brute force intrusions.

By comparison, Linux environments commonly use public-key/private-key pairs to drive authentication which doesn't require the use of guessable passwords.OpenSSH includes tools to help support this, specifically:

• ssh-keygen for generating secure keys
• ssh-agent and ssh-add for securely storing private keys
• scp and sftp to securely copy public key files during initial use of a server

This document provides an overview of how to use these tools on Windows to begin using key authentication with SSH.If you are unfamiliar with SSH key management, we strongly recommend you review NIST document IR 7966 titled 'Security of Interactive and Automated Access Management Using Secure Shell (SSH).'

About key pairs

Key pairs refer to the public and private key files that are used by certain authentication protocols.

SSH public-key authentication uses asymmetric cryptographic algorithms to generate two key files – one 'private' and the other 'public'. The private key files are the equivalent of a password, and should stay protected under all circumstances. If someone acquires your private key, they can log in as you to any SSH server you have access to. The public key is what is placed on the SSH server, and may be shared without compromising the private key.

When using key authentication with an SSH server, the SSH server and client compare the public keys for username provided against the private key. If the server-side public key cannot be validated against the client-side private key, authentication fails.

Multi-factor authentication may be implemented with key pairs by requiring that a passphrase be supplied when the key pair is generated (see key generation below).During authentication the user is prompted for the passphrase, which is used along with the presence of the private key on the SSH client to authenticate the user.

Host key generation

Public keys have specific ACL requirements that, on Windows, equate to only allowing access to administrators and System.To make this easier,

• The OpenSSHUtils PowerShell module has been created to set the key ACLs properly, and should be installed on the server
• On first use of sshd, the key pair for the host will be automatically generated. If ssh-agent is running, the keys will be automatically added to the local store.

To make key authentication easy with an SSH server, run the following commands from an elevated PowerShell prompt:

Since there is no user associated with the sshd service, the host keys are stored under ProgramDatassh.

User key generation

To use key-based authentication, you first need to generate some public/private key pairs for your client.From PowerShell or cmd, use ssh-keygen to generate some key files.

This should display something like the following (where 'username' is replaced by your user name)

Ssh Forward Agent Log

You can hit Enter to accept the default, or specify a path where you'd like your keys to be generated.At this point, you'll be prompted to use a passphrase to encrypt your private key files.The passphrase works with the key file to provide 2-factor authentication.For this example, we are leaving the passphrase empty.

Now you have a public/private ED25519 key pair(the .pub files are public keys and the rest are private keys):

Remember that private key files are the equivalent of a password should be protected the same way you protect your password.To help with that, use ssh-agent to securely store the private keys within a Windows security context, associated with your Windows login.To do that, start the ssh-agent service as Administrator and use ssh-add to store the private key.

After completing these steps, whenever a private key is needed for authentication from this client, ssh-agent will automatically retrieve the local private key and pass it to your SSH client.

Note

It is strongly recommended that you back up your private key to a secure location,then delete it from the local system, after adding it to ssh-agent.The private key cannot be retrieved from the agent.If you lose access to the private key, you would have to create a new key pairand update the public key on all systems you interact with.

Deploying the public key

To use the user key that was created above, the public key needs to be placed on the server into a text file called authorized_keys under usersusername.ssh.The OpenSSH tools include scp, which is a secure file-transfer utility, to help with this.

To move the contents of your public key (~.sshid_ed25519.pub) into a text file called authorized_keys in ~.ssh on your server/host.

This example uses the Repair-AuthorizedKeyPermissions function in the OpenSSHUtils module which was previously installed on the host in the instructions above.

These steps complete the configuration required to use key-based authentication with SSH on Windows.After this, the user can connect to the sshd host from any client that has the private key.

The Secure Shell is widely used to provide secure access to remotesystems, and everybody who uses it is familiar with routine passwordaccess. This is the easiest to set up, is available by default, butsuffers from a number of limitations. These include both securityand usability issues, and we hope to cover them here.

In this paper, we'll present the various forms of authenticationavailable to the Secure Shell user and contrast the securityand usability tradeoffs of each. Then we'll add the extra functionalityof agent key forwarding, we hope to make the case that using sshpublic key access is a substantial win.

Note - This is not a tutorial on setup or configuration of SecureShell, but is an overview of technology which underlies this system.We do, however, provide some pointers to information on several packageswhich may guide the user in the setup process.

SSH supports access with a username and password, and this is littlemore than an encrypted telnet. Access is, in fact, just like telnet,with the normal username/password exchange.

We'll note that this exchange, and all others in this paper, assumethat an initial exchange of host keys has beencompleted successfully. Though an important part of session security,host validation is not material to the discussion of agent keyforwarding.

All examples start from a user on homepc (perhaps a Windowsworkstation) connecting with PuTTY to a server running OpenSSH.The particular details (program names, mainly) vary from implementationto implementation, but the underlying protocol has been proven tobe highly interoperable.

1	The user makes an initial TCP connection andsends a username. We'll note that unliketelnet, where the username is prompted as part of the connected datastream (with no semantic meaning understood by telnet itself),the username exchange is part of the ssh protocol itself.
2	The ssh daemon on the server responds with ademand for a password, and access to the system has notyet been granted in any way.
3	The ssh client prompts the user for a password,which is relayed through the encrypted connection to the server whereit is compared against the local user base.
4	If the user's password matches the local credential,access to the system is granted and a two-way communications path isestablished, usually to a login shell.

Ssh Forward Agent Account

The main advantage of allowing password authentication is thatit's simple to set up — usually the default — and is easy tounderstand. Systems which require access for many users from many varyinglocations often permit password auth simply to reduce the administrativeburden and to maximize access.

The substantial downside is that by allowing a user toenter a password, it means anybody is allowed to entera password. This opens the door to wholesale password guessing by usersor bots alike, and this has been an increasingly common method of systemcompromise.

Unlike prior-generation ssh worms, which attempted just a few very commonpasswords with common usernames, modern badware has a very extensivedictionary of both usernames and passwords and has proven to be mosteffective in penetrating even systems with 'good' passwords. Only onecompromised account is required to gain entry to a system.

But even putting security issues aside, the other downside of passwordauthentication is that passwords must be remembered and enteredseparately upon every login. For users with just one system to access,this may not be such a burden, but users connecting to many systemsthroughout the day may find repeated password entry tedious.

And having to remember a different password for every system is notconducive to choosing strong passwords.

To counteract the shortcomings of password authentication, ssh supportspublic key access. A user creates a pair of public and private keys, andinstalls the public key in his $HOME/.ssh/authorized_keys fileon the target server. This is nonsensitive information which need notbe guarded, but the other half — the private key — is protectedon the local machine by a (hopefully) strong passphrase.

A public key is a long string of bits encoded in ASCII, and it's storedon one long line (though represented here on three continued lines forreadability). It includes a type (ssh-rsa, or others),the key itself, and a comment:

This key must be installed on the target system — one time— where it is used for subsequent remote access by the holderof the private key.

1	The user makes an initial connection andsends a username along with a request to use a key.
2	The ssh daemon on the server looks in theuser's authorized_keys file, constructs a challenge basedon the public key found there, and sends this challenge back to theuser's ssh client.
3	The ssh client receives the key challenge. Itfinds the user's private key on the local system, but it's protectedby an encrypting passphrase. An RSA key file is named id_rsa onOpenSSH and SecureCRT, keyname.ppk on PuTTY. Other typesof keys (DSA, for instance) have similar name formats.
4	The user is prompted for the passphrase to unlockthe private key. This example is fromPuTTY.
5	ssh uses the private key to construct a keyresponse, and sends it to the waiting sshd on the other end ofthe connection. It does not send the private key itself!
6	sshd validates the key response, and if valid,grants access to the system.

This process involves more steps behind the scenes, but the userexperience is mostly the same: you're prompted for apassphrase rather than a password.But, unlike setting up access to multiple computer systems(each of which may have a different password), using publickey access means you type the same passphraseno matter which system you're connecting to.

This has a substantial, but non-obvious, security benefit: sinceyou're now responsible for just one secret phrase instead ofmany passwords, you type it more often. This makes you morelikely to remember it, and therefore pick a stronger passphraseto protect the private key than you otherwise might.

Trying to remember many separate passwords for different remotesystems is difficult, and does not lend itself to picking strongones. Public key access solves this problem entirely.

We'll note that though public-key accounts can't generally becracked remotely, the mere installation of a public key on atarget system does not disable the use of passwords systemwide.Instead, the server must be explicitly configured to allow onlypublic key encryption by the use of the PasswordAuthentication nokeywords in the sshd_config file.

Ssh Config Agent Forwarding

Now that we've taken the leap into public key access, we'll take the nextstep to enable agent support. In the previous section, the user's privatekey was unlocked at every connection request: this is not functionallydifferent from typing a password, and though it's the same passphraseevery time (which makes it habitual), it nevertheless getstedious in the same manner.

Fortunately, the ssh suite provides a broker known as a 'key agent' whichcan hold and manage private keys on your workstations, and respondingto requests from remote systems to verify your keys. Agents providea tremendous productivity benefit, because once you've unlockedyour private key (one time when you launch the agent), subsequentaccess works with the agent without prompting.

This works much like the key access seen previously, but with a twist.

1	The user makes an initial connection andsends a username along with a request to use a key.
2	The ssh daemon on the server looks [1] in theuser's authorized_keys file, constructs a challenge basedon the key, and sends it [2] back to the user's ssh client.
3	The ssh client receives the key challenge, andforwards it to the waiting agent. The agent, rather than ssh itself,opens the user's private key and discovers that it's protected by apassphrase.
4	The user is prompted for the passphrase to unlockthe private key. This example shows the prompt fromPuTTY'spageant.
5	The agent constructs the key response andhands it back [1] to the ssh process, which sends it off [2]to the sshd waiting on theother end. Unlike the previous example, ssh never sees the privatekey directly, only the key response.
6	sshd validates the key response, and if valid,grants access to the system. Note: the agent still retains the privatekeys in memory, though it's not participating in the ongoing conversation.

As far as the user is concerned, this first exchange is little differentfrom key access shown in the previous section: the only difference iswhich program prompts for the private key (ssh itself versusthe agent).

But where agent support shines is at the next connection requestmade while the agent is still resident. Since it remembers the privatekeys from the first time it was unlocked with the passphrase, it's ableto respond to the key challenge immediately without prompting.The user sees an immediate, direct login without having to type anything.

Many users only unlock their private keys once in the morning whenthey launch their ssh client and agent, and they don't have to enterit again for the rest of the day because the resident agent is handlingall the key challenges. It's wonderfully convenient, as wellas secure.

It's very important to understand that private keys neverleave the agent: instead, the clients ask the agent to performa computation based on the key, and it's done in a way whichallows the agent to prove that it has the private key withouthaving to divulge the key itself. We discuss the challenge/responsein a later section.

Once agent support is enabled, all prompting has now been bypassed,and one can consider performing scripted updates of remote systems.This contrived example copies a .bashrc login config file toeach remote system, then checks for how much disk space is used(via the df command):

Without agent support, each server would require two prompts(first for the copy, then for the remote command execution). Withagent support, there is no prompting at all.

However, these benefits only accrue to outbound connections made fromthe local system to ssh servers elsewhere: once logged into a remoteserver, connecting from there to yet a third server requires eitherpassword access, or setting up the user's private key on the intermediatesystem to pass to the third.

Having agent support on the local system is certainly an improvement,but many of us working remotely often must copy files from one remotesystem to another. Without installing and initializing an agent onthe first remote system, the scp operation will require a passwordor passphrase every time. In a sense, this just pushes the tedium backone link down the ssh chain.

Fortunately, there's a great solution which solves all these issues.

With our Key Agent in place, it's time to enable the final piece ofour puzzle: agent forwarding. In short, this allowsa chain of ssh connections to forward key challenges back to theoriginal agent, obviating the need for passwords or private keyson any intermediate machines.

1	This all starts with an already establishedconnection to the first server, with the agent now holding theuser's private key. The second server plays no part yet.
2	The user launches the ssh client onthe first server with a request to connect to server2,and this passes the username and a use-key request to the sshdaemon (this could likewise be done with the scp securecopy command as well)
3	The ssh daemon consults the user'sauthorized_keys file [1], constructs a key challenge fromthe key, and sends it [2] back down the channel to the clientwhich made the request.
4	This is where the magic occurs: the ssh clienton server receives the key challenge from the target system,and it forwards [1] that challenge to thesshd server on the same machine acting as a key agent. sshd in turn relays [2] thekey challenge down the first connection to the original ssh client.Once back on homepc, the ssh client takes the final step inthe relay process by handing the key challenge off [3] to theresident agent, which knows about the user's private key.
5	The agent running on homepc constructsthe key response and hands it [1] back to the local ssh client,which in turn passes it [2] down the channel to the sshd runningon server. Since sshd is acting as a key agent, it forwards [3]the key response off to the requesting ssh client, which sends it[4] to thewaiting sshd on the target system (server2). Thisforwarding action is all done automatically and near instantly.
6	The ssh daemon on server2 validatesthe key response, and if valid, grants access to the system.

This process can be repeated with even more links in the chain(say, if the user wanted to ssh from server2 to server3),and it all happens automatically. It supports the full suite ofssh-related programs, such as ssh, scp (secure copy),and sftp (secure FTP-like file transfer).

This does require the one-timeinstallation of the user's public — not private!— keys on all the target machines, but this setup cost israpidly recouped by the added productivity provided. Thoseusing public keys with agent forwarding rarely go back.

One of the more clever aspects of the agent is how it can verifya user's identity (or more precisely, possession of a private key) withoutrevealing that private key to anybody. This, like so many otherthings in modern secure communications, uses public key encryption.

When a user wishes access to an ssh server, he presents his usernameto the server with a request to set up a key session. This usernamehelps locate the list of public keys allowed access to that server:typically it's found in the $HOME/.ssh/authorized_keys file.

The server creates a 'challenge' which can only be answered by onein possession of the corresponding private key: it creates andremembers a large random number, then encrypts it with the user'spublic key. This creates a buffer of binary data which is sent tothe user requesting access. To anybody without the private key, it'sjust a pile of bits.

Ssh Forward Agent Software

When the agent receives the challenge, it decrypts it with the privatekey. If this key is the 'other half' of the public key on the server,the decryption will be successful, revealing the original random numbergenerated by the server. Only the holder of the private key could everextract this random number, so this constitutes proof that the user isthe holder of the private key.

The agent takes this random number, appends the SSH session ID (whichvaries from connection to connection), and creates an MD5 hash valueof the resultant string: this result is sent back to the server as thekey response.

The server computes the same MD5 hash (random number + session ID)and compares it with the key response from the agent: if they match,the user must have been in possession of the private key, and accessis granted. If not, the next key in the list (of any) is tried insuccession until a valid key is found, or no more authorized keysare available. At that point, access is denied.

Curiously, the actual random number is never exposedin the client/agent exchange - it's sent encrypted tothe agent, and included in an MD5 hash from the agent.It's likely that this is a security precaution designed tomake it harder to characterize the properties of the randomnumber generator on the server by looking at the the client/agentexchange.

More information on MD5 hashes can be found inAn IllustratedGuide to Cryptographic Hashes, also on this server.

One of the security benefits of agent forwarding is that the user'sprivate key never appears on remote systems or on the wire, even inencrypted form. But the same agent protocol which shields the privatekey may nevertheless expose a different vulnerability: agent hijacking.

Each ssh implementation has to provide a mechanism for clients torequest agent services, and on UNIX/Linux this is typically donewith a UNIX domain socket stored under the /tmp/ directory.On our Linux system running OpenSSH, for instance, we find the file/tmp/ssh-CXkd6094/agent.6094 associated with the SSH daemonservicing a SecureCRT remote client.

Ssh Forward Agent Registration

This socket file is as heavily protected as the operating system allows(restricted to just the user running the process, kept in a protectedsubdirectory), but nothing can really prevent a root user fromaccessing any file anywhere.

If a root user is able to convince his ssh client to use anotheruser's agent, root can impersonate that user on any remote systemwhich authorizes the victim user's public key. Of course, root can dothis on the local system as well, but he can do this directlyanyway without having to resort to ssh tricks.

Several environment variables are used to point a client to an agent,but only SSH_AUTH_SOCK is required in order to use agent forwarding.Setting this variable to a victim's agent socket allows full use of thatsocket if the underlying file is readable. For root, it alwaysis.

One cannot tell just from looking at the socket information which remotesystems will accept the user's key, but it doesn't take too much detectivework to track it down. Running the ps command periodically on thelocal system may show the user running ssh remotesystem,and the netstat command may well point to the user's home base.

Furthermore, the user's $HOME/.ssh/known_hosts file contains alist of machines which the user has a connection: though they may notall be configured to trust the user's key, it's certainly a greatplace to start looking. Modern versions (4.0 and later) of OpenSSHcan optionally hash the known_hosts file to forestall this.

There is no technical method which will prevent a root user from hijackingan SSH agent socket if he has the ability to access it, so this suggeststhat agent forwarding might not be such a good idea when the remote systemcannot be entirely trusted. All ssh clients provide a method to disableagent forwarding.

Up to this point, we've provided essentially no practical how-toinformation on how to install or configure any particular SSHimplementation. Our feeling is that this information is coveredbetter elsewhere, and we're happy to provide some links hereto those other resources.

The Secure Shell: The Definitive Guide, 2 Ed(O'Reilly & Associates).

This is clearly the standout book in its class: it covers Secure Shellfrom A to Z, including many popular implementations. There is nobetter comprehensive source for nearly all aspects of Secure ShellUsage. A worthy addition to any bookshelf.

PuTTY

This is a very popular Open Source ssh client for Windows, and it'snotable for its economy (it will easily fit on a floppy disk).The next resource provides extensive configuration guidance.

Unixwiz.net Tech Tip:Secure Linux/UNIX access with PuTTY and OpenSSH

This is one of our own Tech Tips which is a hands on guide toconfigurating the excellent open sourcePuTTYclient for Windows.Particular coverage was given to public key generation andusage, with plenty of screenshots to guide the way.

Unixwiz.net Tech Tip: Building and configuring OpenSSH

Though this Tech Tip is mainly concerned with configuration of theserver on a UNIX/Linux platform, it also provides coverage of thecommercialSecureCRTWindows client from VanDyke Software (which we use ourselves). Itspecifically details key generation and agent forwarding settings,though briefly.

Unixwiz.net Tech Tip:An Illustrated Guide to Cryptographic Hashes

Though not central to using SSH Agent Forwarding, some coveragecryptographic hashes may help understand the key challenge andresponse mechanism. This Tech Tip provides a good overview ofcrypto hashes in a similarly-illustrated format.

First Published: 2006-02-22