SSH Hopping using SSH Agent Forwarding

It’s one of the coolest feature when you are using SSH-keys Base Authentication. SSH Hopping using SSH Agent Forwarding is a must skill and very time saver, especially if you have to create and manage Multiple VPS instances using SSH-keys. I assure you that SSH Agent Forwarding feature is a must when it comes to manage Multiple Servers. For only one VPS, you don’t need SSH Agent Forwarding.

Objectives:

1. Understanding SSH Agent Forwarding Concept

2. Enabling SSH Agent Forwarding in Putty, WinSCP, or Bitvise

3. Enabling SSH Agent Forwarding on Linux/UNIX Stations

4. Allowing SSH Agent Forwarding on Linux/UNIX Servers

5. SSH Agent Forwarding Security Best Practices

Prerequisites:

A. Basic Debian or Red Hat Linux Knowledge

B. Login to your DigitalOcean or Vultr Account

Recommendations:

For better performance, use VPS with at least 2 CPUs, 4G Memory, 1G Bandwidth, and SSD Storage drive.

Understanding SSH Agent Forwarding Concept

Suppose I have few VPS instances and all of them were created or managed using SSH-Keys, if one day I want to transfer some files between node1 and node2 using scp command line, the moment I try to run such command between 2 VPS instances, I will be prompted by node2 to enter a password.

So, If my VPS instances using SSH-Keys, and SSH Password Authentication is not possible since it’s disabled, then how I’m suppose to transfer files among my VPS instances if SSH Password Authentication is disabled? As mentioned previously, it is very wise to disable SSH Password Authentication after you have successfully tested SSH-Keys Authentication Base.

However, disabling SSH Password Authentication while managing Multiple VPS instances requires some preparation, one of them is to enable and get familiar with SSH Agent Forwarding feature if you are planning to manage multiple servers.

In order to copy files between node1 and node2, I have 2 choices: Copy my Private Key to node1, therefore, I can authenticate to node2, or use SSH Agent Forwarding instead. Remember, copying Private Keys to Public VPS instances is the worst security practice, rather, use SSH Agent Forwarding feature instead.

You already know what SSH Agent is (it saves your private key and passphrase in memory in order to prevent entering passphrase every time you login). Now, add forwarding mechanism to SSH Agent, can you imagine what is going to happen? don’t confused between SSH Agent and SSH Agent Forwarding.

SSH Agent Forwarding Detailed Process

Assuming you have Enabled SSH Agent Forwarding and you wanted to copy some files between the VPS nodes, the moment you run scp command between node1 and node2, the following brief steps are going to happen if SSH Agent Forwarding is enabled:

1. SSH client at node1 will create SSH socket between node1 and your Admin Station.

2. At this point, the SSH Agent Forwarding feature will forward the Authentication request you are trying to make from node1 to node2 back to your Admin Station for processing (as if the Private Key is now located at node1).

3. node1 SSH client will route private key request through node1 SSH socket back to your Admin station in order to authenticate and login to node2.

4. Since node2 was able to compare and match the Public Key that is being used by node1’s ssh client to it’s authorized_keys file, access will be granted.

SSH Agent Forwarding process acted as if your Private Key was located on node1, therefore, node2 have granted you access to use scp command to transfer files between the 2 servers. I believe, this is the coolest part of SSH 🙂

Enabling SSH Agent Forwarding in Putty, WinSCP, or Bitvise

At first, you will need to enable SSH Agent Forwarding feature at the ssh client side, then enable it at the VPS side.

a. Putty Side

Under “Authentication parameters” check the 2 boxes, select both of them to enable SSH Agent Forwarding.

b. WinSCP Side

c. Bitvise Side

Enabling SSH Agent Forwarding on Linux/UNIX Stations

When dealing with Linux/UNIX Admin Stations, it’s important to remember that SSH includes 2 config files:

a. SSH Client uses ssh_config – used to connect to your VPS

b. SSH Server uses sshd_config – used as your personal SSH Server

Therefore, to enable SSH Agent Forwarding, the first part is to enable it at your Admin Station by editing only the SSH client’s config file ssh_config.

At your Linux/UNIX station:

Edit SSH Client’s Config File

nano /etc/ssh/ssh_config

Ctrl-w and search for FrowardAgent, if it’s commented, uncomment and change it’s value from no to yes.

[...]
Host *
   ForwardAgent yes
[...]

Save: Ctrl-X, Hit Y Key, and Enter.

With this, your admin station is ready to accept ssh-agent forwarding from your VPS. Next, let’s enable SSH Agent Forwarding at the VPS (Server) side.

Allowing SSH Agent Forwarding on Linux/UNIX Servers

Allow SSH Agent Forwarding in both files: ssh_config and sshd_config. Yes, VPS runs both: SSH client and Server versions, otherwise, how would you be able to SSH from node1 to node2. node1 at this point will call ssh client, hence, it has it’s own config file.

Run the following steps ( 1 to 2)

1. Edit SSH Client’s Config File

nano /etc/ssh/ssh_config

Ctrl-w and search for FrowardAgent, if it’s commented, uncomment and change it’s value from no to yes.

[...]
Host *
   ForwardAgent yes
[...]

Save: Ctrl-X, Hit Y Key, and Enter.

2. Edit SSH Server’s Config File

nano /etc/ssh/sshd_config

Ctrl-w and search for AllowFrowardAgent and change it’s value from no to yes. If you can’t find it, add it to the end of the file.

[...]
AllowForwardAgent yes
[...]

Save: Ctrl-X, Hit Y Key, and Enter.

Checking Configuration and Restarting SSH Server

Check your SSH configuration before restarting:

/usr/sbin/sshd -t

If there’s an error somewhere, it will be displayed on the screen as shown in the example below:

[email protected]:~# /usr/sbin/sshd -t
/etc/ssh/sshd_config: line 28: Bad configuration option: 8StrictModes
/etc/ssh/sshd_config: terminating, 1 bad configuration options

At line 28, I have typed number 8 by mistake before StrictModes. It’s good practice to check and fix SSH configuration if found before restarting the service. You know what is going to happen if you restart your SSH service with bad configuration 😉

Debian/Ubuntu

service ssh restart

CentOS

service sshd restart

SSH Agent Forwarding Security Best Practices

Private Keys Should Not be Used at the VPS

Instead, use SSH Agent Forwarding to authenticate among your Servers. Is not recommended to go more than 2 hops though, the moment you connect to your first VPS (node1), that is the first hop, and once you connect from node1 to your second VPS (node2), that is the second hop.

SSH Agent Forwarding should be used only for copying, backup, or any other means of file transfer except management. If you needed to manage node2, then what I recommend is to connect straight to node2 from your Admin Station.

Do you trust the Remote Server?

That’s where the risk resides, when you ssh from node1 to node2, there will be an active socket as described above between your Admin Station and the Server. If someone else control node2 as root, then this can be a serious risk Only if you don’t control node2 and don’t trust node2’s SysAdmin.

Node2’s SysAdmin can use your socket session and your Private key without the passphrase! Therefore, always ssh from your node to your node. SSH among the nodes that you control and you are the SysAdmins on those nodes.

Ask the Remote Server’s Sysadmin to Add your Public

Use SSH Agent Forwarding on Servers you control, even if you trust the SysAdmin of the remote system, you still can’t tell if his/her system is clean or infected. Instead, to connect and support someone’s else system, ask them to add your Public Key (especially meant for support) to their system and connect straight from your Admin Station to their system.

By the way, I recommend to create a VM at your laptop specially for Support purpose, then at this VM station, create different users for different customers, hence, each user can have its own SSH-keys Pair specific for that customer.

Subject Related

By | Wikipedia SSH-Agent | UnixWiz | Git Hub | SSH Agent is Dangerous | EveryBody.org