Everything you need to know about SSH Key Management

What is SSH?

Secure Socket Shell (SSH), commonly known as Secure Shell, is a cryptographic protocol that enables network services to be accessed securely through an insecure communication channel. It is mostly used for secure data exchange, login, and other services over a computer network between two machines.  

It uses public-key cryptography to provide mutual and secure authentication of clients with servers across networks over an unsecured channel. SSH is usually built-in by default to operating systems such as Linux, Unix, and MacOS. 

Windows now supports SSH via both PowerShell and OpenSSH; i.e., with newer versions of Windows, it’s no longer essential to install third-party utilities such as PuTTY. This means that you may now create SSH (Secure Shell) links straight from the command line (Windows Terminal or PowerShell), shortening administrative workflow while boosting your system’s security. For some time, PuTTY has served as an SSH client under Windows. Nonetheless, the native OpenSSH project replaces PuTTY’s entire functionality range, i.e., no more installations are required. 

Key features of SSH 

SSH (Secure Shell) is a protocol that ensures secure communication over a network, commonly used for accessing remote servers. It provides a range of features to guarantee the confidentiality, security, and integrity of data. Below are the key features that make SSH an essential tool for secure network communication: 

Encryption 

SSH encrypts the data being transmitted, ensuring that it remains confidential. This means that even if the data is intercepted, unauthorized parties cannot read or decipher it. By using strong encryption algorithms, SSH ensures that sensitive information, such as login credentials and commands, is securely transmitted over potentially insecure networks. 

Authentication 

Authentication in SSH is performed using public-key cryptography, which is significantly more secure than traditional password-based authentication. In this method, the client and server exchange keys to verify identities. Only authorized users with the correct private key can establish a secure connection, protecting against unauthorized access and brute force attacks. 

Integrity 

SSH ensures data integrity by verifying that the information being sent or received has not been tampered with during transmission. It uses cryptographic hash functions to validate the integrity of the data, guaranteeing that the data remains intact and has not been altered in any way. This feature is essential in preventing attacks such as data manipulation or man-in-the-middle attacks. 

What are SSH Keys? 

SSH key refers to a cryptographic key used for authenticating secure communication between the systems participating over the SSH (Secure Shell) protocol. It provides a highly secure alternative to password-based authentication, which ensures that only trusted and authenticated users can access a remote system. 

This is because SSH keys use a cryptographic key pair (private and public keys), which eliminates the risk of credentials being stolen through phishing or guessed through brute-force methods. Private keys remain securely stored on the user’s system or more securely in the Hardware Security Module (HSM), while public keys are shared with the server, ensuring authentication without transmitting sensitive information. 

SSH keys are based on public key cryptography and use two keys together, pairing a public key with a private key, to make an upwardly strong connection.  

Public Key: As the name implies, public key is available publicly on the server. It doesn’t need to be kept secret and is shared openly across the internet. If we talk about SSH key management, the public key is used to verify the identity of the client to the server.  

Private Key: The private key should be kept secret and should only be accessed by the user who owns it. It is always stored in the user’s device or in Hardware Security Modules (HSMs) and is used by the user to prove their identity while initiating the connection.  

How does SSH work?

SSH uses public-key cryptography, encryption, and authentication techniques to ensure the confidentiality, integrity, and authentication of data exchanged between the systems to provide resistance to attacks for security. 

Here is the stepwise breakdown of how SSH works:

1. Initiating the connection

   The client sends a request to the server’s IP address over TCP port 22 (the default SSH port). To enhance security, the default port can be changed to any other non-standard port. The protocol initiates a TCP connection between the client and the server.

2. Server sends the public key and encryption protocol

   As soon as the connection is requested, the server responds by sending back some major information.

   • Public host key

     The server sends its public host key to the client, which helps verify the server’s identity and initiates the encryption process.

   • Encryption Algorithms

     The server provides a list of supported encryption algorithms and the hashing algorithms with their version, and if the client has a similar pair of protocols, then they are used for the session. 

3. Client verifies the server

   The server’s public key is checked against a file on the client machine that stores known hosts. If the public key of the server matches a key stored in the file, the server is considered legitimate for the client.

   If the server’s key doesn’t match the known host’s file, the client usually asks the user to confirm that they want to trust the new server and store its public key for future use.

   

4. Key Exchange process

   A key exchange process is performed to establish a shared session key for encryption. It is typically done using.

   • Diffie-Hellman key exchange is a mathematical process that allows both the client and server to generate a shared secret key over an insecure connection without exchanging the key itself.
   • Elliptic-Curve Diffie-Hellman (ECDH), a more efficient version of Diffie-Hellman, provides the same level of security with shorter key lengths, making it faster.

5. Creating and establishing the secure channel

   Both the client and server initiate a secure and encrypted communication channel using their session key from the key exchange. This session key is symmetric, i.e., it is used to both encrypt and decrypt the messages sent during the session.

6. Client Authentication

   After the secure channel is established, the client must authenticate to prove its identity to the server. There are various ways to authenticate clients, i.e., Password-based Authentication and SSH Key Pair. The preferred way of authentication is SSH Key Pair. Here, the client uses a private key, stored on the local machine or an HSM, and the corresponding public key, which must already be stored on the server.

7. Session Established

   As soon as the session is established, the client can communicate with the server. The client can perform various tasks such as sending commands, executing programs, transferring files, or even creating an encrypted tunnel (port forwarding) over the SSH connection.

8. Maintain session integrity

   During the session, SSH uses Message Authentication Codes (MACs) to ensure the integrity and authenticity of the transmitted data. MACs provide a way to detect tampering or modification of data in transit.

   • Each data packet that is transmitted is accompanied by a MAC. 
   • The MAC is generated using a hashing algorithm (like SHA-256 or SHA-512) and a shared secret between the client and server.
   • Upon receiving a packet, the server computes its own MAC for that packet and verifies it against the MAC that the client sent. If they do, that indicates that the data has not been tampered with.

9. Session Termination

   Once terminated, the encrypted session ends, and the shared session key is discarded, ensuring no further communication can take place without re-establishing a new connection.

SSH Authentication Methods

There are various methods of authentication to securely verify the identity of a user or system before granting access. Using these methods, we can ensure that only authorized users can establish an SSH connection to a remote server.  

Two major SSH Authentication methods are:  

Password-Based Authentication 

This is the simplest form of authentication, where users authenticate themselves by providing a username and password. The credentials are shared through an encrypted tunnel for security reasons. If the credentials provided by the client match the database, the client is marked as an authenticated client for SSH communication. 

Advantages

1. Easy to Setup and Use

   Password-based authentication is one of the simplest methods to implement. It does not require specialized knowledge or complex configurations, making it accessible to users of all technical levels. Creating a password and linking it to a username is quick and intuitive, allowing users to start accessing systems immediately. This universality makes it compatible with most platforms and applications, ensuring seamless integration in various environments.

2. No Need for Additional Software or Key Management

   Unlike other authentication methods, such as certificate-based or hardware token systems, password-based authentication does not require any external tools or devices. This removes the need for organizations to invest in additional hardware or software for key generation and storage. Users also benefit from not having to carry physical keys or devices, reducing the risk of loss or mismanagement. This simplicity contributes to its cost-effectiveness, particularly for small businesses or individuals.

Disadvantages

1. Vulnerability to Weak Passwords

   Weak or easily guessable passwords expose systems to brute-force attacks. Hackers use automated tools to systematically guess passwords, exploiting common patterns, dictionary words, or reused credentials. While account lockout mechanisms can provide some protection, they may not always be implemented effectively. Additionally, password reuse across multiple platforms increases the impact of a single breach.

2. Frequent Password Updates Required

   To prevent such an event, plenty of organizations perform periodic password reset policies. This makes the chances of long-term exposure/article much less but leads to a lot of frustration for users who also have to remember and manage multiple complex passwords. This tends to result in bad practices such as writing down passwords or using sequential tweaks (e.g., “Password1” to “Password2”) that undermine the security architecture.

3. Limited Protection Against Advanced Attacks

   Password-based systems are highly susceptible to phishing, credential stuffing, and malware-based attacks like keyloggers. Cybercriminals can trick users into sharing their passwords or capture credentials via infected devices. Without additional layers of protection, such as two-factor authentication (2FA), passwords alone cannot defend against sophisticated threat vectors.

4. Dependence on Secure Transmission Channels

   The security of password-based authentication is based on secure communication channels. If the encrypted tunnel used to transmit credentials is compromised, sensitive information can be intercepted by attackers. Organizations must invest in maintaining the best encryption protocols, which require regular updates and monitoring to counteract evolving threats.

SSH Key Pair 

SSH Key Pair Authentication is one of the most secure and widely used SSH authentication methods. It uses a cryptographic key pair for the authentication process. The process includes:  

1. Connection Initiation: The client initiates the connection with the server.  

2. The server sends a challenge: The server checks for the public key in the authorized_key file that matches the user’s account. Here, the server then generates a challenge message, encrypts it using the public key, and sends the encrypted message to the client.  

3. Client Response: Now, the client decrypts the challenge message using its private key and then sends the decrypted message back to the server.  

4. Server Verification: The server then verifies the decrypted message sent by the client, and if the message matches the original challenge message, then it grants access to that client. 

Advantages

1. Enhanced Security

   SSH key pair authentication is far more secure than using a password-based method. It is based on the knowledge of a private key that is highly complex from the cryptographic point of view and thus very difficult to brute-force or guess, so this way of authentication is immune to the majority of common attacks.

2. Elimination of Password Transmission

   This method eliminates the need to share or transmit passwords over the network, reducing the risk of interception or exposure. The absence of plaintext passwords results in a reduction of vulnerabilities even if an attacker gains access to the network.

3. Convenience for Automation

   When automating processes on servers, such as running scripts or configuring cron jobs, key-based authentication is very useful. After the keys are set up, you have a smooth process flow without having to manually enter passwords.

Disadvantages

1. Risk of Private Key Compromise

   If the private key ends up in the wrong hands, it basically gives attackers free rein to the client’s systems. It is important to encrypt the private key and safely store it to prevent unauthorized access.

2. Key Management Challenges

   With the increasing number of users and systems, managing multiple key pairs (key sprawl) becomes cumbersome. To secure the system, organizations need to adopt a clear process of managing key distribution, rotation, and revocation.

Need for SSH keys Protection?

SSH keys are critical in ensuring secure remote communications. Their protection is essential to prevent any unauthorized access, data breaches, and security incidents.  

Some major needs for SSH key protection are:

Prevent Unauthorized access 

Because SSH keys have critical infrastructure (servers, cloud instances, network devices, etc.), they are more secure than password-based authentication. It provides improved and more secure access control methods. 

Mitigate the Risk of Data Breaches 

One of the most significant threats when SSH keys are compromised is the Data Breach. If, somehow, the SSH keys are stolen or leaked, attackers may gain the power to breach the systems and access the sensitive data of any organization. 

Guard Against Malware and System Takeovers

Attackers are best at installing malware, initiating DDoS attacks, or taking full access to the remote system using the compromised keys. Therefore, proper SSH key protection must be implemented.  

Encrypt Private keys using a Strong Passphrase 

Encrypting private SSH keys with strong passphrases is essential for securing access to sensitive systems. A strong passphrase adds an extra layer of protection, ensuring that even if the private key is stolen or exposed, it remains unusable without the passphrase. This prevents unauthorized access, mitigates the risk of key misuse, and provides critical time to revoke compromised keys. By using strong passphrases, organizations enhance security, reduce vulnerabilities, and align with best practices for protecting privileged access. 

Regularly reviewing SSH keys

Periodic review of SSH keys to remove unneeded or obsolete keys is an essential security practice. If you don’t remove unused keys, they can become an attack vector, especially when they are from old employees or old processes. Outdated keys may use weak ciphering or could be compromised in such a way that no one is noticed, causing major security threats. 

Hence, SSH key protection is an essential part of SSH key management for maintaining the security of the system, data, and infrastructure. Various practices for proper SSH key management could include using a strong passphrase, rotating the keys regularly, and implementing centralized storage for the keys. 

SSH vs. SSL/TLS certificates – what’s the difference?

Both SSH (Secure Shell) and SSL/TLS (Secure Sockets Layer/Transport Layer Security) are cryptographic protocols designed to secure communication over a network. As stated, SSH is used to establish a secure connection between two remote systems over an unsecured network. In contrast, SSL is primarily used for secure connection between a web browser and a web server.  

Here is a summary of the major differences between SSH and SSL.  

Potential Risks of SSH Key Management 

While SSH is a powerful and secure protocol, when not properly managed, access to SSH keys can lead to severe security vulnerabilities. Some of the risks are: 

Private Key Theft

Private keys are stored locally by the user on local devices and should never be shared. Anyhow, if an attacker gains access to the private key, they can replicate themselves as the key owner and access the system.

Key Sprawl

As an organization runs for years and years, more and more keys are created. With this large number of keys, it is difficult to track and manage the keys in circulation. Due to these circumstances, attackers are likely to compromise the keys that have not been in the accounts for a long time.  

Lack of Expiration date

SSH keys do not have an expiration date like SSL/TLS Certificates do. The longer a key pair is in existence, the easier it is for the attacker to compromise it. Older keys are less likely to be rotated, too, as system administrators may not know the purpose behind a key. Deletion of older keys also rarely occurs, as system administrators fear they will block important access if they delete the key pair.

SSH-Based attacks

Using weak encryption algorithms or not protecting the private keys with strong passphrases increases the risk of SSH-based cyber-attacks. It is always recommended to use the best encryption algorithm and a strong passphrase to secure your keys.

Weak or No Passphrase on SSH Keys

Passphrase works as an additional security layer for private keys. If an existing key is compromised and it doesn’t have a passphrase, attackers can immediately use it to access the system. Not using a passphrase or using a weak one can make it easier for attackers to utilize stolen private keys. 

Poor Key Generation

Using outdated algorithms or insufficient key lengths can lower the effectiveness of SSH keys. These keys can be exploited by an attacker to impersonate users more easily. An outdated algorithm may be requested for an SSH key to generate it (e.g., DSA (1024-bit), while we should be using modern algorithms such as ED25519 or RSA with adequate length).

Lack of Auditing and Monitoring

Regular auditing and monitoring of SSH Keys is required to detect various risks when using them. Some of the risks associated with it are:

• Undetected Key Misuse: Compromised keys might be used by attackers if not monitored at the right time.
• Orphaned or Stale Keys: Keys associated with former employees or decommissioned systems may remain active, creating unnecessary attack vectors.
• No Accountability: Without monitoring, it is impossible to determine who is using which key and for what purpose.

SSH Agent Forwarding 

Before examining the risks of SSH Agent Forwarding, let’s first understand the concept of SSH Agent and SSH Agent Forwarding. 

In a general scenario, whenever you are required to use your private key, you’ll have to enter your passphrase every time, which is a bit annoying task. To manage this, we generally implement an agent which keeps your decrypted key in memory. This results in unlocking the key only once, and it will persist until you restart, letting you log into your servers securely without a passphrase prompt. This is what SSH Agent is. 

SSH Agent Forwarding is a feature of SSH (Secure Shell) that allows users to authenticate to multiple servers in a chain using their local private key without copying the private key to each server in the chain. It works by forwarding the authentication requests from remote servers back to the local machine’s SSH agent, which holds the private key. 

Suppose you want your remote server to pull some code from GitHub. When GitHub asks the server, “Who are you?” the server typically uses its own id_rsa files to authenticate. However, with SSH agent forwarding enabled, the server forwards this authentication request to your local machine. Your local machine responds to GitHub’s query without exposing your private key. The response is then sent back to the server, which relays it to GitHub. From GitHub’s perspective, the authentication has been successfully completed, and it allows the connection, regardless of where the response originated. 

Risks Associated

• Agent Hijacking

  If the first remote server in the chain is compromised, an attacker on that server can hijack the forwarded agent to authenticate to other servers without the private key.

• Untrusted Server

  The server might not be a trusted server every time, which is a major issue using SSH Agent forwarding. Since agent forwarding passes the local SSH agent credentials (essentially your private key) to the remote server, an attacker who has control over the remote server can potentially exploit this feature.

Best Practices for Key Management

We need to make sure all the keys are managed properly with security vulnerability management. Here are some best practices:  

Discovery and Consolidation

The first step towards managing SSH keys is to know how many keys exist and where they are present within the organization. Every key should be tied to its user and the server it is connecting to. It also tracks what key has not been used and for how long. Decommissioning the keys of former employees should also be taken into consideration. Key sprawl and unauthorized solo keys are potential risks that can be mitigated with a consolidated key management process in a central repository. 

Best Practices for Key Discovery are: 

• Proper Key Scan

  The first step is scanning and discovering SSH keys wherever they might exist, like on client devices like laptops or desktops used by developers and administrators, on-premises servers hosting critical applications, cloud instances running virtual machines, and any VMs used for automated processes. Each environment is a potential repository for SSH keys, and failing to account for them can leave gaps in your security framework. 

• Proper Key Storage 

  Once these keys are discovered, it’s equally important to store and maintain them in a centralized and up-to-date inventory. Think of this inventory as your single source of truth for all SSH keys within the organization. With centralized control, administrators can monitor which keys belong to whom, the systems they access, and their usage patterns. It also makes revoking or rotating keys much simpler when users leave the organization or when keys are no longer needed. 

• Automation in Discovery 

  To make this process more efficient and less error-prone, automating key discovery is essential. Manual searches across vast environments are not only time-consuming but also susceptible to human error. Automation tools can scan across all devices, servers, and cloud environments systematically, ensuring no key goes unnoticed. By reducing manual overhead, organizations can focus on analyzing and securing their key management processes rather than spending excessive time locating keys.

Policy Creation and Enforcement 

Policies need to be created to protect the keys from misuse. These policies should include:  

• Who can create keys?
• How do you create and store keys?
• How long should keys exist before rotation? 
• When to delete a key?
• What is the maximum access limit a key should have?

Best practices for policy creation and Enforcement are: 

• Grouping similar keys

  A key management strategy to decouple, organize, control, and keep the keys secure is to group the keys by their functionality. For Example, keys utilized for administrative purposes, application integrations, or developer access can be grouped into different categories. This makes it easier for administrators to map policies to groups of keys; if multiple keys serve similar purposes, they will be ensured to share the same security measures and access controls. 

• RBAC Implementation 

  With the implementation of Role-Based Access Control (RBAC), key management is made stronger by assigning access rights based on roles that are specific to the organization. Rather than giving wide-ranging access, users get only what they need for their jobs. This reduces the chances of unauthorized access and ensures that SSH keys correspond to organizational roles, thus increasing security and accountability.

• Custom Policy Creation 

  Custom policies can also be established around key lifecycle management. For example, organizations can enforce policies that define how keys are generated, specifying key lengths and algorithms for optimal security. Regular key rotation policies ensure that old keys are replaced periodically, reducing the risk of compromise. Time-bound access policies can limit the validity of specific keys, automatically revoking them after a predetermined period. These measures collectively help maintain a secure and controlled SSH key environment while adapting to organizational needs and regulatory requirements. 

Generate Strong Keys

While generating SSH keys, we should choose a strong cryptographic algorithm with a suitable key length, using algorithms like RSA or ED25519, which is strongly recommended. RSA comes with a larger key length of 2048 or 4096 bits. On the other hand, ED25519 comes with a smaller key length of 256 bits. Both algorithms are highly recommended due to their resilience against brute-force attacks. Algorithms like DSA are not considered due to their vulnerable nature. 

Key Rotation 

Once a key has been used for a while, it should be rotated to make it less prone to cyber-attacks. The complete key pair must be rotated with a new key pair. Key rotation minimizes the opportunity for attackers to misuse compromised keys. Key rotation also ensures better access control by ensuring regular updates on privileges. 

Best Practices involved in Key Rotation are: 

• Rotation Regularity 

  It is a good practice to periodically change SSH keys to strengthen security and minimize the likelihood of unauthorized access. Operation teams should establish a periodic time span for key rotation (For example, every 30 or 45 days) to protect from having old or potentially vulnerable keys in deployment.

• Automation in Rotation

  Automating the execution of key rotation is strongly encouraged, as automation helps simplify the key rotation process and reduce the chances of error. On a large scale with many keys, it is prone to errors, especially when replacing them manually. An automation system allows new keys to be created, existing key repositories to be modified, and to be deployed onto the appropriate systems with little to no human interaction. They can also link up to centralized inventory systems to confirm that all important information is updated so that all accurate records are tracked. Automating key rotation addresses several operational pain points, such as reducing operational overhead, ensuring consistent security policy across all environments, and avoiding the human factors leading to lost security.

Continuous Monitoring and Auditing 

Finally, the whole process needs to be monitored continuously. Auditing and monitoring if the keys are rotated and deleted at the appropriate time need to be performed. Risks such as key sprawl and solo keys can be eliminated through continuous monitoring and auditing.  

Best practices that can be kept in consideration are: 

• Enable SSH Logging

  When detailed logging is enabled, administrators can track and monitor all SSH activity, giving them insight into who is accessing the system and from where. Monitoring these logs helps to quickly identify abnormal or unauthorized access attempts and respond quickly to possible security threats. 

• Real-Time Alerting

  We can use the logs from SSH and integrate them with Security Information and Event Management (SIEM) systems to have real-time alerts. SIEM is responsible for analyzing security data throughout the enterprise in real-time and correlating it to identify unusual patterns. If suspicious activity occurs, for example, many login attempts or login attempts outside of the normal hours of access, the SIEM system can raise alarms, allowing security teams to be notified without delay and to take action.

• Auditing SSH Access 

  Some SSH keys might become unused or obsolete over time, particularly when employees leave or roles are altered. It also aids in the periodic review and auditing of access, wherein only active and authorized keys are there to work. 

How can Encryption Consulting help? 

Encryption Consulting provides end-to-end solutions to strengthen your organization’s data security and encryption practices. Our Advisory Service helps identify potential weaknesses in cryptographic protocols, minimizing the risk of data breaches and unauthorized access. We also enhance identity protection by assessing encryption and authentication methods, preventing identity threats such as theft or misidentification, and ensuring that only authorized individuals can access sensitive information.    

We also offer services for building effective SSH key management systems, including monthly webinars on SSH and encryption key management, cloud key protection, and more. Our assessments and training for cloud key management on platforms like AWS, Azure, and Google Cloud ensure compliance with industry standards and robust data protection. To keep you informed, we publish weekly blogs with best practices and insights that helps in keeping your organization’s security needs up to date.

Conclusion

SSH, or Secure Shell, is a high-end cryptographic protocol designed to ensure secure and authenticated communication over unsecured networks. By utilizing public-key cryptography, encryption, and advanced authentication methods, SSH safeguards sensitive data and prevents unauthorized access to remote systems.  

They offer a more solid approach compared to password-based authentication yet can introduce risks such as tampering with the private key, key sprawl, and cyberattacks if not managed and protected effectively. These include implementing best practices such as key rotation, centralized storage, and continuous monitoring to ensure that SSH keys remain secure not just now but into the future.