SSH Key Management Best Practices

April 25, 2025

Proper SSH key management is crucial for maintaining secure access to your systems while avoiding common pitfalls that could compromise your security or disrupt your workflow. This guide covers essential SSH key management best practices, including key generation, protection, rotation, and backup strategies. By implementing these recommendations, you’ll establish a robust security posture that balances convenience with strong protection of your sensitive systems and data.

Choosing the Right Key Type and Size

The foundation of effective SSH key management starts with generating the right type of key with appropriate parameters:

Key Types

Linux supports several SSH key types, each with different security characteristics:

RSA: The traditional option, still widely compatible but requires larger key sizes (minimum 3072 bits recommended)
Ed25519: Modern, offers excellent security with smaller key sizes and better performance
ECDSA: Provides a good balance between security and compatibility

Generate Strong Keys

For optimal security on Ubuntu systems, use these commands to generate keys:

# Ed25519 (recommended for most users)
ssh-keygen -t ed25519 -C "[email protected]"

# RSA (if compatibility with older systems is needed)
ssh-keygen -t rsa -b 4096 -C "[email protected]"

The -C flag adds a comment, typically your email address, which helps identify the key’s purpose later.

Protecting Your Private Keys

Private keys are the most sensitive component of your SSH authentication system:

Use Strong Passphrases

Always protect your private keys with strong passphrases:

# Generate a new key with passphrase protection
ssh-keygen -t ed25519 -C "[email protected]"

# Add a passphrase to an existing key
ssh-keygen -p -f ~/.ssh/id_ed25519

A strong passphrase should:

Be at least 12 characters long
Include a mix of uppercase, lowercase, numbers, and special characters
Not be used for any other accounts or systems

Set Proper File Permissions

SSH is sensitive to file permissions for security reasons. Set appropriate permissions:

# Set secure permissions on private key
chmod 600 ~/.ssh/id_ed25519

# Set permissions on public key
chmod 644 ~/.ssh/id_ed25519.pub

# Set permissions on .ssh directory
chmod 700 ~/.ssh

Incorrect permissions can lead to SSH refusing to use your keys.

Using SSH-Agent Effectively

SSH-Agent provides convenience without compromising security:

Start and Configure SSH-Agent

# Start the agent
eval $(ssh-agent -s)

# Add your key with a timeout (e.g., 4 hours)
ssh-add -t 4h ~/.ssh/id_ed25519

Adding a timeout ensures your keys aren’t held in memory indefinitely, reducing the risk if your session is compromised.

Configure SSH-Agent in Your Shell Profile

For persistent configuration, add these lines to your ~/.bashrc or ~/.zshrc:

# Start SSH agent if not already running
if [ -z "$SSH_AUTH_SOCK" ]; then
    eval $(ssh-agent -s)
    ssh-add ~/.ssh/id_ed25519 </dev/null
fi

Key Rotation Strategies

Regular key rotation is an essential security practice:

Implementing a Rotation Schedule

Personal keys: Rotate every 6-12 months
Automated system keys: Rotate every 3-6 months
High-security environments: Consider more frequent rotation

Process for Safe Key Rotation

Generate a new key pair:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new -C "[email protected]"

Add the new public key to servers while keeping the old one:

ssh-copy-id -i ~/.ssh/id_ed25519_new.pub username@remote-server

Test connectivity with the new key:

ssh -i ~/.ssh/id_ed25519_new username@remote-server

Once verified, replace the old key with the new one:

mv ~/.ssh/id_ed25519_new ~/.ssh/id_ed25519
mv ~/.ssh/id_ed25519_new.pub ~/.ssh/id_ed25519.pub

Remove the old public key from authorized_keys on remote servers.

Secure Backup Strategies

Losing access to your SSH keys can be catastrophic, making backups essential:

Encrypted Backups

Create encrypted backups of your SSH keys:

# Create an encrypted archive of your .ssh directory
tar -czf - ~/.ssh | gpg -c > ssh_keys_backup.tar.gz.gpg

Store this encrypted backup in a secure location, such as:

An encrypted USB drive kept in a safe location
A trusted password manager with file attachment capabilities
A secure cloud storage service with client-side encryption

Recovery Documentation

Create documentation for your key recovery process that includes:

Locations of backups
Encryption passwords (stored separately)
List of systems where keys are deployed
Steps to follow for key restoration

Managing Multiple Keys

As your infrastructure grows, you’ll likely need multiple keys for different purposes:

Organising Keys by Function

Create dedicated keys for different contexts:

Personal development work
Production system access
Automated backup processes
Different clients or projects

Name keys descriptively:

~/.ssh/id_ed25519_personal
~/.ssh/id_ed25519_production
~/.ssh/id_rsa_legacy_systems

Using SSH Config for Key Management

Configure ~/.ssh/config to specify which key to use for each host:

# Personal development server
Host dev-server
    HostName dev.example.com
    User developer
    IdentityFile ~/.ssh/id_ed25519_personal
    IdentitiesOnly yes

# Production servers
Host prod-*
    User admin
    IdentityFile ~/.ssh/id_ed25519_production
    IdentitiesOnly yes

# Legacy system with older SSH implementation
Host legacy
    HostName legacy.example.com
    User admin
    IdentityFile ~/.ssh/id_rsa_legacy
    IdentitiesOnly yes

The IdentitiesOnly yes setting ensures only the specified key is offered during authentication attempts.

Security Considerations for Converted Keys

If you’re using PuTTY keys converted for Ubuntu, additional considerations apply:

Validating Converted Keys

After converting PuTTY keys:

Verify file permissions are set correctly
Test with a simple SSH connection before using in automated processes
Consider generating fresh keys natively on Ubuntu for maximum security

For more information on converting PuTTY keys to use on Ubuntu, see our detailed guide on Converting PuTTY SSH Keys for Ubuntu.

Conclusion

Implementing these SSH key management best practices for Ubuntu systems creates a robust security foundation for your infrastructure. By generating strong keys, protecting them with passphrases and proper permissions, using SSH-agent effectively, implementing key rotation, maintaining secure backups, and properly managing multiple keys, you’ll significantly reduce the risk of unauthorized access while maintaining convenient authentication workflows. Remember that SSH key management is an ongoing process that requires regular attention to ensure continued security of your systems.

For specific issues related to SSH authentication with tools like rsync, refer to our troubleshooting guide on Troubleshooting rsync SSH Authentication Issues.