Troubleshooting rsync SSH Authentication Issues

I’m sure you’ve bumped into situations where rsync just refuses to work over SSH even though your normal SSH connections work perfectly fine. This maybe happened right when you were in the middle of a critical backup or trying to sync important files between servers. If you’ve tried searching around the interwebs for solutions, you’d surely know there’s not many comprehensive guides available and most people would just casually tell you “check your permissions” or “use ssh-agent” without any real explanation.

Now, both might work, but I just got properly annoyed when dealing with a particularly stubborn case of rsync throwing that infamous “Permission denied (publickey)” error. The standard SSH connection worked flawlessly to the same server, but rsync just wouldn’t play nice. To make things worse, I was trying to sync a massive directory structure that I absolutely needed transferred yesterday, and every failed attempt wasted precious time.

This guide focuses specifically on troubleshooting those hair-pulling rsync SSH authentication issues on Ubuntu systems. Yeah, that means addressing all those weird edge cases where your keys seem to work fine for regular SSH but rsync decides to have a tantrum. I’ll walk you through the exact process to diagnose and fix these problems, especially for system administrators and developers who rely on rsync for efficient file synchronisation and can’t afford these random authentication failures.

Common SSH Authentication Problems with rsync

While direct SSH connections might work perfectly, rsync operations often encounter unique authentication challenges. The most common error message you’ll encounter is:

Permission denied (publickey)

This error typically occurs when:

1. The rsync command cannot locate or access your SSH keys
2. SSH agent configuration isn’t properly set up for rsync operations
3. Key permissions or formats are incompatible with the requirements
4. Remote server configurations restrict rsync specifically

These problems can be particularly frustrating because they might occur even when standard SSH connections to the same server work flawlessly.

Explicitly Specifying Your SSH Key for rsync

The most direct solution is to explicitly tell rsync which SSH key to use with the -e flag:

rsync -avzh --progress --ignore-existing -e "ssh -i /path/to/your/private_key" \
username@remote-server:/path/to/source/directory/ \
/path/to/destination/directory/

This approach bypasses any confusion about which key should be used by directly specifying the path to your private key file.

Ensuring ssh-agent Is Properly Configured

If you prefer using ssh-agent for key management, ensure it’s running correctly and has your key loaded:

eval $(ssh-agent)
ssh-add /path/to/your/private_key

You can verify the key has been added successfully with:

ssh-add -l

After confirming your key is loaded in the agent, try your rsync command again without explicitly specifying the key path.

Using the SSH Connection String Format

Sometimes, simplifying your rsync command by using the SSH connection string format can resolve authentication issues:

rsync -avzh --progress --ignore-existing \
"username@remote-server:/path/to/source/directory/" \
/path/to/destination/directory/

This format allows the SSH client to use its default authentication methods, which might solve issues when complex command structures interfere with authentication.

Checking for Permission Restrictions

The remote server might allow standard SSH connections but restrict rsync operations specifically. Verify these potential restrictions:

1. Directory permissions: Ensure the remote user has appropriate read/write permissions for the directories being accessed
2. SSH configuration: Check both ~/.ssh/config on your local machine and /etc/ssh/sshd_config on the remote server for any restrictions that might impact rsync
3. Available commands: Confirm that rsync is installed and accessible to the remote user
4. Command restrictions: Some SSH configurations use ForceCommand or similar restrictions in the authorized_keys file that might prevent rsync

Debugging with Verbose Output

When persistent authentication issues occur, enabling verbose output can provide valuable diagnostic information:

rsync -avvzh --progress --ignore-existing -e "ssh -v" \
username@remote-server:/path/to/source/directory/ \
/path/to/destination/directory/

The -v flag for SSH produces detailed output about the authentication process, helping identify exactly where the failure occurs. Adding multiple ‘v’s to rsync (e.g., -avvvzh) will increase rsync’s own verbosity levels.

SSH Key Conversion Issues

If you’re using keys that were originally in PuTTY’s .ppk format and converted them for use on Ubuntu, ensure they were properly converted. Improperly converted keys can work for basic SSH but fail with rsync.

For a complete guide on converting PuTTY keys for Ubuntu, refer to our detailed article on Converting PuTTY SSH Keys for Ubuntu.

Creating a Dedicated SSH Configuration

For persistent rsync connections, creating a dedicated entry in your SSH config file can resolve many authentication issues:

# Edit your SSH config file
nano ~/.ssh/config

# Add a dedicated host entry
Host backup-server
    HostName remote-server.example.com
    User username
    IdentityFile /path/to/your/private_key
    IdentitiesOnly yes

Then use this simplified rsync command:

rsync -avzh --progress --ignore-existing \
backup-server:/path/to/source/directory/ \
/path/to/destination/directory/

Conclusion

Troubleshooting rsync SSH authentication issues on Ubuntu requires systematic investigation of key access, configuration settings, and permissions. By explicitly specifying SSH keys, properly configuring ssh-agent, simplifying commands, checking permissions, and using verbose output for debugging, you can identify and resolve most authentication problems. For persistent issues, creating dedicated SSH configurations often provides a reliable long-term solution. With these techniques, you can ensure smooth rsync operations for your file synchronisation and backup needs on Ubuntu systems.

For more information on SSH key management, including security best practices, refer to our guide on SSH Key Management Best Practices.