0d29ff67-9bc6-4d66-a4cb-b34692ba9f46 Skip to content
evan.guru

SSH Security Upgrade: Migrating to Key-Based Auth with Hardware Tokens (2025)

Password-based SSH logins are vulnerable to brute-force attacks. Migrating to SSH key-based authentication is a fundamental step in secure ssh connection best practices. This guide covers the ssh key authentication setup linux process, how to disable ssh password authentication, and takes it a step further by explaining how to leverage hardware security tokens like YubiKeys for even greater security.

Why SSH Keys are More Secure than Passwords

  • Complexity: SSH keys are significantly longer and more complex than human-memorable passwords, making them practically impossible to brute-force.
  • No Password Transmission: The private key never leaves your client machine during authentication. The server only needs the public key.
  • Automation: Keys facilitate secure automated logins for scripts and services.

Step 1: Generating Your SSH Key Pair

On your client machine (the one you connect from), open a terminal.

Terminal window
# Recommended: Use ED25519 algorithm (modern, secure, efficient)
ssh-keygen -t ed25519 -C "your_email@example.com"


# Or, use RSA (ensure sufficient key length)
# ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
  • -t ed25519: Specifies the key type. ED25519 is generally preferred over RSA today.
  • -b 4096: (If using RSA) Specifies a strong key length.
  • -C "your_email@example.com": A comment to help identify the key (optional but good practice).

Key Prompts:

  1. File location: Press Enter to accept the default (~/.ssh/id_ed25519 or ~/.ssh/id_rsa).
  2. Passphrase: Highly Recommended! Enter a strong passphrase. This encrypts your private key on your disk. If someone steals your private key file, they still need the passphrase to use it. You’ll need to enter this passphrase once per session when using the key (or use an ssh-agent).

This creates two files:

  • ~/.ssh/id_ed25519 (or id_rsa): Your private key. Keep this secret and secure!
  • ~/.ssh/id_ed25519.pub (or id_rsa.pub): Your public key. This is what you copy to servers.

Step 2: Copying Your Public Key to the Server

You need to add the content of your public key file (id_ed25519.pub) to the ~/.ssh/authorized_keys file on the server you want to connect to, under the user account you want to log in as.

Method 1: Using ssh-copy-id (Easiest)

If you still have password access to the server, this command automates the process:

Terminal window
ssh-copy-id user@your_server_ip_or_hostname

It will prompt for the user’s password on the server one last time to copy the key.

Method 2: Manual Copy

  1. Display your public key on your client machine:
    Terminal window
    cat ~/.ssh/id_ed25519.pub
  2. Copy the entire output (it’s one long line starting with ssh-ed25519 ...).
  3. SSH into your server using your password:
    Terminal window
    ssh user@your_server_ip_or_hostname
  4. On the server, create the .ssh directory if it doesn’t exist and set permissions:
    Terminal window
    mkdir -p ~/.ssh
    chmod 700 ~/.ssh
  5. Append your public key to the authorized_keys file (create it if needed) and set permissions:
    Terminal window
    echo "<paste-your-public-key-here>" >> ~/.ssh/authorized_keys
    chmod 600 ~/.ssh/authorized_keys
  6. Exit the server session (exit).

Step 3: Testing Key-Based Login

Try logging into the server again:

Terminal window
ssh user@your_server_ip_or_hostname
  • If you set a passphrase for your key, you’ll be prompted to enter it.
  • If successful, you should log in without being asked for the server user’s password.

Step 4: Disabling Password Authentication on the Server (Crucial!)

Once you’ve confirmed key-based login works reliably, you must disable password authentication on the server for improved security.

  1. SSH into your server (using your key).
  2. Edit the SSH daemon configuration file (usually /etc/ssh/sshd_config) with root privileges (e.g., sudo nano /etc/ssh/sshd_config).
  3. Find the following lines (or add them if they don’t exist) and ensure they are set as shown: 
    PasswordAuthentication no
    PubkeyAuthentication yes
    ChallengeResponseAuthentication no # Often needed to fully disable passwords
    UsePAM no # Optional, but can prevent PAM modules from allowing passwords
  4. Save the file and exit the editor.
  5. Validate the configuration:
    Terminal window
    sudo sshd -t
  6. If the syntax is OK, restart the SSH service:
    Terminal window
    sudo systemctl restart sshd # For systemd systems (most modern Linux)
    # or
    # sudo service ssh restart # For older init systems

Important: Before logging out, open a new terminal window and try SSHing into the server again with your key to ensure you didn’t lock yourself out! If it works, password authentication is now disabled.

Storing your SSH private key directly on your computer means it could be compromised if your machine is infected with malware. Hardware security keys add a physical layer of security. The private key is generated on and never leaves the hardware token.

Methods for using YubiKey (or similar FIDO2/PIV tokens) for SSH:

  1. FIDO2/U2F Authentication (Recommended, Modern):

    • Key Generation: Generate a special type of SSH key directly tied to your hardware key.

      Terminal window
      # Generate an ed25519-sk (FIDO/U2F) key
      ssh-keygen -t ed25519-sk -C "your_email@example.com_yubikey"
      

      # Or ecdsa-sk
      # ssh-keygen -t ecdsa-sk -C "your_email@example.com_yubikey"

      You will be prompted to touch your hardware key during generation.

    • How it works: Creates id_ed25519_sk and id_ed25519_sk.pub (or ecdsa_sk). The private key file (id_ed25519_sk) doesn’t contain the actual private key, but rather information needed to interact with the hardware token.

    • Server Support: Requires OpenSSH 8.2+ on both client and server.

    • Usage: Copy the .pub key to the server’s authorized_keys file as before. When you SSH, you’ll be prompted to touch your hardware key to authorize the login.

    • This is the FIDO2 ssh authentication guide approach.

  2. PIV (Personal Identity Verification) Mode:

    • Key Generation: Use the YubiKey Manager tool or ykman CLI to generate a key pair within one of the PIV slots (e.g., Authentication, Signing).
    • How it works: The private key resides securely within the YubiKey’s PIV applet. You use ssh-agent with a library that can talk to the PIV applet (like opensc-pkcs11.so) to make the key available for SSH authentication.
    • Setup: More complex, involves configuring ssh-agent and potentially OpenSC.
    • Usage: Add the public key (retrieved using ssh-add -L after configuring the agent) to the server’s authorized_keys. Requires the YubiKey to be plugged in and potentially the PIV PIN entered into the agent.

  3. GPG Agent: Use GnuPG and gpg-agent as a substitute for ssh-agent. You can generate keys on the YubiKey via GPG and configure gpg-agent to handle SSH authentication.

Using YubiKey for ssh authentication via FIDO2 (ed25519-sk or ecdsa-sk) is generally the simplest and most modern approach if your OpenSSH versions support it.

YubiKey 5 Series

YubiKey 5 Series

amazon.com

Hardware security keys from Yubico supporting multiple authentication protocols including FIDO2, U2F, and PIV

Managing Multiple Keys and SSH Agent

  • Multiple Keys: You can have multiple key pairs (e.g., one per client device, one per hardware token). Simply generate them with different filenames (ssh-keygen -f ~/.ssh/my_other_key ...).

  • ssh-agent: This helper program keeps your decrypted private keys in memory, so you only need to enter your passphrase once per session.

    • Start the agent: eval $(ssh-agent -s)
    • Add your key: ssh-add ~/.ssh/id_ed25519 (will prompt for passphrase)
    • List added keys: ssh-add -l
    • Add hardware key (PIV example): ssh-add -s /path/to/opensc-pkcs11.so

  • SSH Config File (~/.ssh/config): Simplify connections by defining hosts and specifying which key to use for each:

    Host my-server
        HostName your_server_ip_or_hostname
        User your_user
        IdentityFile ~/.ssh/id_ed25519
    

    Host work-server
        HostName work.example.com
        User work_user
        IdentityFile ~/.ssh/work_key_ed25519_sk
        IdentitiesOnly yes # Prevent trying default keys

    Now you can just type ssh my-server.

Conclusion

Migrating from passwords to SSH keys is a vital security enhancement. Generating a strong key pair (preferably ED25519), copying the public key correctly, and crucially, disabling ssh password authentication on your server significantly hardens your defenses. For the highest level of security, using YubiKey for ssh authentication (or other hardware tokens) via FIDO2 or PIV ensures your private keys cannot be easily stolen from your client machine. Properly managing multiple ssh keys securely using ssh-agent and the SSH config file makes the process convenient without sacrificing security.