SSH Login for Beginners: Easy Setup & Troubleshooting Guide

Aliakbar Hosseinzadeh

6 min read

·

Nov 19, 2025

--

Listen

Share

SSH is the standard way to log in to a Linux server from another Linux machine.
In this article you’ll learn:

• How to connect from Linux → Linux using SSH
• How to set up key-based login (no password)
• Exactly what ssh-copy-id user@server-ip does
• A practical SSH troubleshooting cheat sheet (auth errors, permissions, firewall, etc.)

No fluff. Just what you actually need.

Press enter or click to view image in full size

1. Mental model: what SSH is

SSH (Secure Shell) lets you open a secure terminal on another machine:

Your laptop  --->  encrypted SSH  --->  Remote Linux server

• Default port: 22
• Program on your machine: ssh
• Program on the server: sshd (SSH daemon)

You type commands on your computer; they run on the server.

2. One-time setup on the server (Ubuntu/Debian)

On the server (the machine you want to connect to):

# Install server
sudo apt update
sudo apt install -y openssh-server

# Make sure SSH is running and enabled
sudo systemctl enable --now ssh
sudo systemctl status ssh

Find the server’s IP address:

ip -4 addr | grep inet

Look for something like:

• 192.168.x.x
• 10.x.x.x
• 172.16.x.x – 172.31.x.x

If you use UFW firewall, allow SSH:

sudo ufw allow ssh     # opens port 22
sudo ufw enable        # if not already enabled
sudo ufw status

Quick troubleshooting (server setup)

• ssh: connect to host ... port 22: Connection refused
  → sshd not running or listening.
  Check: sudo systemctl status ssh
• ssh: connect to host ... port 22: No route to host or timeout
  → network / firewall issue.
  - Check you can ping: ping server-ip
- Check firewall: sudo ufw status
- If cloud VM: also check cloud security group rules.

3. First connection from Linux to Linux (with password)

On your client machine (your laptop / another Linux box):

ssh username@server-ip

Example:

ssh ali@192.168.1.105

First time, you’ll see a fingerprint question:

Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type:

yes

Then enter the password of the user on the server (e.g. the password for ali on that server).
If it works → you now have a shell on the remote machine.

Troubleshooting (password connection)

• Permission denied (publickey,password)
  → Common causes:
  - Wrong username (e.g. root vs ali)
  - Wrong password
  - SSH configured to disallow password auth

Try:

ssh -v username@server-ip

• The -v (verbose) output will tell you why it failed (no matching auth methods, wrong password, etc.).
• username@server-ip: Name or service not known
  → Typo in hostname or DNS issue. Try using the raw IP address.

4. Key-based login: ssh-keygen + ssh-copy-id

Password login works, but keys are more secure and more convenient.

4.1 Generate a key pair on the client

On your Linux client (not on the server):

ssh-keygen -t ed25519 -C "my-laptop"

• Press Enter to accept the default path (~/.ssh/id_ed25519)
• Set a passphrase (recommended)

This creates:

• ~/.ssh/id_ed25519 → private key (keep secret)
• ~/.ssh/id_ed25519.pub → public key (you copy this to servers)

Check:

ls -l ~/.ssh

You should see the two files.

4.2 Copy key to server (what ssh-copy-id user@server-ip actually does)

Now run on your client:

ssh-copy-id username@server-ip

Example:

ssh-copy-id ali@192.168.1.105

You’ll be asked for the server password ONE LAST TIME.

What this command does internally:

1. Connects to username@server-ip using SSH (with your password).
2. Creates the directory ~/.ssh on the server (if missing).
3. Appends your public key (id_ed25519.pub) to:

~/.ssh/authorized_keys    # on the server

4. Fixes permissions so SSH accepts it.

So after ssh-copy-id:

• Your public key is now trusted by the server.
• Your private key stays on your client.

You’re ready to log in without typing the account password.

Troubleshooting ssh-copy-id

• ssh-copy-id: command not found
  → Install OpenSSH client tools:

sudo apt install openssh-client

Permission denied (publickey,password) during ssh-copy-id
→ Same as normal SSH:

• Wrong user / password
• Server disabled password login already
• Firewall / network issues

Fix SSH/password login first, then run ssh-copy-id.

• Still asks for a password after ssh-copy-id?
  Check on the server:

ls -ld ~/.ssh
ls -l ~/.ssh/authorized_keys

Permissions should be:

~/.ssh              -> 700  (drwx------)
~/.ssh/authorized_keys -> 600  (-rw-------)

If not, fix them:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Also confirm your public key is there:

cat ~/.ssh/authorized_keys

You should see a long line starting with ssh-ed25519 or ssh-rsa.

4.3 Connect using the key (no account password)

Now, simply:

ssh username@server-ip

SSH will:

1. Read your private key (e.g. ~/.ssh/id_ed25519)
2. Prove to the server that you own the matching key
3. Log you in without asking for the account password
   (it may ask for your key passphrase, which is good)

For debug:

ssh -v username@server-ip

Look for Offering public key and Authentication succeeded (publickey).

5. Optional: secure your SSH server

Once key-based login works reliably, you can harden SSH:

On the server:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
sudo nano /etc/ssh/sshd_config

Make sure you have:

Protocol 2
PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication no

Then restart:

sudo systemctl restart ssh

Important safety tip:

• Keep one SSH session open while you restart SSH.
• Open a second terminal and test a new login.
• Only close the original session when you know the new setup works.

6. SSH Troubleshooting Cheat Sheet (auth, permissions, firewall)

Here’s a compact “fix it fast” section you can refer to anytime.

A. Cannot connect at all

Error:

ssh: connect to host X.X.X.X port 22: Connection refused

Check:

sudo systemctl status ssh
sudo ss -tlnp | grep ':22'

Fix:

sudo systemctl enable --now ssh

Error:

ssh: connect to host X.X.X.X port 22: No route to host
# or timeout

Check:

ping server-ip
sudo ufw status

• If ping fails → network problem / wrong IP.
• If UFW is active and no rule for 22 → add:

sudo ufw allow ssh

If it’s a cloud server, also check your cloud firewall / security group.

B. “Permission denied” / auth errors

Error:

Permission denied (publickey,password)

1. Check user + host

ssh -v username@server-ip

• Did you type correct username? (e.g. ubuntu, ec2-user, ali?)
• Are you connecting to the right server?

2. Check server-side authorized_keys

On the server (using some working access, even console):

ls -ld ~/.ssh
ls -l ~/.ssh/authorized_keys

Permissions must be:

.ssh            -> 700
authorized_keys -> 600

If wrong:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

3. Check the key entry itself

cat ~/.ssh/authorized_keys

• Each key must be on one line (no line breaks in the middle).
• It should start with ssh-ed25519 or ssh-rsa.

4. Check client is using the correct key

ssh -v username@server-ip

Look for:

Offering public key: /home/you/.ssh/id_ed25519
Authentication succeeded (publickey)

If you see “no mutual signature supported” or “no supported methods”, your key type might be unsupported or disabled.

C. “REMOTE HOST IDENTIFICATION HAS CHANGED!”

Error:

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

SSH stores server fingerprints in ~/.ssh/known_hosts.

This warning appears if:

• The server was rebuilt
• The IP now points to a different machine
• Or a real man-in-the-middle (rare, but possible)

If you know the server legitimately changed:

nano ~/.ssh/known_hosts
# or
ssh-keygen -R server-ip

Then connect again and accept the new fingerprint.

If you’re not sure → double-check with the admin or cloud panel before continuing.

D. Problems with ssh-copy-id

Error:

ssh-copy-id: command not found

Install OpenSSH client:

sudo apt install openssh-client

Error: still asks for password after ssh-copy-id

• Fix permissions on the server:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

• Confirm your key is in that file:

grep "ssh-ed25519" ~/.ssh/authorized_keys

• Make sure you’re connecting with the same user and same key you just copied.

E. Where to look in logs

On the server:

sudo journalctl -u ssh --since "1 hour ago"
# or
sudo less /var/log/auth.log

Search for:

sudo grep 'Failed password' /var/log/auth.log
sudo grep 'Accepted publickey' /var/log/auth.log

This tells you:

• Which usernames attackers try
• Whether your key login is working (Accepted publickey)

7. Final recap

If you only remember this flow, you’re good:

1. On the server: install and start OpenSSH server, open port 22.
2. From Linux client:
   ssh username@server-ip → test password login.
3. Generate key:
   ssh-keygen -t ed25519
4. Copy key to server:
   ssh-copy-id username@server-ip
5. Log in with key:
   ssh username@server-ip
6. Harden later: disable password login, root SSH, add Fail2ban & firewall.

And when something breaks, use the SSH troubleshooting cheat sheet above to quickly see whether it’s:

• Service not running
• Port/firewall issue
• Wrong user / wrong key
• File permissions
• Or host key mismatch

Ref

How SSH (Secure Shell) Works | Medium

What is SSH? | Secure Shell (SSH) protocol | Cloudflare

What Is SSH? How Does SSH Work? — Huawei

Better is better than best.