RoamSwitch Icon RoamSwitch by Lafine Systems Design

Know the attack, stop the attack

SSH and remote access

From sources in a network namespace, cause a login after failed guesses, a planted key, an ssh -R tunnel and tunnelling tools, and weak sshd and sudo settings; learn to read the alerts and records and close the door.

  • Level: Basics
  • About 45 min
  • 2026-11-12

#How attackers use SSH

SSH is the door you use to manage a server. It is the same door for an attacker. What happens after they get in usually falls into four steps.

StepWhat the attacker doesWhat this lesson looks at
Get inLog in with a guessed passwordThe "login succeeded after a brute-force attempt" alert of SSH brute-force detection and blocking
StayAdd their own key to authorized_keysThe Critical system file tamper monitoring alert, and the session that wrote it
Go deeperOpen an SSH tunnel, or run a tunnelling toolthe SSH tunnel detection alert, roamswitch ports -a, and EXR-010 of Process execution recording and attack-technique detection
Widen the doorLeave root login or passwordless sudo in placeSSH and sudo settings check

Banning brute force and limiting SSH to listed sources are covered in Configuring SSH protection. This lesson picks up after that: when someone gets in without reaching the ban threshold, what can you see, and what can you not?

Practise in a disposable environment

This lesson deliberately weakens the sshd and sudo settings. Do not try it on a real machine. The VM here is driven with tart exec, so a wrong SSH setting does not lock you out.

#Prepare the practice setup

To play an attacker from outside, create a network namespace inside the VM. The steps are the same as in Configuring SSH protection. 10.99.0.2 and 10.99.0.3 are the outside sources, and 10.99.0.1 is the VM side.

Create a test user and an outside source

This adds a user and network settings (this changes the system). The password is a practice dummy.

sudo useradd -m -s /bin/bash labuser
echo 'labuser:Practice-Pass-123' | sudo chpasswd
sudo install -d -m 700 -o labuser -g labuser /home/labuser/.ssh
sudo apt install -y sshpass
sudo ip netns add outside
sudo ip link add veth-host type veth peer name veth-out
sudo ip link set veth-out netns outside
sudo ip addr add 10.99.0.1/24 dev veth-host
sudo ip link set veth-host up
sudo ip netns exec outside ip addr add 10.99.0.2/24 dev veth-out
sudo ip netns exec outside ip addr add 10.99.0.3/24 dev veth-out
sudo ip netns exec outside ip link set veth-out up
sudo roamswitch server restart
sudo roamswitch bans | sed -n 5,6p
Output excerpt
  ✓ roamswitch-server service successfully restarted.
  State     : detects only (does not ban)
  Reason    : fail2ban already guards SSH, so the ban is left to it (ssh_guard_yield_to_fail2ban=true).

fail2ban watches sshd on this VM, so the RoamSwitch guard only detects. This lesson leaves fail2ban running. The alerts you will see appear in detect-only mode too.

The steps to undo everything are at the end of the last exercise.

#A guessed password gets in

A ban happens when failures reach ssh_guard_max_failures (default 6). If the attacker guesses right in fewer tries, there is no ban. Instead, the guard reports "a success after failures".

In the code, a source that failed at least half the threshold (3 by default) within the window (ssh_guard_window_secs, default 600 seconds) is reported when it then logs in.

Get in after three wrong tries, and plant a key

From 10.99.0.2, try three wrong passwords. On the fourth try, log in with the right one and add the attacker's key to authorized_keys. The key is a practice key made on the spot.

F="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o PubkeyAuthentication=no -o NumberOfPasswordPrompts=1 -o LogLevel=ERROR -o ConnectTimeout=5"
ssh-keygen -q -t ed25519 -N "" -C lab-dummy-key -f /tmp/lab_key
for p in wrong-1 wrong-2 wrong-3; do sudo ip netns exec outside sshpass -p $p ssh $F -b 10.99.0.2 labuser@10.99.0.1 true; done
sudo ip netns exec outside sshpass -p Practice-Pass-123 ssh $F -b 10.99.0.2 labuser@10.99.0.1 "echo logged in from \$SSH_CLIENT; sleep 2; echo '$(cat /tmp/lab_key.pub)' >> ~/.ssh/authorized_keys; sleep 3; true"
Output
labuser@10.99.0.1: Permission denied (publickey,password).
labuser@10.99.0.1: Permission denied (publickey,password).
labuser@10.99.0.1: Permission denied (publickey,password).
logged in from 10.99.0.2 47977 22

Look at the notification history. The newest is on top.

sudo roamswitch notifications | head -34
Output excerpt
  2026-11-12T02:47:06.128900839+00:00
  Critical Path FIM Tampering Detected (1 violation)
    Host: ubuntu (IP: 192.168.64.13)
    Integrity violations in critical system files:
    /home/labuser/.ssh/authorized_keys: new file
      Process: labuser / /usr/bin/bash / command: bash -c echo logged in from $SSH_CLIENT; sleep 2; echo 'ssh-ed25519 SHA256:PbSI6GAtsZUh5XuvA2mrP7WsJSfM09GLlfRtQLm6KiA lab-dummy-key' >> ~/.ssh/authorized_keys;… (pid 1732)
      SSH login: the session labuser opened from 10.99.0.2:47977 at 2026-11-12 02:47:03 (authentication: password)
    Recommended Action: Run `sudo roamswitch fim verify` to inspect changes.
    Timestamp: 2026-11-12 02:47:06

  2026-11-12T02:47:03.762222996+00:00
  CRITICAL: SSH login succeeded after a brute-force attempt: 10.99.0.2
    Host: ubuntu (IP: 192.168.64.13)
    Source: 10.99.0.2
    Account logged into: labuser
    Method: password
    Failures before it: 3
    The password may have been guessed. If you do not recognise this login, act now.
    - Check the open sessions with who or last, and cut a suspicious one with loginctl terminate-session
    - Change that account's password and allow key authentication only (PasswordAuthentication no)
    - Look for changes you do not recognise in authorized_keys, cron and the user's systemd services
    The session was not cut.
    Timestamp: 2026-11-12 02:47:03

Two alerts came in. The lower one came first.

  • Login succeeded after a brute-force attempt: the success after three failures is reported as critical. Times are UTC; 02:47:03 is 11:47:03 in Japan
  • The FIM alert: right after the write, it reports that authorized_keys was added. It carries the command that wrote it, and the SSH login that process belongs to (source, port, time, method)
  • The session was not cut: the attacker's session goes on after the alert. A person decides whether to cut it
  • A source the guard never bans: the SSH guard does not ban the source of an open SSH session, so you are not locked out of your own work. The same holds when the session is the attacker's, so cutting it is up to a person

A public key in the command line is shown as its SHA256: fingerprint. A long command line is cut off with ….

When "login succeeded after a brute-force attempt" arrives

First, find out whether the login was you.

  • It was you (you just mistyped the password): switch to key logins. The last exercise of this lesson turns password logins off
  • You do not recognise it: someone knows the password. Cut the session and change the password. Also look for FIM alerts around the same time. As in the example above, a key may have been planted right after the login

If you cannot tell, treat it as one you do not recognise.

#Find a planted key

Changing the password does not remove a planted key. The attacker can come back with the key. A key login needs no failures, so the "login succeeded after a brute-force attempt" alert does not appear either.

Log in with the planted key, and match the alert to the records

The attacker logs in with the planted key from another source, 10.99.0.3. Once in, they change authorized_keys again.

K="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o ConnectTimeout=5 -o BatchMode=yes -i /tmp/lab_key"
sudo ip netns exec outside ssh $K -b 10.99.0.3 labuser@10.99.0.1 'echo key login from $SSH_CLIENT; sleep 2; echo "# lab note" >> ~/.ssh/authorized_keys; sleep 3; true'
sudo roamswitch notifications | head -14
Output excerpt
key login from 10.99.0.3 33285 22

  2026-11-12T02:47:15.363484648+00:00
  Critical Path FIM Tampering Detected (1 violation)
    Host: ubuntu (IP: 192.168.64.13)
    Integrity violations in critical system files:
    /home/labuser/.ssh/authorized_keys: new file
      Process: labuser / /usr/bin/bash / command: bash -c echo key login from $SSH_CLIENT; sleep 2; echo "# lab note" >> ~/.ssh/authorized_keys; sleep 3; true (pid 1874)
      SSH login: the session labuser opened from 10.99.0.3:33285 at 2026-11-12 02:47:13 (authentication: publickey, key SHA256:PbSI6GAtsZUh5XuvA2mrP7WsJSfM09GLlfRtQLm6KiA)
    Recommended Action: Run `sudo roamswitch fim verify` to inspect changes.
    Timestamp: 2026-11-12 02:47:15

This time the method is publickey, with the key's fingerprint. It still says "new file" because the baseline has no such file. The check compares against the baseline, so the second change also shows as a new file.

Check which key the fingerprint belongs to. The sshd log has the same fingerprint.

ssh-keygen -lf /tmp/lab_key.pub
sudo journalctl -u ssh --since "-3min" --no-pager | grep Accepted
Output
256 SHA256:PbSI6GAtsZUh5XuvA2mrP7WsJSfM09GLlfRtQLm6KiA lab-dummy-key (ED25519)
Nov 12 02:47:03 ubuntu sshd[1598]: Accepted password for labuser from 10.99.0.2 port 47977 ssh2
Nov 12 02:47:13 ubuntu sshd[1826]: Accepted publickey for labuser from 10.99.0.3 port 33285 ssh2: ED25519 SHA256:PbSI6GAtsZUh5XuvA2mrP7WsJSfM09GLlfRtQLm6KiA

In a real investigation, run ssh-keygen -lf ~labuser/.ssh/authorized_keys instead of using /tmp/lab_key.pub. It lists the fingerprint of every key in the file. The line that matches the log is the key that was used.

detection-coverage shows what is watched.

sudo roamswitch detection-coverage --server | grep -A2 T1098.004
Output
  🟡 T1098.004 Account Manipulation: SSH Authorized Keys  [partial]  on
      covered by: criticalPathFim
      blind spot: Only authorized_keys and authorized_keys2 in the .ssh folder of login accounts and /root/.ssh are watched; keys in a location set by sshd's AuthorizedKeysFile, or served by AuthorizedKeysCommand, are not.

Remove the key (this changes the system). Use fim verify to see which file changed, and check again after deleting it.

sudo roamswitch fim verify
sudo rm /home/labuser/.ssh/authorized_keys
sudo roamswitch fim verify
Output
  🔍 Verifying integrity of critical system binaries and configs... (93 files)
  🚨 File modification or tampering detected!
    [modified] /etc/passwd
    [modified] /etc/group
    [new file] /home/labuser/.ssh/authorized_keys
    [modified] /etc/gshadow
    [modified] /etc/shadow
  4 of these were left out of the baseline by the package update at 2026-11-12 02:46, which does not explain them.
  🔍 Checking the SUID / SGID file inventory...
  ✓ No SUID / SGID file was added or changed. (20 files checked)

  If these are legitimate changes (e.g. OS upgrade), run 'sudo roamswitch fim update' to baseline.
  🔍 Verifying integrity of critical system binaries and configs... (93 files)
  🚨 File modification or tampering detected!
    [modified] /etc/shadow
    [modified] /etc/gshadow
    [modified] /etc/group
    [modified] /etc/passwd
  4 of these were left out of the baseline by the package update at 2026-11-12 02:46, which does not explain them.
  🔍 Checking the SUID / SGID file inventory...
  ✓ No SUID / SGID file was added or changed. (20 files checked)

  If these are legitimate changes (e.g. OS upgrade), run 'sudo roamswitch fim update' to baseline.

After the deletion, the authorized_keys line is gone. Here the whole file was deleted. On a real machine where the file also holds legitimate keys, delete only the attacker's line.

The four files left are the changes from creating labuser in the first exercise. The package update that installed sshpass right after could not explain them either, so it did not take them into the baseline: a package update does not absorb changes unrelated to it. Confirm they are yours, then update the baseline.

sudo grep -c labuser /etc/passwd /etc/group /etc/shadow /etc/gshadow
sudo roamswitch fim update | tail -3
sudo roamswitch fim verify
Output
/etc/passwd:1
/etc/group:1
/etc/shadow:1
/etc/gshadow:1
  ✓ FIM baseline database successfully updated and saved. (92 entries recorded)
  ✓ SUID / SGID baseline updated. (20 entries recorded)

  🔍 Verifying integrity of critical system binaries and configs... (92 files)
  ✓ No tampering detected. All files verified healthy. (92 files monitored)
  🔍 Checking the SUID / SGID file inventory...
  ✓ No SUID / SGID file was added or changed. (20 files checked)
Running `fim update` before you investigate

sudo roamswitch fim update makes the current state the baseline as it is. Run it while the attacker's key is still there, and that key becomes the "correct state" and no longer shows in fim verify. Look at what changed first, keep only the changes you know are yours, and then run it.

#Tunnels, and tunnelling tools

Once in, an attacker uses the server as a stepping stone and opens a path into the network behind it. SSH alone can do this. ssh -R opens a listening port on the server side and carries connections to it back to the attacker.

Open an ssh -R tunnel, then find it and cut it

From 10.99.0.3, open a session that only carries a tunnel. -N means run no command. Connections to 127.0.0.1:9000 on the server are carried to 10.99.0.3:8080 on the attacker's side.

sudo ip netns exec outside setsid sshpass -p Practice-Pass-123 ssh $F -b 10.99.0.3 -N -R 127.0.0.1:9000:10.99.0.3:8080 labuser@10.99.0.1 </dev/null >/dev/null 2>&1 &
sleep 5
sudo roamswitch ports -a | grep -A1 "Port 9000"
Output
  • Port 9000  sshd            PID: 2076   127.0.0.1 (local only)[MED]
    ↳ SSH Version Banner Disclosed: Keep SSH updated to the latest version, and minimize the banner's detail if your SSH implementation allows it.

sshd is listening on port 9000. Without -a, a listener on 127.0.0.1 only is left out of the list. The note on the second line is there because the port belongs to sshd; it is not a tunnel verdict.

Look at the notification history too.

sleep 5
sudo roamswitch notifications | sed -n 5,20p
Output
  2026-11-12T02:47:32.328497253+00:00
  An SSH session opened a listening port: 127.0.0.1:9000 (labuser)
    Host: ubuntu (IP: 192.168.64.13)
    A session logged in over SSH (sshd PID 2076, account labuser) listens on 127.0.0.1:9000 of this machine. Only this machine can connect to it. With an `ssh -R` tunnel, every connection to this port is carried to the other end of the session.
    The session comes from 10.99.0.3:55015.
    If it is a tunnel of your own, allow it: sudo roamswitch server allowlist add ssh-remote-listen labuser:9000 --days 30 <reason>
    If you do not know it, end the session: sudo kill 2076
    Timestamp: 2026-11-12 02:47:32

  2026-11-12T02:47:29.389405831+00:00
  🔎 Suspicious execution pattern detected
    An ssh tunnel (-R / -D) was opened (EXR-016)
    Process: /usr/bin/ssh (PID: 2003) / triage: Medium
    Details: ssh opened a reverse tunnel (-R: a listener on the other side, carried to this machine) (127.0.0.1:9000:10.99.0.3:8080) as user root; command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o PubkeyAuthentication=no -o NumberOfPasswordPrompts=1 -o LogLevel=ERROR -o ConnectTimeout=5 -b 10.99.0.3 -N -R 127.0.0.1:9000:10.99.0.3:8080 labuser@10.99.0.1
    Next step: sudo roamswitch events tree 2003
    (notification only — nothing was blocked)

The upper alert is the one the server side sees. An SSH session (a child of sshd) opened a listener, so it is reported with the port, address, account and source.

The lower one, EXR-016, is about the ssh that opened the tunnel. It appears here because the attacker's ssh also runs on this VM, in the namespace. When the tunnel comes from the attacker's own machine, only the upper alert appears. Like the other alerts, it is shown in the reader's language (LANG=en_US.UTF-8 here).

Find out which session the tunnel belongs to.

who | wc -l
loginctl list-sessions
P=$(sudo ss -ltnpH 'sport = :9000' | sed -n 's/.*pid=\([0-9]*\).*/\1/p')
ps -o pid,user,args -p $P
loginctl show-session $(cat /proc/$P/sessionid) -p Name -p RemoteHost -p Service
Output
0
SESSION  UID USER    SEAT TTY STATE  IDLE SINCE
      4 1002 labuser -    -   active no   -

1 sessions listed.
    PID USER     COMMAND
   2076 labuser  sshd: labuser
Name=labuser
RemoteHost=10.99.0.3
Service=sshd

who printed 0 lines. A session without a terminal (TTY) does not show in who. It does show in loginctl, which gives the source as 10.99.0.3. The PID is the same as the "sshd PID" in the alert.

Cut the session (this changes the system).

sudo loginctl terminate-session $(cat /proc/$P/sessionid)
sleep 2
sudo ss -ltnH 'sport = :9000' | wc -l
Output
0

Cutting the session also removed the tunnel's listener.

For an -R tunnel of your own, allow it as the alert says: sudo roamswitch server allowlist add ssh-remote-listen labuser:9000 --days 30 <reason>. It matches the session's account and port. With that entry added, opening the same tunnel again raised no alert. Remove the entry with sudo roamswitch server allowlist remove ssh-remote-listen labuser:9000.

Instead of SSH, an attacker may bring a dedicated tunnelling tool. Process execution recording and attack-technique detection counts tools such as chisel and ligolo-ng by name. When a scripting or AI-agent runtime (python, node and the like) starts three different ones within 120 seconds, it reports EXR-010.

Start three tunnelling tools from a script

Create dummies that borrow only the names and do nothing. Each one just runs echo (this changes the system).

sudo mkdir -p /opt/labtools
for t in chisel ligolo-ng suo5; do printf '#!/bin/sh\necho "dummy %s"\n' $t | sudo tee /opt/labtools/$t >/dev/null; sudo chmod 755 /opt/labtools/$t; done
python3 -c '
import subprocess
for t in ["chisel", "ligolo-ng", "suo5"]:
    subprocess.run(["/opt/labtools/" + t])
'
sleep 8
sudo roamswitch notifications | grep -m1 -B2 -A7 EXR-010
Output
dummy chisel
dummy ligolo-ng
dummy suo5
  2026-11-12T02:47:56.335790684+00:00
  🔎 Suspicious execution pattern detected
    A script or agent launched several penetration-testing tools in quick succession (EXR-010)
    Process: /usr/bin/python3.12 (PID: 2359) / triage: Medium
    Details: python3.12 (pid 2359) started 3 distinct offensive-security tools within 120s: chisel, ligolo-ng, suo5; a scripting/agent runtime driving pentest tools
    Next step: sudo roamswitch events tree 2359
    (notification only — nothing was blocked)
    Process: admin / /usr/bin/dash / command: /bin/sh /opt/labtools/suo5 (pid 2362)
    Parent processes: python3.12 ← dash ← dash ← tart-guest-agent ← systemd

The triage level is Medium, and nothing is blocked. The last two lines are the process that ran the third tool, and its parent python3.12. Now run the same three, one after another, from the shell of a person logged in over SSH.

sudo ip netns exec outside sshpass -p Practice-Pass-123 ssh $F -b 10.99.0.2 labuser@10.99.0.1 '/opt/labtools/chisel; /opt/labtools/ligolo-ng; /opt/labtools/suo5'
sleep 8
sudo roamswitch notifications | grep -c EXR-010
Output
dummy chisel
dummy ligolo-ng
dummy suo5
1

EXR-010 stayed at one alert. People run such tools from a terminal for administration and testing all the time, so that case is outside this rule.

#Close the door

The attacks so far used password logins and an SSH tunnel. Both can be stopped in the sshd settings. SSH and sudo settings check reports the weak spots in those settings.

Weaken the settings on purpose, and read the audit

Allow direct root login, and give labuser passwordless sudo (this changes the system). The way back is at the end of this exercise.

printf 'PermitRootLogin yes\n' | sudo tee /etc/ssh/sshd_config.d/00-lab.conf
echo 'labuser ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/90-lab
sudo chmod 440 /etc/sudoers.d/90-lab
sudo roamswitch status --server | grep -A2 -E "SSH Root Login|Sudoers Privilege"
Output excerpt
  [WARN] SSH Root Login & Password Auth - Insecure (Root login or password auth enabled)
         Enforces key-based authentication and prohibits direct root login.
         ↳ Set 'PermitRootLogin no' and 'PasswordAuthentication no' in /etc/ssh/sshd_config.
--
  [WARN] Sudoers Privilege Hygiene - Passwordless sudo rule with full privileges (NOPASSWD: ALL)
         Audits sudoers rules for unauthorized NOPASSWD or ALL directives.
         ↳ Restrict NOPASSWD entries to dedicated service commands only.

The audit showed it before sshd was restarted. The SSH audit reads the configuration as sshd evaluates it (sshd -T). The setting takes effect only once sshd restarts.

Files in sshd_config.d are read in name order, and for a repeated setting the first value wins. The 00- prefix makes this file read before the others.

You can look for passwordless sudo yourself as well.

sudo grep -rn NOPASSWD /etc/sudoers /etc/sudoers.d/
Output
/etc/sudoers.d/90-cloud-init-users:4:admin ALL=(ALL) NOPASSWD:ALL
/etc/sudoers.d/90-cloud-init-users:7:ubuntu ALL=(ALL) NOPASSWD:ALL
/etc/sudoers.d/90-lab:1:labuser ALL=(ALL) NOPASSWD: ALL

Besides the 90-lab file you added, there are lines that cloud-init gave the first users (admin and ubuntu).

A file placed in /etc/sudoers.d or /etc/ssh/sshd_config.d raises a Critical system file tamper monitoring alert on the spot, with the process that wrote it.

sudo roamswitch notifications | sed -n 5,13p
Output
  2026-11-12T02:48:24.305952734+00:00
  Critical Path FIM Tampering Detected (1 violation)
    Host: ubuntu (IP: 192.168.64.13)
    Integrity violations in critical system files:
    /etc/sudoers.d/90-lab: new file
      Process: root / /usr/bin/tee / command: tee /etc/sudoers.d/90-lab (pid 2725) (The process had already exited, so this comes from the exec record made at 2026-11-12 02:48:24.)
      No SSH login was found (it is a process of the systemd service tart-guest-agent.service)
    Recommended Action: Run `sudo roamswitch fim verify` to inspect changes.
    Timestamp: 2026-11-12 02:48:24

This VM is driven with tart exec, so the writer shows as tart-guest-agent.service rather than an SSH login. 00-lab.conf raised the same kind of alert.

Now close the door. Before turning password logins off, register your own key. Then turn off root login, password login and TCP forwarding (tunnels).

sudo rm /etc/sudoers.d/90-lab
ssh-keygen -q -t ed25519 -N "" -C admin-key -f /tmp/admin_key
sudo install -m 600 -o labuser -g labuser /tmp/admin_key.pub /home/labuser/.ssh/authorized_keys
printf 'PermitRootLogin no\nPasswordAuthentication no\nKbdInteractiveAuthentication no\nAllowTcpForwarding no\n' | sudo tee /etc/ssh/sshd_config.d/00-lab.conf
sudo sshd -t && sudo systemctl restart ssh
sudo roamswitch status --server | grep "SSH Root Login"
Output excerpt
  [PASS] SSH Root Login & Password Auth - Secure (Root login prohibited, keys enforced)

sshd -t checks the configuration for errors. If there is one, sshd is not restarted.

Try the attacker's three moves again: a password, a key, and a key with a tunnel.

A="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o ConnectTimeout=5 -o BatchMode=yes -i /tmp/admin_key"
sudo ip netns exec outside sshpass -p Practice-Pass-123 ssh $F -b 10.99.0.2 labuser@10.99.0.1 true
sudo ip netns exec outside ssh $A -b 10.99.0.2 labuser@10.99.0.1 'echo key ok from $SSH_CLIENT'
sudo ip netns exec outside ssh $A -o ExitOnForwardFailure=yes -b 10.99.0.2 -N -R 127.0.0.1:9000:10.99.0.2:8080 labuser@10.99.0.1
Output
labuser@10.99.0.1: Permission denied (publickey).
key ok from 10.99.0.2 52909 22
Error: remote port forwarding failed for listen port 9000

The right password no longer gets in, and the registered key does. The tunnel is refused even after a key login.

Undo the changes and clear away the practice setup.

sudo rm /etc/ssh/sshd_config.d/00-lab.conf
sudo systemctl restart ssh
sudo rm -rf /opt/labtools /tmp/lab_key /tmp/lab_key.pub /tmp/admin_key /tmp/admin_key.pub
sudo ip netns del outside
sudo loginctl terminate-user labuser
sudo userdel -r labuser
sudo sshd -T | grep -E '^(permitrootlogin|passwordauthentication|allowtcpforwarding) '
Output excerpt
permitrootlogin without-password
passwordauthentication yes
allowtcpforwarding yes

These are the VM's original values.

Key-only logins do not remove a planted key

PasswordAuthentication no stops password logins only. It does not stop key logins. If the attacker has already planted a key, they still get in after you tighten the settings. Before tightening, check authorized_keys for keys you do not recognise.

#What you cannot see

What was checked, and what was not

This is what was checked in a disposable Ubuntu 24.04 VM, and in the code.

  • "Login succeeded after a brute-force attempt" needs failures before the success. In the code, at least half the threshold (3 by default) must fail within the window. A login with the right password from the start, or with a stolen key, does not raise it
  • The alert does not cut the session. A person cuts it with loginctl terminate-session
  • Key watching covers authorized_keys and authorized_keys2 only. Keys in a location set by AuthorizedKeysFile, and keys served by AuthorizedKeysCommand, are not watched (as detection-coverage says)
  • The alert for an ssh -R listener only reports. It cuts neither the tunnel nor the session. A person cuts it with loginctl terminate-session
  • Tunnelling tools are counted by name. A renamed tool is not counted. A note in the code says ngrok, frp and ssh are left out of the name list on purpose, because people use them every day
  • The SSH audit looks at PermitRootLogin and PasswordAuthentication. AllowTcpForwarding is not part of it (checked in the code)
Check yourself

"An SSH session opened a listening port" arrived, but who does not show the session. How do you check it?

Show the explanation

A session without a terminal, such as one running only ssh -N -R, does not show in who. From the sshd PID in the alert (the same as the listener's PID in ss -ltnp), the sessionid gives the account and the source in loginctl. If you do not know it, cut it with loginctl terminate-session.

Check yourself

"Login succeeded after a brute-force attempt" arrived, and you changed the password. What is left to check?

Show the explanation

If a key was planted right after the login, the attacker gets in with the key after the password change. A key login needs no failures, so the same alert does not come again. Running fim update first puts the planted key into the baseline.

#What to learn next

  • Banning brute force and limiting SSH sources to an allow list are covered in Configuring SSH protection. Even with a planted key, new connections from outside the allow list are refused
  • Watching critical files for tampering as a whole will be covered in "Persistence and tampering" (coming soon)
  • Every setting in the configuration file is in the Linux Server operations manual
Features covered

Related techniques (ATT&CK): T1572 T1098.004 T1548.003 T1059.004 T1027 T1105 T1553.001

What was verified

Verified on 2026-11-12. Target: Linux 1.17.39. Ran every lab top to bottom in Japanese and English (LANG=en_US.UTF-8) on two disposable Ubuntu 24.04 VMs with the published 1.17.39, with fail2ban left running and sources in a network namespace, including the alert for an ssh -R listener, its allow-list entry and the masking of command lines.

Command output varies with the product version and the environment. If yours differs, check the version first.