Skip to content

SSH Access & Hardening

Access Policy

Security Policy

Never SSH as root. Always use joe user with sudo.

Host Access Table

Host SSH Access Sudo Notes
Blue joe@10.15.0.1 Passwordless Gateway, full access
Owl joe@192.168.194.10 (ZT) or IPv6 Passwordless Gateway, HE tunnel for IPv6
Pluto joe@pluto or joe@52.32.80.62 Passwordless AWS production
Mickey joe@mickey or ephemeral IP Passwordless AWS dev (IP changes)
Dumbo joe@dumbo or joe@34.44.33.3 Passwordless GCE
Bogart joe@bogart or joe@35.209.219.216 Passwordless GCE
Rocky joe@rocky or joe@193.8.172.100 Passwordless Meanservers (ZeroTier preferred, public SSH available)

Access Priority

  1. ZeroTier (192.168.194.x) - Preferred, fastest, full network access
  2. Direct SSH (public IP) - Works when ZeroTier unavailable
  3. Cloud Console (SSM/IAP) - Emergency fallback, always works if instance running

SSH Configuration

Client Config (~/.ssh/config)

# AWS Instances
Host pluto
    HostName 52.32.80.62
    User joe

Host mickey
    HostName 44.245.154.242  # Update with current IP
    User joe

# GCE Instances
Host dumbo
    HostName 34.44.33.3
    User joe

Host bogart
    HostName 35.209.219.216
    User joe

# Meanservers (ZeroTier preferred, public SSH available)
Host rocky
    HostName 192.168.194.132  # ZeroTier IP (preferred)
    # HostName 193.8.172.100  # Public IP (fallback)
    User joe

# Gateways (via ZeroTier)
Host owl
    HostName 192.168.194.10
    User joe

# Cloudflare Zero Trust
Host ssh-pluto.scandora.net
    ProxyCommand /opt/homebrew/bin/cloudflared access ssh --hostname %h
    User joe

Host ssh-dumbo.scandora.net
    ProxyCommand /opt/homebrew/bin/cloudflared access ssh --hostname %h
    User joe

SSH Hardening

All internet-exposed hosts have hardened SSH configuration:

Settings

Setting Value Purpose
PasswordAuthentication no Key-only auth
PermitRootLogin no No root SSH
MaxAuthTries 3 Limited attempts
LoginGraceTime 30 Quick timeout
X11Forwarding no Disable X11
LogLevel VERBOSE Enhanced logging

sshd_config

# /etc/ssh/sshd_config (managed by Ansible)
PasswordAuthentication no
PermitRootLogin no
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
LogLevel VERBOSE

fail2ban

All hosts run fail2ban with sshd jail:

Setting Value
bantime 3600 (1 hour)
maxretry 3 (matches MaxAuthTries)
findtime 600 (10 minutes)

Check Status

# View banned IPs
sudo fail2ban-client status sshd

# Unban an IP
sudo fail2ban-client set sshd unbanip 1.2.3.4

GeoIP Country Filtering

All cloud hosts filter SSH connections by geographic location using TCP Wrappers.

Defense in Depth

GeoIP filtering blocks connections before authentication - attackers from blocked countries never get a chance to try passwords or keys.

How It Works

Incoming SSH Connection
┌─────────────────────────┐
│  TCP Wrappers           │  /etc/hosts.allow checks GeoIP
│  (hosts.allow/deny)     │  via aclexec script
└─────────────────────────┘
    GeoIP Check ──────────► DENY (non-US) → Connection refused
         ▼ ALLOW (US or private IP)
┌─────────────────────────┐
│  SSH Daemon             │  Key authentication
└─────────────────────────┘
         ▼ Auth failed
┌─────────────────────────┐
│  fail2ban               │  Bans after 3 failures
└─────────────────────────┘

Current Configuration

Setting Value
Allowed Countries US only
Private IPs Always allowed (10.x, 192.168.x, 172.16-31.x)
Filter Script /usr/local/bin/ssh-geoip-filter.sh
Logging syslog with tag ssh-geoip

Configuration Files

# /etc/hosts.allow - calls GeoIP filter
sshd: ALL: aclexec /usr/local/bin/ssh-geoip-filter.sh %a

# /etc/hosts.deny - default deny
sshd: ALL

aclexec vs spawn

Must use aclexec (not spawn). Only aclexec enforces the exit code for allow/deny decisions.

Viewing Logs

# Real-time GeoIP decisions
sudo journalctl -t ssh-geoip -f

# Recent entries
sudo journalctl -t ssh-geoip --since "1 hour ago"

# Example output:
# ALLOW 216.147.124.117 (US)
# DENY 185.220.101.1 (DE)
# ALLOW 192.168.194.10 (private/local)

Testing the Filter

# Test a US IP (should return exit 0)
/usr/local/bin/ssh-geoip-filter.sh 8.8.8.8
echo $?  # 0 = allow

# Test a foreign IP (should return exit 1)
/usr/local/bin/ssh-geoip-filter.sh 185.220.101.1
echo $?  # 1 = deny

# Test a private IP (should return exit 0)
/usr/local/bin/ssh-geoip-filter.sh 192.168.194.10
echo $?  # 0 = allow

Adding Countries

Edit cloud/ansible/roles/base/defaults/main.yml:

ssh_geoip_allowed_countries:
  - "US"              # United States
  - "CA"              # Canada
  - "GB"              # United Kingdom

Then redeploy:

cd cloud/ansible
ansible-playbook -i inventory/HOST.yml playbooks/site.yml --tags base

Country codes: ISO 3166-1 alpha-2

Troubleshooting

Connection Refused After Deploy

If locked out after deploying GeoIP filtering:

  1. Use emergency access (SSM for AWS, IAP for GCE)
  2. Check your IP's country: curl -s ipinfo.io/country
  3. If wrongly classified, add your country to allowed list

Private IPs Being Denied

The filter should allow all RFC1918 addresses automatically. If not:

# Check the script handles private IPs
grep -A5 "private/local" /usr/local/bin/ssh-geoip-filter.sh

Disable GeoIP Temporarily

# Comment out the sshd line in hosts.allow
sudo sed -i 's/^sshd:/#sshd:/' /etc/hosts.allow

# Re-enable
sudo sed -i 's/^#sshd:/sshd:/' /etc/hosts.allow

Per-Host Overrides

To allow different countries on specific hosts, create host_vars:

# cloud/ansible/inventory/host_vars/pluto.yml
ssh_geoip_allowed_countries:
  - "US"
  - "CA"
  - "MX"

SSH Key Management

Adding Keys

Keys are managed via Ansible base role and deployed to /home/joe/.ssh/authorized_keys.

To add a new key:

  1. Add public key to cloud/ansible/files/ssh/authorized_keys
  2. Run Ansible playbook: ansible-playbook playbooks/base.yml

Current Keys

Keys are named by source machine:

  • saturn
  • triton
  • pluto
  • luna

Troubleshooting

Connection Refused

  1. Check if host is running
  2. Verify IP address is correct
  3. Try ZeroTier IP (192.168.194.x)
  4. Use emergency access (SSM/IAP)

Permission Denied

  1. Verify SSH key is loaded: ssh-add -l
  2. Check correct user (joe, not root)
  3. Verify key is in authorized_keys on host

Timeout

  1. Check firewall rules (port 22)
  2. Verify ZeroTier status: zerotier-cli listnetworks
  3. Try direct public IP instead of ZeroTier

Banned by fail2ban

If your IP was banned:

# From another host or emergency access
sudo fail2ban-client set sshd unbanip YOUR_IP

Wait 1 hour, or check from a different IP.

Gateway SSH (OPNsense)

OPNsense gateways run FreeBSD, not Linux:

# SSH to gateway
ssh joe@192.168.194.10

# Commands differ from Linux
# Use configctl for service management
sudo configctl service status

# View logs
clog -f /var/log/system.log

FreeBSD Shell

joe uses /bin/sh on Owl and /bin/tcsh on Blue. Consider standardizing.