Skip to content

Emergency Access

Overview

Cloud instances have emergency access methods that work even when:

  • SSH port 22 is unreachable
  • ZeroTier is down
  • The host is unresponsive to normal connections

These use cloud provider infrastructure, not the public internet.

Access Priority

  1. ZeroTier (192.168.194.x) - Preferred, fastest
  2. Direct SSH (public IP) - Standard access
  3. Cloud Console (SSM/IAP) - Emergency fallback

AWS: SSM Session Manager

Prerequisites

  • IAM role pluto-ssm-role with AmazonSSMManagedInstanceCore policy
  • Agent: amazon-ssm-agent (snap, auto-installed on Ubuntu 24.04)
  • No inbound ports required - agent connects outbound to AWS APIs

Interactive Shell

aws ssm start-session --target i-05e7dd5e009d6d766 --region us-west-2

AWS Console

  1. Go to EC2 → Instances → pluto
  2. Click Connect
  3. Select Session Manager tab
  4. Click Connect

Run Remote Command

aws ssm send-command --instance-ids i-05e7dd5e009d6d766 \
  --document-name "AWS-RunShellScript" \
  --parameters 'commands=["systemctl status zerotier-one"]' \
  --region us-west-2

View Command Output

# Get command ID from send-command output
aws ssm list-command-invocations \
  --command-id COMMAND_ID \
  --details \
  --region us-west-2

Troubleshooting SSM

# Check if SSM agent is running
ssh joe@pluto "sudo systemctl status snap.amazon-ssm-agent.amazon-ssm-agent"

# Restart agent
ssh joe@pluto "sudo systemctl restart snap.amazon-ssm-agent.amazon-ssm-agent"

# Check connectivity to SSM endpoints
ssh joe@pluto "curl -s https://ssm.us-west-2.amazonaws.com"

GCE: IAP SSH

Prerequisites

  • Firewall rule allow-iap-ssh (35.235.240.0/20 → port 22)
  • Network tag iap-ssh on the instance
  • No public SSH exposure - connections route through Google's IAP infrastructure

SSH Through IAP

gcloud compute ssh dumbo --zone=us-central1-a --tunnel-through-iap

GCP Console

  1. Go to Compute Engine → VM instances → dumbo
  2. Click SSH dropdown
  3. Select Open in browser window

Run Remote Command

gcloud compute ssh dumbo --zone=us-central1-a --tunnel-through-iap \
  --command="systemctl status zerotier-one"

Troubleshooting IAP

# Verify firewall rule exists
gcloud compute firewall-rules describe allow-iap-ssh

# Check instance has correct tag
gcloud compute instances describe dumbo --zone=us-central1-a \
  --format="get(tags.items)"

# Test from inside GCP
gcloud compute ssh dumbo --zone=us-central1-a --tunnel-through-iap \
  --command="curl -s http://metadata.google.internal"

Common Scenarios

SSH Timeout

  1. Try ZeroTier IP: ssh joe@192.168.194.x
  2. Try direct IP: ssh joe@52.32.80.62
  3. Use SSM/IAP as fallback

Host Unresponsive

  1. Check instance status in cloud console
  2. Use SSM/IAP to get shell access
  3. Check logs: journalctl -xb

ZeroTier Down

# Via SSM (AWS)
aws ssm start-session --target i-05e7dd5e009d6d766 --region us-west-2

# Check ZeroTier status
zerotier-cli listnetworks

# Restart ZeroTier
sudo systemctl restart zerotier-one

Banned by fail2ban

If your IP was banned:

# Use SSM/IAP to access host
aws ssm start-session --target i-05e7dd5e009d6d766 --region us-west-2

# Unban your IP
sudo fail2ban-client set sshd unbanip YOUR_IP

Instance Won't Start

AWS:

# Check instance status
aws ec2 describe-instance-status --instance-ids i-xxx

# Get console output
aws ec2 get-console-output --instance-id i-xxx

GCE:

# View serial console
gcloud compute instances get-serial-port-output dumbo --zone=us-central1-a

Instance Details

AWS - Pluto

Attribute Value
Instance ID i-05e7dd5e009d6d766
Region us-west-2
SSM Role pluto-ssm-role
Public IP 52.32.80.62

GCE - Dumbo

Attribute Value
Instance Name dumbo
Zone us-central1-a
Network Tag iap-ssh
Public IP 34.44.33.3

Security Notes

  • SSM/IAP connections are logged in CloudTrail/Cloud Audit
  • These methods bypass normal SSH - use only when necessary
  • Always prefer ZeroTier or direct SSH for regular access
  • Consider setting up alerts for SSM/IAP usage