I am setting up a distributed architecture on AWS and need to establish a secure connection between my "Bastion" host and a private "Application" server, both running Amazon Linux 2023. I've attempted to use the standard SSH command, but I keep getting "Connection Timed Out" or "Permission Denied (publickey)" errors. Should I be using the private IP address for this, and do I need to physically copy my .pem private key onto the first instance, or is there a more secure method like SSH Agent Forwarding or IAM Instance Profiles?
3 answers
The most common hurdle is the Security Group configuration. You must ensure the destination instance has an Inbound Rule allowing SSH (Port 22) specifically from the Security Group ID or Private IP of the source instance. For the actual connection, I strongly recommend against storing your .pem key on the first server. Instead, use SSH Agent Forwarding. On your local machine, run ssh-add -K your-key.pem and then connect to the first instance using the -A flag. This allows you to jump to the second instance using its Private IP without ever exposing your private key on the cloud filesystem.
Are you ensuring that both instances are residing within the same Virtual Private Cloud (VPC), and have you verified that the Subnet's Network ACLs aren't blocking internal traffic?
You need to authorize the source instance's IP in the destination's Security Group and use ssh -i key.pem user@private-ip. Using the Private IP is much faster.
I agree with Nancy. Just remember that if you go the "copying the key" route, you must run chmod 400 key.pem on the first instance or SSH will reject the key for being too accessible!
Christopher makes a vital point about the VPC structure. Even if your Security Groups are perfect, if those instances are in different subnets with restrictive NACLs, the traffic will never reach the destination. Joseph, you should also check if you are using the Private IP. Attempting to connect via Public IP from within the same VPC often fails unless you have an Internet Gateway and a Public IP assigned to both, which is less secure and adds unnecessary latency compared to internal routing.