AWS Systems Manager Agent is a software agent that can be installed and configured on public or private EC2 instances, on-premise servers or virtual machines. The agent makes it possible for Systems Manager to update, manage, and configure these resources. The following guide shows you how to securely use the SSM agent along with the Systems Manager API to use port forwarding via a tunnel to connect into your private EC2 without running bastion hosts/jump boxes and without opening inbound ports to the instance.
Routing Requirements
The agent must be able to communicate with the Systems Manager Service Endpoint, so even if the EC2 is within a private subnet it must have a routing table to a NAT Gateway, NAT instance or configured with an AWS Private Link. For the full list of System Manager service endpoints see https://docs.aws.amazon.com/general/latest/gr/ssm.html
Install SSM Agent on Windows
SSM Agent is installed by default on instances created from Windows Server 2016/2019 AMIs, and on instances created from Windows Server 2003-2012 R2 AMIs published in November 2016 or later.
If your EC2 download the agent directly through a browser or use powershell
To download the latest agent, see https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/windows_amd64/AmazonSSMAgentSetup.exe
For further information including powershell instructions see https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-install-win.html
Install SSM Agent on Linux
SSM Agent is installed, by default, on Amazon Linux base AMIs dated 2017.09 and later. SSM Agent is also installed, by default, on Amazon Linux 2, Ubuntu Server 16.04, and Ubuntu Server 18.04 LTS AMIs.
To manually install the agent on other versions of Linux see https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-install.html
To check if it is running in Linux running the following command which should show it active and running
sudo systemctl status amazon-ssm-agent
Create an IAM User with the necessary permissions
Sign into the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/
Select Users then Add User, give it a logical name and select programmatic access
On the permissions page, select attach existing policies directly, then create policy, switch to the JSON tab and paste the following code below;
Please be aware this JSON is for full (administrative) access to all sessions, to restrict it down to certain instances or instances based on tags see here
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ssm:StartSession",
"ssm:TerminateSession",
"ssm:ResumeSession",
"ssm:DescribeSessions",
"ssm:GetConnectionStatus"
],
"Effect": "Allow",
"Resource": [
"*"
]
}
]
}Review the policy, adding a logical name and description
Once the policy has been created, add it to your user account.
Once reviewed select create user and download the access keys for use later.
Create an IAM Role with the necessary permissions for the EC2 to call AWS services
Still within IAM, select Roles and Create Role
Select AWS Service then EC2 then Next> Permissions
Search for SSM then select AmazonEC2RoleforSSM
Select Next and add any tags and give the role a logical name
Once the role has been created, switch to EC2 management and attach the role
Select the role created earlier and attach it to the EC2, providing the SSM endpoint is available and communicating and the agent is running it should appear as online within Systems Manager > Managed Instances as below
Once your EC2 is at this stage you can now attempt connection from your client
On your client
Install the latest session manager plugin,
Linux:
64-bit
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm" -o "session-manager-plugin.rpm"
32-bit
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_32bit/session-manager-plugin.rpm" -o "session-manager-plugin.rpm"
Install
sudo yum install -y session-manager-plugin.rpm
Install the latest AWS Command Line Interface
Windows x64: https://s3.amazonaws.com/aws-cli/AWSCLI64PY3.msi
Windows x32: https://s3.amazonaws.com/aws-cli/AWSCLI32PY3.msi
Linux: Comes preinstalled with a Linux AMI alternatively install it via the following command but it does require Python 2.6.5 or greater
pip install awscliInitiate connection through AWS
Linux
A CLI profile using the aws configure command will be required before accessing it entering the access keys created earlier, once this is created you should be able to setup the reverse tunnel using the following,
# find the instance ID based on Tag Name
INSTANCE_ID=$(aws ec2 describe-instances \
--filter "Name=tag:Name,Values=CodeStack/NewsBlogInstance" \
--query "Reservations[].Instances[?State.Name == 'running'].InstanceId[]" \
--output text)
# create the port forwarding tunnel
aws ssm start-session --target $INSTANCE_ID \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["443"],"localPortNumber":["9999"]}'Once the session is established, you should now be able to connect through the tunnel through your local browser on the port specified
Windows
Open up command line or powershell, change to the installed AWS directory
cd "C:\Program Files\Amazon\AWSCLI\bin"Run a simple command to ensure it is installed and return your AWS CLI version
aws --version
Add the access keys and set the region to a CLI profile using the aws configure command to create a profile
All being well you should now be able to run the following command against your private instance which should start the session
aws ssm start-session --target "Your Instance ID" --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["443"],"localPortNumber"=["9999"]
Once the session is established, you should now be able to connect through the tunnel through your local browser on the port specified
Alternatively, you could SSH in on port 22 using the following command
aws ssm start-session --target "Your Instance ID" --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["22"],"localPortNumber"=["9999"]
Once the connection has established you can use putty against your localhost on port 9999, remember to specify the correct PPK file if used.
So you are now in a position to work on your private instance when outside your AWS estate without a VPN, public IP or bastion/jump box.
To close the connection, hit Ctrl+C in the session that you used to initiate the connection.
Troubleshooting
If you get the error "An error occurred (TargetNotConnected) when calling the StartSession operation: <instance id> is not connected"
- Firstly ensure that the SSM agent is up to date, see https://github.com/aws/amazon-ssm-agent/releases for the latest version
If you get an error similar to the following;
"----------ERROR-------
Setting up data channel with id InfraEngineer-PortForwardingUsingSSM-03069fe658b04d9a9 failed: failed to create websocket for datachannel with error: CreateDataChannel failed with no output or error: createDataChannel request failed: failed to sign the request: EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request
status code: 404, request id:
caused by: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>404 - Not Found</title>
</head>
<body>
<h1>404 - Not Found</h1>
</body>
</html>"
Ensure that an IAM Role is setup and attached to the EC2 as described above so that it is registered as a managed instance in systems manager, once attached it may take a few minutes but it should appear in AWS systems manager similar to the following