Use a SSH-key to access your cloud resources with socks-proxy
When creating machines and services in the cloud we can specify the authentication method for SSH. We can authenticate with a password or by using a public key. It's very common to access your Linux machines with a SSH connection using the public-key mechanism because it's more secure and convenient.
You should not share a single key for all your SSH connections. A good practice is to create a new key per 'context'. For example, a new key for different clients and environments.
Set up a simple passphrase to encrypt your private key. When your private-key is stolen/leaked, it won't be a direct threat to access your resources: they will also need your passphrase. ssh-agent can remember this passphrase, so you don't need to type in the password every time you make a connection.
SSH key pair 🔑
Lets create a new ssh-key pair with a specific name:
# your machine # if the .ssh folder does not exist in your home, create it with access right 700 ssh-keygen -f ~/.ssh/client_x_cloud_development Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/abij/.ssh/client_x_cloud_development. Your public key has been saved in /Users/abij/.ssh/client_x_cloud_development.pub. The key fingerprint is: SHA256:LTEV93R8l9ScJyBMwyj5sMVjVK+G/vzJfOR+CJhgRaM
Two files have been generated: - client_x_cloud_development: private key - client_x_cloud_development.pub: public key end with .pub
Never give away your private key, it's stored with permission 600 for a reason!
When you have created your Linux node with the corresponding public key you should be able to access it. Here is an example how to create a VM-instance in GoogleCloudPlatform with the public ssh-key:
Now spin up the machine and wait a while until the machine becomes available. To connect to the machine we have to look up its public/external IP-address. As you can see below for this VM it is 184.108.40.206 with the hostname
Lets connect with SSH to the machine and install a webserver to demonstrate port forwarding.
# on your machine, verbose connection string: ssh -i ~/.ssh/client_x_cloud_development email@example.com <password to decrypt your private key> abij@my-cool-node:~# # on remote-machine, install a webserver sudo apt-get install -y apache2 # sudo yum install -y apache2 # Now your apache-server listens on port 80 for http traffic. # You see (-l listen -t tcp -n numbers): ss -ltn State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 128 *:80 *:* LISTEN 0 128 :::22 :::*
SSH tunnel, single port forward
By default the website is not accessible over public internet (http://220.127.116.11:80): if you have a private cluster you don't want to expose it publicly. If we want to view the website on the server we can tunnel the port through the SSH-session.
SSH tunnel with -L [MyPort]:[AfterTunnel_address]:[DestPort]
# on your machine, verbose connection string with tunnel ssh -i ~/.ssh/client_x_cloud_development firstname.lastname@example.org -L8080:localhost:80
Now we can access the website with the address: http://localhost:8080 going to localhost 8080 in the tunnel. After the tunnel to localhost port 80.
To make your life better, we can configure this connection string in the ssh-config file.
# your machine, edit your SSH-config file touch ~/.ssh/config vim ~/.ssh/config host client_x_dev HostName 18.104.22.168 IdentityFile "~/.ssh/client_x_cloud_development" User abij LocalForward 8080 localhost:80
Now we can connect using:
ssh client_x_dev from your machine.
Make your linux-node more secure by disallowing username/password login. You can check the config of the ssh daemon on the remote linux node
/etc/ssh/sshd_configmust have ChallengeResponseAuthentication and PasswordAuthentication set to no. This is default for cloud machines created with public-key-login.
This works nice for a single port on one machine, but a cluster consists of multiple servers with multiple ports, with links to each other. The links will not work with this setup, because we are using localhost / 127.0.0.1 in our browser. For this scenario we can use the socks proxy.
A socks proxy will forward all the ports and can forward the DNS-lookups. All traffic will be transported through the SSH-connection and is thereby encrypted when the proxy is enabled. All requests go through the tunnel! You should not internet in the same browser window with the socks proxy enabled. You should have a separate window, or like me a different browser for internet.
Socks proxy can be configured in any browser. I use Firefox, because its not my default and the Socks Proxy plugin is super easy and simple to use.
- Install Firefox
- Install Add-On: Socks Proxy
By default it will forward DNS-lookups, connect to 127.0.0.1 on port 1337. Lets change the ssh config for DynamicForward instead of LocalForward to enable the sock-proxy.
SSH DynamicForward with command-line option -D [localport]
# your machine, ssh config file: ~/.ssh.config host client_x_dev HostName 22.214.171.124 IdentityFile "~/.ssh/client_x_cloud_development" User abij DynamicForward 1337
When you connect to your linux node with
ssh client_x_dev you are ready to go! You can now access the internal ip-addresses and internal DNS-server names to access your websites. Internal links between servers also work!
When creating a cloud resources or linux servers in general you should prefer SSH-keys with a passphrase over password login. Simplify your ssh connections with the ~/.ssh/config file.
If you have a single port you want to access you can use port forwarding with LocalFoward in your config file. You can reach the port by using localhost on your browser. A limitation is that you cannot use DNS-lookups, link to other nodes don't work and have to add every host/port as a separate entry the ssh-config file.
With the dynamicForward we can use a socks proxy to enable these features. This works very well with a cluster in the cloud without public internet access. The communication between the browser and the remote-machine is encrypted.
GoDataDriven open source contribution: December 2017 edition
January 09, 2018
Post-Christmas Coding, December 28
December 13, 2017
GDD @ NIPS 2017
December 12, 2017
Spark packages from a password protected Repository
December 11, 2017
GoDataDriven open source contribution: November 2017 edition
November 30, 2017
How to Decorate Your Scikit-learn Models Like a Christmas Tree
November 22, 2017