How to Work on a Remote Machine via SSH

Welcome to my tutorial on how to set up a remote machine and deploy your code there. I will first analyze SSH and then look at two specific applications: coding in Python and Julia.

Setup

In order to start working on a remote server you need

• the server
• local shell
• SSH installed

SSH, or Secure Shell, is a protocol designed to transfer data between a client and a server (two computers basically) over an untrusted network.

The way SSH works is it encrypts the connection using a pair of keys and the server, which is the computer you would connect to, is usually waiting for an SSH connection on Port 22.

SSH is normally installed by default. To check if you have SSH installed, open the terminal and write ssh. You should receive a message that looks like this

usage: ssh [-1246AaCfGgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
[-D [bind_address:]port] [-E log_file] [-e escape_char]
[-F configfile] [-I pkcs11] [-i identity_file]
[-J [user@]host[:port]] [-L address] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-Q query_option] [-R address] [-S ctl_path] [-W host:port] [-w local_tun[:remote_tun]]
[user@]hostname [command]

If SSH is not installed, you can install it using the following commands.

sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh

Now that you have installed SSH, we are ready to setup a remote connection.

From the computer you want to access remotey, generate the public key.

ssh-keygen -t rsa

You will be asked for a location. If you decide to enter one manually then that will be the pair’s location, if you leave the default one it will be inside the .ssh hidden folder in your home directory.

Now you will be prompted for a password. If you enter one you will be asked for it every time you use the key, this works for added security. If you don’t want a password just press enter and continue without one.

Two files were created. One file ends with the ‘.pub’ extension and the other one doesn’t. The file that ends with ‘.pub’ is your public key. This key needs to be in the computer you want to connect to (the server) inside a file called authorized_keys . You can accomplish this with the following command:

ssh-copy-id username@ip

For example in my case to send the key to my computer it would be:

ssh-copy-id sergiop@132.132.132.132

If you have MacOS there’s a chance you don’t have ssh-copy-id installed, in that case you can install it using

brew install ssh-copy-id

If you haven’t installed brew, you can install it by following this guide.

Connect

To permanently add the SSH key, you can use the follwing command

ssh-add directory\key.pem

Lastly, to connect, just type the following command.

ssh username@ip

Where username is the server name and ip is the public IP adress, e.g. 132.132.132.132.

If your server is not public, you will not be able to access it.

If your server is password protected, you will be prompted to insert a password when you connect. If not, you should protect it with a password.

Managing screens

While you are connected to the remote terminal, any disturbance to your connection will interrupt the code. In order to avoid that, you want to create separate screens. This will allow your code to run remotely undisturbed, irrespectively of your connection.

First, you need to install screen.

brew install screen

To create a new screen, just type

screen

Now you can lunch your code.

After that, you want to detach from that screen so that the code can run remotely undisturbed.

screen -d

Another option is to use ctrl+a followed by ctrl+d. This will detach the screen without the need to type anythin in the terminal, in case the terminal is busy (most likely).

To list the current active screens type

screen -ls

If you want to check at any time that your code is running, without re-attaching to the screen, you can just type

top

which is the general command to check active processes. To exit, use ctrl+z, which generally terminates processes in the terminal.

To reattach to your screen, type

screen -r

In case you have multiple screens (you can check with screen -ls), you can reattach to a specific one by typing

screen -r 12345

where 12345 is the id of the screen.

To kill a screen, type

screen -XS 12345 quit

where again 12345 is the id of the screen.

Python and Pycharm

If you are coding in Python, PyCharm is one of the best IDEs. Among many features, it offers the possibility to set a remote compiler for your pthon console and to sync input and output files automatically.

First, you need to have setup a remote SSH connection following the steps above. Importantly, you need to have added the public key to your machine using the ssh-add command, as explained above.

Then open Pytharm, go to the lower-right corner, where the current interpreter is listed (e.g. Pytohn 3.8), click it and select interpreter settings.

Click on the gear icon ⚙️ on the top-right corner and select add.

Insert the server host (IP address, e.g. 132.132.132.132) and username (e.g. sergiop).

Next, you have to insert your credentials. If you have a password, insert it, otherwise you have to insert the path to your SSH key file.

Lastly, select the remote interpreter. If you are using a python version that is not default, browse to the preferred python installation folder. Also, check the box for execute code giving this interpreter with root privileges via sudo.

You can also select which remote folder to sync with your local project. By default, you are given a tmp/pycharm_project_XX folder. You can change it if you want. I recommend also to have the last option checked: automatically sync project files to the server. This will automatically synch all remote changes with your local machine, in your local project folder.

Julia and Juno

If you are coding in Julia, Juno is the best IDE around. It’s an integration with Atom with a dedicated compiler, local variables, syntax highlight, autocompletion.

On Atom, you first need to install the ftp-remote-edit package.

Then go to the menu item Packages > Ftp-Remote-Edit > Toggle.

A new Remote panel will open with the default button to Edit a new server.

Click it and you will be able to set up your remote connection.

• Press New
• Insert your username in The name of the server, for example sergiop
• Insert your ip adress in The hostname or IP adress of the server, for example 123.123.123.123
• Select SFTP - SSH File Transfer Protocol under Protocol
• Select your Logon option. You can either insert your password every time, just once, or use a keyfile.
• Insert again your username in Username for autentication, again for example sergiop
• If you don’t want to start from the root folder, you can change the Initial Directory

Now you will be able to see your remote directory (named for example sergiop) in the Remote panel.

To start using Julia remotely, just start a new remote Julia process from the menu on the left.

Now you are ready to deploy your Julia code on your remote server!

Jupyter Notebooks

If you want to have a Jupyter Notebook running remotely, the steps are the following. The main advantage of a Jupyter Notebook is that it allows you to mix text and code in a single file, similarly to RMarkdown, with the advantage of not being contrained to use a R (or Python) kernel. For example, I often use Jupyter Notebook with Julia or Matlab Kernels. Moreover, you can also make nice slides out of it!

First, connect to the remote machine. Look at section 1 to set up your SSH connection.

ssh username@ip

Start a Jupyter Notebook in the remote machine.

jupyter notebook --no-browser

The command will open a jupyter notebook in the remote machine. To connect to it, we need to know which port it used. The default port is 8888. If that port is busy, it will look for another available one. We can see the port from the output in terminal.

Jupyter Notebook is running at: http://localhost:XXXX/…

Where XXXX is the repote port used.

Now we need to forward the remote port XXXX to our local YYYY port.

Open a new local shell. Type

ssh -L localhost:YYYY:localhost:XXXX username@ip

Where YYYY can be anything. I’d use the default port 8888.

ssh -L localhost:8889:localhost:8888 username@ip

Now go to your browser and type

localhost:YYYY

Which in my case is

localhost:8889

This will open the remote Jupyter Notebook.

Done!

In case you want to check which Jupiter notebooks are running, type

jupyter notebook list

To kill a notebook use

jupyter notebook stop XXXX

Sources

• How To Setup And Use SSH For Remote Connections
• Connecting to a Julia session on a remote machine
• Running a Jupyter notebook from a remote server