Connecting to the HPC

How to connect to kennedy

Access to kennedy is only possible from within the St Andrews network. Unless you are in a University building or a student hall, you need to connect to the St Andrews VPN.

Most users will use the cluster by logging in to the login node and interacting with it via the Linux command line. To do this, you will need:

  • a terminal program
  • secure shell (ssh). In most cases this is included with the terminal program
  • an X11 server to allow you to open graphical applications (anything with output not entirely inside the text terminal) on your screen.

The solutions depend on your operating system.

Linux

You probably have everything installed. Connect to kennedy using the -X or -Y flag to enable X11 forwarding:
  ssh -Y <user>@kennedy.st-andrews.ac.uk

Mac OS

The terminal program is simply called "terminal". For X11 forwarding, you need to install and start XQuartz. This is available for free download from its website. You connect by entering (in the terminal)
  ssh -Y <user>@kennedy.st-andrews.ac.uk

Windows

There are multiple solutions, both commercial and free. The most commonly used ones are PuTTY as the terminal and ssh client with VcXsrv as the X11 server, or cygwin (a complete Linux emulation on Windows). 

Data transfer

Methods to transfer data to and from a remote Linux server such as kennedy also depend on your operating system, as well as the structure of the data (e.g. if you are dealing with individual files or whole (sub-)directories).

sftp

On Unix-based platforms (including MacOS and CYGWIN), the command-line based sftp offers a fast and reliable transfer mechanism. The syntax is

sftp <user>@<host>

e.g. "sftp hf63@kennedy.st-andrews.ac.uk". You will be prompted for authentication information such as password and/or passphrase. Once logged in, you have access to a limited number of Unix commands (such as cd or ls), and in addition the commands "put" and "get".

put <file>

will transfer the file <file> from your local computer to the remote server. The command

get <file>

works in the opposite direction. Whole directories can be transferred with the -r (recursive) flag, for example with

get -r <dir>

rsync

The rsync command is a more flexible and efficient way of synchronizing whole subdirectories. It can be used to preserve the modification date of a directory (-p or -a flags) or only copying files that are newer on the source machine than on the target (-u flag). A typical ecample would be

rsync -avu hf63@kennedy.st-andrews.ac.uk:work/testdir/ testdir/

The flags in this case mean:
-a "archive": copy directories recursively and preserve dates and permissions
-v "verbose": show which files are copied
-u "update": only copy files that are not of the same age or newer on the target computer

WinSCP

On Windows, a simple and intuitive graphical user interface is WinSCP, which can be downloaded free from the WinSCP website. In its default installation mode, it presents a double window (a la Norton Commander) where you can drag and drop files or directories between your local Windows computer and a remote Linux server.

FileZilla

Similar to WinSCP. FileZilla is slightly more complex, but it is available for Windows, MacOS and Linux.

Here are instructions for setting it up to use ssh keys and a password:

In the main menu, go to Edit -> Settings -> Connection -> SFTP
"Add key file" and browse to your ppk file (or private key on Linux or MacOS)

In the main menu, go to File -> Site Manager
Create a new site "kennedy"
Protocol "SFTP"
Logon Type: "Ask for password"
Press "Connect" key

In future connections, just select kennedy from the Site Manager and connect.

Authenticating with ssh keys and Unix password

To log in to kennedy, you need a password, and additionally a pair of ssh keys.

Password

When your account is created, you will be given an initial password. You should change this as soon as possible to a password that only you know. The Linux command for changing your password is

passwd

It will ask you for your current password, and then for the new one (twice, for confirmation). The minimum requirements are:

  • At least 10 characters
  • At least one of them not a letter or digit
  • non-obvious (passwords are checked against a list of known bad passwords)

ssh keys

An ssh key pair consists of two elements:

  • A private key file, stored on your computer, and not shared with anybody
  • A public key, which can be distributed using insecure mechanisms such as email or used at multiple sites, because on its own it does not allow anybody to impersonate you as a user. 

Please send your public key to the system administrator. Do not send the private key!

How you create a key pair depends on your operating system:

Linux / MacOS / CYGWIN

The command is

ssh-keygen

Accept all default filenames (different filenames and storage locations are possible, but complicate things; ask if you are unsure). Then send the public key, which is the file ~/.ssh/id_rsa.pub. The file without .pub suffix is your private key and should not be sent.

If you have problems locating the file, this is likely due to the fact that the .ssh directory is "invisible" to the file browser. Just copy it to your home directory:

cp ~/.ssh/id_rsa.pub ~

You can delete it from here after sending.

Windows (PuTTY)

If you install PuTTY from the installer, the PuTTY folder contains a program called PuTTYgen. Here is a screenshot demonstrating how to use it:

PuTTYgen screenshot

  • Press the "Generate" key
  • Move your mouse through the window to create some "randomness" until the progress bar disappears
  • Save the private key (lower circled button in picture). You will need this in PuTTY or WinSCP.
  • Copy the long string in the upper part (circled) and send to the system administrator. You may need to use the scrollbar to see the complete string. This is your public key in Unix format (not the file you get with "Save public key").
  • Do NOT send or share the private key!