Remote Access

From CLSP Wiki
Jump to: navigation, search

Getting started with ssh

To ssh into the CLSP grid from a Unix (Mac or Linux) machine, open a terminal on your local machine and type:

ssh <username>@login.clsp.jhu.edu

If this does not bring up a prompt for password, something is wrong. You may need to set up ssh on your machine (especially if running Windows), but at this point you should probably ask for help from a human.

If you are accessing the CLSP grid from a Windows machine you will likely want to download and use PuTTY, an ssh client for Windows. To ssh into the CLSP grid using PuTTY, first start PuTTY, then type login.clsp.jhu.edu into the hostname/IP address box and click the "Open" button at the bottom of the window, and then enter your username and password in the terminal window that is created (following the prompts).

If you do not have a <username>, email clsphelp@clsp.jhu.edu and ask for one.

Once you have ssh-ed into login (or login2), you should immediately ssh into a different node (e.g. a05) and then work from that node.

Mitigating network interruptions

You may want to use tmux (or screen, an older alternative) so that you don't lose your place (open editors, running commands, terminal output) when your SSH connection is interrupted or you close your laptop.

To start a new tmux session run tmux. By default the tmux command prefix is Ctrl-b; to use Ctrl-b in a program running inside tmux (such as vi, in which Ctrl-b scrolls up one page) you must type it twice (so to scroll up in vi you type Ctrl-b Ctrl-b). To explicitly detach from a running tmux session type Ctrl-b d; to reattach run tmux a -d. If your SSH connection is interrupted or you end it (by, for example, using your mouse to close your terminal), you can later SSH back in and reattach to your tmux session to resume work. If however you type exit at the shell prompt, the tmux session will end and you will need to start a new one.

You can have multiple windows inside a tmux session, allowing you to (for example) edit a file in one window while running tests on it in another; for information about this feature and others, search for "tmux tutorial" on the web.

Easily SSH directly to a/b machines

Assuming that your username is clspuser save the following as `.ssh/config` on your local computer

$ cat .ssh/config
Host clsp
   HostName login2.clsp.jhu.edu
   User clspuser

Host b*
   ProxyCommand ssh -q clsp nc -q0 %h 22
   User clspuser

Host a*
   ProxyCommand ssh -q clsp nc -q0 %h 22
   User clspuser

Now you can directly type `ssh clsp` or `ssh a15` or `ssh b15` from your personal machine to directly ssh to the respective machines.

Easily SSH to other machines on the CLSP network

For example, if you need to easily SSH to Magnesia (and already have an account). If you need an account, e-mail Chu-Cheng Lin (chu.cheng.lin@gmail.com) or Hongyuan Mei (hongyuanmei@gmail.com) to get set up. Note: You may have to be a member of Jason's lab (Argo) in order to get set up with this machine, but this trick should work for any other machines on the CLSP network. You should use these instructions if the section above didn't work for you.

This trick is from Matthew Francis-Landau: On the CLSP grid machine, create an executable file

~/bin/clsp-proxy

with the following:

#!/bin/bash
pre="clspg-"
host=${1#$pre}
exec nc -q0 $host $2

To make executable, type:

chmod +x ~/bin/clsp-proxy

Then on your personal machine edit

~/.ssh/config 

to contain:

Host clspg
    HostName login.clsp.jhu.edu
    User USERNAME
Host clspg-*
    User USERNAME
    ProxyCommand ssh -qax clspg "/home/USERNAME/bin/clsp-proxy" %h %p

You should be able to type:

ssh clspg-a10 -L 8888:localhost:8888

from your local computer to connect to a10 (or replace "a10" with the machine you want).

Running IPython Notebooks remotely on CLSP and interacting on local computer

The following command starts a ipython notebook server in its dedicated screen on port 7000 and then port forwards that port to `localhost:6003`. The following function assumes that you have already created the `.ssh/config` files described above

 function remotebook () {
   ssh -f b15 'screen -S ipython_server -d -m ipython notebook --no-browser --port=7000'
   ssh -Nf -L 6003:localhost:7000 b15
 }


Mail when Job is Over

Sometimes we may want to receive an email when a job is over. Or when the total number of jobs has dropped to 1 or 2 instead of receiving an email every time a job finishes. The following bash function shows a way to send such a mail.

 mailWhenJobsDropTo () {
       minjobs=${1-0};
       qstatlinecount=$((2 + ${minjobs}));
       myname=$(whoami);
       while [ `qstat -u $myname | wc -l` -gt $qstatlinecount ] ; do sleep 300; done;
       cat <(qstat -u ${myname}) | mail -s "$(domainname) jobs dropped below ${minjobs}" 
email@example.org;
 }