Difference between revisions of "Persistent Terminal Sessions"

From UFRC
Jump to navigation Jump to search
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
=Introduction=
 
=Introduction=
−
Most of the time when you need to connect to HiPerGator to get work done by submitting jobs to the scheduler via an application that provides a graphical user interface - whether web based like [[JupyterHub]] or [https://galaxy.rc.ufl.edu Galaxy] or in a [[GUI Programs|GUI Session on HiPerGator]]. On the command line you use one of the multitude of SSH client applications or terminal applications depending on the operating system (Linux, MacOS, or Windows) and your personal preference. Linux has a great selection of terminal emulators that are only a package install away. In case of MacOS you can either use an included Terminal application or a more featureful 3-rd party terminal emulator like [https://www.iterm2.com/ iTerm2]. On windows there are many applications that provide terminal emulation like [https://www.chiark.greenend.org.uk/~sgtatham/putty/ Putty.exe], [https://mobaxterm.mobatek.net/ MobaXTerm], [https://winscp.net/eng/docs/free_ssh_client_for_windows WinSCP] and so on. You can even enable [https://docs.microsoft.com/en-us/windows/wsl/install-win10 Windows Subsystem for Linux] and use the terminal emulators that come with the Linux distribution you installed under WSL. In all cases the work you are doing on the command-line can be interrupted by a networking issue especially if you work off-campus. Most of the time you can ssh back in as user@hpg.rc.ufl.edu and continue your work. However, there some scenarios when it is very important that your terminal session on HiPerGator is not interrupted. Whatever the reason - perhaps you are running a test, working in an interactive scheduled session, monitoring a run, or in the middle of a long session of editing job scripts or configuration files or traversing the filesystem and you don't want to lose the place you are at - you need to be able to have a stable terminal environment for a while. There are several approaches you can take like starting a Terminal within a [[JupyterHub|Jupyter]] or [[GUI_Programs#Open_OnDemand|OnDemand]] session. On the command line you have two very powerful applications that you can use to keep your work from interruptions - [[Screen]] and [[Tmux]]. They are similar, but independent implementations of the concept of 'terminal multiplexing'. This means that you can ssh into HiPerGator, start a tmux or screen session, create multiple terminal windows inside that session, and be assured that you can disconnect (detach) and re-connect (re-attach) to those sessions as needed or after a network interruption. There are some caveats to this, of course. Since our login nodes are load-balanced you cannot be sure which login node you'll end up on after you ssh into HiPerGator. In that case you should remember which node you started your terminal multiplexer session on and ssh into it once you are on the system. E.g. if you started your session on login3, but end up on login2 after reconnecting you can ssh into login3 internally with <code>ssh login3</code>, load the appropriate environment module, and reattach to your running terminal multiplexer session.
+
__NOTOC__
 +
SSH sessions can be interrupted by a networking issue. Sometimes it is necessary to have a long-term session on a login node say for an interactive SLURM job, monitoring an analysis, or editing job scripts or configuration files, for example. There are several methods to get a long-term terminal session like starting a terminal within [[Jupyter#JupyterHub| JupyterLab]], [[GUI_Programs#Open_OnDemand|OnDemand]] session, or using a terminal multiplexer. We have two such applications on HiPerGator: [[Screen]] and [[Tmux]]. They are similar implementations of the concept of 'terminal multiplexing' or having multiple terminals in a single instance of the program.  
 +
 
 +
=Workflow=
 +
In short, working with a terminal multiplexer to have multiple persistent (do not disappear when you log out or your network connection drops) terminals on HPG login nodes usually goes as follows
 +
# Ssh into HiPerGator. You end up on a random login node. E.g.
 +
#*<pre>$ ssh myname@hpg.rc.ufl.edu</pre>
 +
# Pick a login node for your session and ssh into it. E.g.
 +
#*<pre>$ ssh login2</pre>
 +
# Start a tmux or screen. E.g.
 +
#*<pre>$ ml screen</pre>
 +
#*<pre>$ screen</pre>
 +
# Create as many terminal windows in the screen session as needed with 'Ctrl+a c' key combination. See [[Screen]] documentation.
 +
# Now, if you start an interactive SLURM job in one of the terminals in your screen session that job will still be running if you disconnect and re-connect.
 +
# Detach (disconnect) when needed with 'Ctrl+a d'
 +
# Ssh back into the login node you picked later. '''Note:''' this is important - ssh into ''the same'' node you started the session on as the screen sessions are ''per-node''.
 +
# Reattach to the screen session(re-attach)
 +
#*<pre>$ ml screen</pre>
 +
#*<pre>$ screen -dr</pre>
 +
 
 +
'''Warning:''' again, you cannot be sure which login node you'll end up on after you ssh into HiPerGator from your local computer. Remember which node you started your screen session on and ssh into it once you are on the HiPerGator. E.g. if you started your session on login3, but end up on login2 after reconnecting you can ssh into login3 internally with <code>$ ssh login3</code>.
 +
 
 +
'''Any login node can be rebooted for maintenance. All screen sessions on the node will be lost on reboot. You will be warned with a node-wide terminal message before that happens if this is a scheduled maintenance as opposed to a node problem. Let our sysadmins know if something critical is going on and ask them to wait if possible.'''
  
 
=Using Screen=
 
=Using Screen=
−
To use GNU Screen as a terminal multiplexer on HiPerGator load the 'screen' environment module with <code>module load screen</code> and start Screen with the <code>screen</code> command. At first glance nothing will happen because the default screen configuration is pretty spartan, but you can copy our example 'fancy' screen configuration with <code>cp $HPC_SCREEN_CONF ~</code> to get a much nicer view and to serve as a starting point for your own customizations. Since the [https://www.gnu.org/software/screen/manual/screen.html GNU Screen Manual] is very comprehensive, but may take effort to navigate I would recommend doing a web search for one of hundreds of screen tutorials. It's a very popular tool, so there is no lack of more accessible documentation. As a minimum the following commands are useful. Note that 'Ctrl+A' means that you hold the CTRL key on the keyboard, then press ''a'' and release both keys before typing the key that corresponds to the command you wan to issue to the screen. The commands with '$' are shell commands, whereas all 'Ctrl+a...' commands are issued inside a screen session.
+
To use GNU Screen as a terminal multiplexer on HiPerGator load the 'screen' environment module with <code>module load screen</code> and start Screen with the <code>screen</code> command. At first glance nothing will happen because the default screen configuration is pretty spartan, but you can copy our example 'fancy' screen configuration with <code>cp $HPC_SCREEN_CONF ~</code> to get a much nicer view and to serve as a starting point for your own customizations. Since the [https://www.gnu.org/software/screen/manual/screen.html GNU Screen Manual] is very comprehensive, but may take effort to navigate I would recommend doing a web search for one of hundreds of screen tutorials. It's a very popular tool, so there is no lack of more accessible documentation. As a minimum the following commands are useful. Note that 'Ctrl+A' means that you hold the CTRL key on the keyboard, then press ''a'' and release both keys before typing the key that corresponds to the command you want to issue to the screen. The commands with '$' are shell commands, whereas all 'Ctrl+a...' commands are issued inside a screen session.
−
 
+
<div style="column-count:2">
 
* Load the module
 
* Load the module
 
  $ module load screen
 
  $ module load screen
 
* Start a '''new''' screen
 
* Start a '''new''' screen
−
$ screen
+
$ screen
 
* Create a new terminal window in a screen session
 
* Create a new terminal window in a screen session
 
  Ctrl+a c
 
  Ctrl+a c
Line 21: Line 43:
 
* Re-attach to the screen session after logging into the same interactive node
 
* Re-attach to the screen session after logging into the same interactive node
 
  $ screen -dr
 
  $ screen -dr
 +
</div>
  
 
=Using tmux=
 
=Using tmux=
 
Tmux is a newer and arguably more 'powerful' approach to terminal multiplexing than GNU Screen, but ultimately, it's a matter of choice. Use tmux if it suits your work style better.
 
Tmux is a newer and arguably more 'powerful' approach to terminal multiplexing than GNU Screen, but ultimately, it's a matter of choice. Use tmux if it suits your work style better.

Latest revision as of 13:36, 2 January 2023

Introduction

SSH sessions can be interrupted by a networking issue. Sometimes it is necessary to have a long-term session on a login node say for an interactive SLURM job, monitoring an analysis, or editing job scripts or configuration files, for example. There are several methods to get a long-term terminal session like starting a terminal within JupyterLab, OnDemand session, or using a terminal multiplexer. We have two such applications on HiPerGator: Screen and Tmux. They are similar implementations of the concept of 'terminal multiplexing' or having multiple terminals in a single instance of the program.

Workflow

In short, working with a terminal multiplexer to have multiple persistent (do not disappear when you log out or your network connection drops) terminals on HPG login nodes usually goes as follows

  1. Ssh into HiPerGator. You end up on a random login node. E.g. 
    • $ ssh myname@hpg.rc.ufl.edu
  2. Pick a login node for your session and ssh into it. E.g. 
    • $ ssh login2
  3. Start a tmux or screen. E.g. 
    • $ ml screen
    • $ screen
  4. Create as many terminal windows in the screen session as needed with 'Ctrl+a c' key combination. See Screen documentation.
  5. Now, if you start an interactive SLURM job in one of the terminals in your screen session that job will still be running if you disconnect and re-connect.
  6. Detach (disconnect) when needed with 'Ctrl+a d'
  7. Ssh back into the login node you picked later. Note: this is important - ssh into the same node you started the session on as the screen sessions are per-node.
  8. Reattach to the screen session(re-attach) 
    • $ ml screen
    • $ screen -dr

Warning: again, you cannot be sure which login node you'll end up on after you ssh into HiPerGator from your local computer. Remember which node you started your screen session on and ssh into it once you are on the HiPerGator. E.g. if you started your session on login3, but end up on login2 after reconnecting you can ssh into login3 internally with $ ssh login3.

Any login node can be rebooted for maintenance. All screen sessions on the node will be lost on reboot. You will be warned with a node-wide terminal message before that happens if this is a scheduled maintenance as opposed to a node problem. Let our sysadmins know if something critical is going on and ask them to wait if possible.

Using Screen

To use GNU Screen as a terminal multiplexer on HiPerGator load the 'screen' environment module with module load screen and start Screen with the screen command. At first glance nothing will happen because the default screen configuration is pretty spartan, but you can copy our example 'fancy' screen configuration with cp $HPC_SCREEN_CONF ~ to get a much nicer view and to serve as a starting point for your own customizations. Since the GNU Screen Manual is very comprehensive, but may take effort to navigate I would recommend doing a web search for one of hundreds of screen tutorials. It's a very popular tool, so there is no lack of more accessible documentation. As a minimum the following commands are useful. Note that 'Ctrl+A' means that you hold the CTRL key on the keyboard, then press a and release both keys before typing the key that corresponds to the command you want to issue to the screen. The commands with '$' are shell commands, whereas all 'Ctrl+a...' commands are issued inside a screen session.

  • Load the module
$ module load screen
  • Start a new screen
$ screen
  • Create a new terminal window in a screen session
Ctrl+a c
  • Switch to numbered window #0 (first window)
Ctrl+a 0
  • Switch to the next window
Ctrl+a n
  • Switch to the previous window
Ctrl+a p
  • Detach from the session now
Ctrl+a d
  • Re-attach to the screen session after logging into the same interactive node
$ screen -dr

Using tmux

Tmux is a newer and arguably more 'powerful' approach to terminal multiplexing than GNU Screen, but ultimately, it's a matter of choice. Use tmux if it suits your work style better.

Retrieved from "https://help.rc.ufl.edu/mediawiki/index.php?title=Persistent_Terminal_Sessions&oldid=27122"