Transfer Data

Overview

This document describes transferring data between a local computer (client) and HiPerGator (HPG). For file sharing on the cluster see Sharing Files

Tools

There are a variety of command-line, GUI, and even web-based tools available for transferring data to or from HiPerGator. Command-line tools include, for example, 'cp, mv, scp, rsync, sftp, wget, curl, or ncftp (from the ncftp environment module]'. GUI tools like Cyberduck, WinSCP, BitVise SFTP and similar tools can be used on your local computer. Globus data transfer tool is available via a web interface in addition to command-line and GUI versions that can be run on your local computer. Please visit the HPG how-to video series on Data Transfer for more details.

Transferring Data within HiPerGator

Login servers on HiPergator (hpg.rc.ufl.edu) can be used for rsync, copying files with cp or mv depending on the permissions or filesystem ACLs (extended permissions).

Between a local computer and HiPerGator

Globus

If your data files are large (hundreds of megabytes or gigabytes) then try Globus first.

SFTP/Rsync

For smaller file sizes, or if Globus is not an option, use SFTP/rsync/scp by connecting to the 'hpg.rc.ufl.edu' or 'sftp.rc.ufl.edu'. Set port to 22 if you have to specify it. Use your gatorlink credentials to connect. Make sure MFA (multi-factor authentication) is taken into account to avoid having to go through MFA for every file transferred. You can also use terminal interfaces like Tabby and MobaXterm, which include an GUI SFTP function embedded.

Note

If you are using Cyberduck on a MacOS computer and notice that it is using the old password resulting in a security lockout uncheck 'Use Keychain' in 'Preferences > General'.

JupyterHub

The easiest way to upload data using JypyterHub is by using the Upload Files button included in the File Browser (Ctrl+Shift+F) menu.

From HiPerGator to a remote system/site

If you are logged into HiPerGator and need to transfer data to or from a remote system or a site use the login nodes. Transfers can also be made from within developmental sessions. You can use ftp (lftp command), sftp, scp, rsync, or Globus to transfer data out.

Providing access to your data

If you need to share data with another RC user or publicly for collaboration, view instructions at Providing Access To Data