Galaxy Data Import

Introduction

Small files residing on your local machine can be easily added to a Galaxy history through the "Get Data>Upload File" tool.

However, larger files may take too long to upload directly into galaxy via web browser especially if you are using a slow or unstable network connection. Those files need to be transferred to the UFRC and put under '/data/apps/galaxy/incoming/your@email' directory.

Get Data > Upload File' tool shows all files under /data/apps/galaxy/incoming/your@email listed in the upload interface. Simply click on the checkboxes in front of the files you want to upload and then click on the 'Execute' button and the upload should commence. Once you uploaded the files please remove them from /data/apps/galaxy/incoming/your@email. The above approach should be much simpler and faster than using the Large Data Upload procedure through the Incoming data library and doesn't require any changes. Please give the system at least 15 minutes to fix permissions on the newly uploaded files if they are not readable by the galaxy user.

---

Note: The "Incoming" library must only be used to upload data, not for storage. Please delete the uploaded datasets once they've been imported into histories. All datasets and folders in the "Incoming" shared data library may be deleted after a period of 30 days if they weren't cleaned up by the owner.

Detailed Procedure

• Find out your Galaxy username. Click on the "User" menu and read the "Logged in as user@hostname" line.
• Create your Galaxy incoming directory

If your Galaxy username is "jdoe@ufl.edu" then the incoming directory you need to create is /data/apps/galaxy/incoming/jdoe@ufl.edu

mkdir /data/apps/galaxy/incoming/jdoe@ufl.edu

• Copy data to the Galaxy incoming directory

Note

You can use either command-line or GUI tools like FileZilla to transfer your data to HiPerGator for upload into Galaxy.

• Copy your data directory to your galaxy incoming directory inside the /data/apps/galaxy/incoming directory.

For instance, if your Galaxy username is "jdoe@ufl.edu" then your data directory must be copied to /data/apps/galaxy/incoming/jdoe@ufl.edu, so in the end the data files will be inside the

/data/apps/galaxy/incoming/jdoe@ufl.edu/data_dir

directory. Substitute your actual Galaxy username and the name of the directory you copied for "jdoe@ufl.edu" and "data_dir".

• Make sure the files are readable by setting the correct access mode

Set the 755 mode for the directory and 644 for the files either from within the software you used for transferring the files or by logging into HPC and running the following commands:

cd /data/apps/galaxy/incoming/jdoe@ufl.edu
chmod 755 data_dir
chmod 644 data_dir/*

Notes

• You can use scp/sftp or Globus as described in the Transfer Data help page to copy your data from a local system to your incoming galaxy directory.
• The username is your email address. If you don't remember what email address you used for HiPerGator account registration it can always be looked up by clicking on the User item of the Galaxy's main menu.
• Your data files should be in a single sub-directory. For instance, if there is a directory called test that you are transferring to HiPerGator then the full path to the directory with data files of a user jdoe@ufl.edu will be /data/apps/galaxy/incoming/jdoe@ufl.edu/test