Rclone

Description

rclone website  

Rclone is a command line program to manage files on cloud storage.

Environment Modules

Run module spider rclone to find out what environment modules are available for this application.

System Variables

• HPC_RCLONE_DIR - installation directory

Examples

Add DropBox Remote

Expand to see an example to add DropBox Remote.

I've verified that the https://rclone.org/dropbox/ and https://rclone.org/drive/ procedures worked on HiPerGator. Here's dropbox remote setup:

1. Start an Open OnDemand Desktop session: https://ood.rc.ufl.edu/pun/sys/dashboard/batch_connect/sys/bc_desktop/hipergator/session_contexts/new
2. Once the session starts connect to it and open a terminal by clicking on its icon.
3. Use 'New > Open Tab' menu option or 'Ctrl+Shift+T' key combo in the terminal to make sure you have two tabs to start a web browser
   • Tab1:

     • module load rclone

   • Tab2:

     • module load ubuntu firefox

4. Follow the https://rclone.org/dropbox/ procedure. Note that the ubuntu module has chrome and firefox browsers, so the web authentication part of defining a remote can be done by pasting the generated link into it.

Tab1:

$ rclone config
Current remotes:

Name                 Type
====                 ====
gdrive               drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> dbox
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
 1 / 1Fichier
   \ (fichier)
...
12 / Dropbox
   \ (dropbox)
...
46 / seafile
   \ (seafile)
Storage> dropbox
Option client_id.
OAuth Client Id.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_id> 
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret> 
Edit advanced config?
y) Yes
n) No (default)
y/n> n
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine

y) Yes (default)
n) No
y/n> y
2022/09/28 15:29:19 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=2B9dREDACTED_gzfrZg
2022/09/28 15:29:19 NOTICE: Log in and authorize rclone for access

<At this point copy the link above and paste it into 'firefox' URL bar. Log in and authorize the connection. Return to Tab1>

2022/09/28 15:29:19 NOTICE: Waiting for code...
2022/09/28 15:30:47 NOTICE: Got code
--------------------
[dbox]
type = dropbox
token = {"access_token":"sl.BQJDjte4UkfWREDACTEDyRcXqyi-oAcmScjT_-mn3b81BAc-pLgPcX5PtDRRREDACTEDwIaulrxz5ncFYrLQyummJpG5","token_type":"bearer","refresh_token":"93j-33PYjZgAAAAAAAREDACTED_NwenaxE3Y5VBbLD93g6r","expiry":"2022-09-28T19:30:48.69886106-04:00"}
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name                 Type
====                 ====
dbox                 dropbox
gdrive               drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config

Test the Dropbox remote:

$ rclone lsd dbox:
          -1 2022-09-28 15:31:03        -1 GCP 2019 Photos
          -1 2022-09-28 15:31:03        -1 SKITE
          -1 2022-09-28 15:31:03        -1 Sent files
          -1 2022-09-28 15:31:03        -1 UFIT-RC-SysSecPlan