Difference between revisions of "SRA"
Moskalenko (talk | contribs) |
m (Fix formatting) |
||
| (3 intermediate revisions by one other user not shown) | |||
| Line 22: | Line 22: | ||
;Note: sra will create a $HOME/ncib/public directory and cache the prefetched data files there. However, home directory has a 20gb limit and its use for job data storage is a violation of the [https://www.rc.ufl.edu/about/policies/storage/ UFRC storage policy]. You must change that location to a directory in your ufrc space before running the sra toolkit. The official approach is to use the vdb-config tool | ;Note: sra will create a $HOME/ncib/public directory and cache the prefetched data files there. However, home directory has a 20gb limit and its use for job data storage is a violation of the [https://www.rc.ufl.edu/about/policies/storage/ UFRC storage policy]. You must change that location to a directory in your ufrc space before running the sra toolkit. The official approach is to use the vdb-config tool | ||
vdb-config -i | vdb-config -i | ||
| − | and change the directory to, for example, / | + | and change the directory to, for example, /blue/$GROUP/$USER/ncbi/public. See the [https://github.com/ncbi/sra-tools/wiki/Toolkit-Configuration SRA Toolkit Configuration Documentation] for more details. |
| − | Alternatively, create an 'ncbi' directory in your / | + | Alternatively, create an 'ncbi' directory in your /blue space and symlink it to ~/ncbi. E.g. |
| − | $ mkdir / | + | $ mkdir /blue/mygroup/$USER/ncbi |
| − | $ ln -s / | + | $ ln -s /blue/mygroup/$USER/ncbi ~/ncbi |
| + | |||
| + | ==Uploads== | ||
| + | It appears that data uploads to NCBI only work from login servers. Start a screen session before beginning an upload if there are any concerns about being disconnected. | ||
<!--Modules--> | <!--Modules--> | ||
==Required Modules== | ==Required Modules== | ||
| Line 34: | Line 37: | ||
*{{#var:app}} | *{{#var:app}} | ||
==System Variables== | ==System Variables== | ||
| − | * HPC_{{ | + | * HPC_{{uc:{{#var:app}}}}_DIR - installation directory |
* HPC_SRA_BIN - location of the executables directory | * HPC_SRA_BIN - location of the executables directory | ||
* HPC_SRA_DOC - location of the documentation directory | * HPC_SRA_DOC - location of the documentation directory | ||
Revision as of 19:05, 6 January 2022
Description
This is the NCBI Short Read Archive Toolkit.
- Note
- sra will create a $HOME/ncib/public directory and cache the prefetched data files there. However, home directory has a 20gb limit and its use for job data storage is a violation of the UFRC storage policy. You must change that location to a directory in your ufrc space before running the sra toolkit. The official approach is to use the vdb-config tool
vdb-config -i
and change the directory to, for example, /blue/$GROUP/$USER/ncbi/public. See the SRA Toolkit Configuration Documentation for more details.
Alternatively, create an 'ncbi' directory in your /blue space and symlink it to ~/ncbi. E.g.
$ mkdir /blue/mygroup/$USER/ncbi $ ln -s /blue/mygroup/$USER/ncbi ~/ncbi
Uploads
It appears that data uploads to NCBI only work from login servers. Start a screen session before beginning an upload if there are any concerns about being disconnected.
Required Modules
Serial
- sra
System Variables
- HPC_SRA_DIR - installation directory
- HPC_SRA_BIN - location of the executables directory
- HPC_SRA_DOC - location of the documentation directory
Additional Information
Aspera Connect
To download SRA data you can use the "ascp" utility from the Aspera Connect browser plugin package. We have a copy installed and provided by the sra module. A wrapper script ascp.sh that automatically uses the ssh key is available. For instance:
ascp.sh -QT anonftp@ftp-private.ncbi.nlm.nih.gov:/genomes/Bacteria/all.faa.tar.gz faa
will download the all.faa.tar.gz archive to the faa directory.
Note: if the download fails to run with a "Session Stop (Error: Client unable to connect to server (check UDP port and firewall))" error please submit a support request. This means that the remote site has not been allowed through the firewall. Please be sure to include the path to a script you used to run the data transfer command into the request. Do not put any sensitive information like passwords, keys, and such into the request.