Difference between revisions of "SRA"
Moskalenko (talk | contribs) m (Text replacement - "#uppercase" to "uc") |
Moskalenko (talk | contribs) |
||
| (7 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
__NOEDITSECTION__ | __NOEDITSECTION__ | ||
| − | [[Category:Software]] | + | [[Category:Software]][[Category:Biology]][[Category:NGS]] |
{|<!--Main settings - REQUIRED--> | {|<!--Main settings - REQUIRED--> | ||
|{{#vardefine:app|sra}} | |{{#vardefine:app|sra}} | ||
| Line 20: | Line 20: | ||
This is the NCBI Short Read Archive Toolkit. | This is the NCBI Short Read Archive Toolkit. | ||
| − | ;Note: sra will create a $HOME/ | + | ;Note: sra will create a $HOME/ncbi/public directory and cache the prefetched data files there. However, home directory has a 40gb limit and its use for job data storage is a violation of the [https://www.rc.ufl.edu/about/policies/storage/ UFRC storage policy]. You must change that location to a directory in your ufrc space before running the sra toolkit. The official approach is to use the vdb-config tool |
vdb-config -i | vdb-config -i | ||
| − | and change the directory to, for example, / | + | and change the directory to, for example, /blue/$GROUP/$USER/ncbi/public. See the [https://github.com/ncbi/sra-tools/wiki/Toolkit-Configuration SRA Toolkit Configuration Documentation] for more details. |
| − | Alternatively, create an 'ncbi' directory in your / | + | Alternatively, create an 'ncbi' directory in your /blue space and symlink it to ~/ncbi. E.g. |
| − | $ mkdir / | + | $ mkdir /blue/mygroup/$USER/ncbi |
| − | $ ln -s / | + | $ ln -s /blue/mygroup/$USER/ncbi ~/ncbi |
| + | |||
| + | ==Uploads== | ||
| + | It appears that data uploads to NCBI only work from login servers. Start a screen session before beginning an upload if there are any concerns about being disconnected. | ||
<!--Modules--> | <!--Modules--> | ||
| − | == | + | ==Environment Modules== |
| − | + | Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for this application. | |
| − | |||
| − | |||
==System Variables== | ==System Variables== | ||
* HPC_{{uc:{{#var:app}}}}_DIR - installation directory | * HPC_{{uc:{{#var:app}}}}_DIR - installation directory | ||
| Line 45: | Line 46: | ||
'''Note:''' if the download fails to run with a "Session Stop (Error: Client unable to connect to server (check UDP port and firewall))" | '''Note:''' if the download fails to run with a "Session Stop (Error: Client unable to connect to server (check UDP port and firewall))" | ||
| − | error please submit a support request. This means that the remote site has not been allowed through the firewall. Please be sure to include the path to a script you used to run the data transfer command into the request. Do not put any sensitive information like passwords, keys, and such into the request. | + | error ''make sure you are running aspera connect on a login node''. If you are already doing the transfer on a login node please submit a support request. This means that the remote site has not been allowed through the firewall. Please be sure to include the path to a script you used to run the data transfer command into the request. Do not put any sensitive information like passwords, keys, and such into the request. |
|}} | |}} | ||
{{#if: {{#var: conf}}|==Configuration== | {{#if: {{#var: conf}}|==Configuration== | ||
Latest revision as of 19:10, 27 November 2023
Description
This is the NCBI Short Read Archive Toolkit.
- Note
- sra will create a $HOME/ncbi/public directory and cache the prefetched data files there. However, home directory has a 40gb limit and its use for job data storage is a violation of the UFRC storage policy. You must change that location to a directory in your ufrc space before running the sra toolkit. The official approach is to use the vdb-config tool
vdb-config -i
and change the directory to, for example, /blue/$GROUP/$USER/ncbi/public. See the SRA Toolkit Configuration Documentation for more details.
Alternatively, create an 'ncbi' directory in your /blue space and symlink it to ~/ncbi. E.g.
$ mkdir /blue/mygroup/$USER/ncbi $ ln -s /blue/mygroup/$USER/ncbi ~/ncbi
Uploads
It appears that data uploads to NCBI only work from login servers. Start a screen session before beginning an upload if there are any concerns about being disconnected.
Environment Modules
Run module spider sra to find out what environment modules are available for this application.
System Variables
- HPC_SRA_DIR - installation directory
- HPC_SRA_BIN - location of the executables directory
- HPC_SRA_DOC - location of the documentation directory
Additional Information
Aspera Connect
To download SRA data you can use the "ascp" utility from the Aspera Connect browser plugin package. We have a copy installed and provided by the sra module. A wrapper script ascp.sh that automatically uses the ssh key is available. For instance:
ascp.sh -QT anonftp@ftp-private.ncbi.nlm.nih.gov:/genomes/Bacteria/all.faa.tar.gz faa
will download the all.faa.tar.gz archive to the faa directory.
Note: if the download fails to run with a "Session Stop (Error: Client unable to connect to server (check UDP port and firewall))" error make sure you are running aspera connect on a login node. If you are already doing the transfer on a login node please submit a support request. This means that the remote site has not been allowed through the firewall. Please be sure to include the path to a script you used to run the data transfer command into the request. Do not put any sensitive information like passwords, keys, and such into the request.