Difference between revisions of "QIIME"
Moskalenko (talk | contribs) m (Text replace - "==Running the application using modules==" to "==Execution Environment and Modules==") |
|||
| (15 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | [[Category:Software]][[Category: | + | [[Category:Software]][[Category:Biology]][[Category:genomics]] |
{|<!--CONFIGURATION: REQUIRED--> | {|<!--CONFIGURATION: REQUIRED--> | ||
|{{#vardefine:app|qiime}} | |{{#vardefine:app|qiime}} | ||
| Line 7: | Line 7: | ||
|{{#vardefine:intel|}} <!-- "11.1" - Compiler Module Choice --> | |{{#vardefine:intel|}} <!-- "11.1" - Compiler Module Choice --> | ||
|{{#vardefine:mpi|}} <!-- "openmpi/1.3.4"- MPI Module Choice --> | |{{#vardefine:mpi|}} <!-- "openmpi/1.3.4"- MPI Module Choice --> | ||
| − | |{{#vardefine:exe|}} <!--RUNNING --> | + | |{{#vardefine:exe|1}} <!--RUNNING --> |
|{{#vardefine:conf|}} <!--CONFIGS--> | |{{#vardefine:conf|}} <!--CONFIGS--> | ||
|{{#vardefine:pbs|}} <!--PBS SCRIPTS--> | |{{#vardefine:pbs|}} <!--PBS SCRIPTS--> | ||
| Line 13: | Line 13: | ||
|{{#vardefine:testing|}} <!--TESTING/PROFILING--> | |{{#vardefine:testing|}} <!--TESTING/PROFILING--> | ||
|{{#vardefine:faq|}} <!--FAQ--> | |{{#vardefine:faq|}} <!--FAQ--> | ||
| − | |{{#vardefine:citation|}} <!--CITATION--> | + | |{{#vardefine:citation|1}} <!--CITATION--> |
|{{#vardefine:installation|}} <!--INSTALLATION--> | |{{#vardefine:installation|}} <!--INSTALLATION--> | ||
|} | |} | ||
| Line 23: | Line 23: | ||
QIIME (pronounced "chime") stands for Quantitative Insights Into Microbial Ecology. QIIME is an open source software package for comparison and analysis of microbial communities, primarily based on high-throughput amplicon sequencing data (such as SSU rRNA) generated on a variety of platforms, but also supporting analysis of other types of data (such as shotgun metagenomic data). QIIME takes users from their raw sequencing output through initial analyses such as OTU picking, taxonomic assignment, and construction of phylogenetic trees from representative sequences of OTUs, and through downstream statistical analysis, visualization, and production of publication-quality graphics. QIIME has been applied to single studies based on billions of sequences from thousands of samples. | QIIME (pronounced "chime") stands for Quantitative Insights Into Microbial Ecology. QIIME is an open source software package for comparison and analysis of microbial communities, primarily based on high-throughput amplicon sequencing data (such as SSU rRNA) generated on a variety of platforms, but also supporting analysis of other types of data (such as shotgun metagenomic data). QIIME takes users from their raw sequencing output through initial analyses such as OTU picking, taxonomic assignment, and construction of phylogenetic trees from representative sequences of OTUs, and through downstream statistical analysis, visualization, and production of publication-quality graphics. QIIME has been applied to single studies based on billions of sequences from thousands of samples. | ||
| − | |||
| − | |||
| − | |||
<!--Modules--> | <!--Modules--> | ||
| − | + | ==Environment Modules== | |
| − | + | Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for this application. | |
| − | + | ==System Variables== | |
| + | * HPC_{{uc:{{#var:app}}}}_DIR - installation directory | ||
<!--Run--> | <!--Run--> | ||
| + | |||
{{#if: {{#var: exe}}|==How To Run== | {{#if: {{#var: exe}}|==How To Run== | ||
| − | + | ||
| + | * TMPDIR | ||
| + | QIIME will use /tmp by default, which will fill up memory disks on HPG2 nodes and cause node and job failures. Our qiime modules will automatically create a 'tmp' directory inside the current working directory and set the variable TMPDIR to point to it. The tmp directory will be removed on the qiime module unload. | ||
| + | |||
| + | * Tasks vs Cores for parallel runs | ||
| + | Python threads in a parallel QIIME job will be bound to the same CPU core even if multiple ntasks are specified in the job script. Use cpus-per-task to parallelize QIIME jobs correctly. For example, for an 8-thread parallel QIIME job use the following resource request in your job script: | ||
| + | #SBATCH --ntasks=1 | ||
| + | #SBATCH --cpus-per-task=8 | ||
| + | |||
| + | See the single-threaded and multi-threaded examples on the [[Sample SLURM Scripts]] page for more details. | ||
|}} | |}} | ||
<!--Configuration--> | <!--Configuration--> | ||
| Line 42: | Line 50: | ||
See the [[{{PAGENAME}}_PBS]] page for {{#var: app}} PBS script examples.|}} | See the [[{{PAGENAME}}_PBS]] page for {{#var: app}} PBS script examples.|}} | ||
<!--Policy--> | <!--Policy--> | ||
| − | {{#if: {{#var: policy}}|==Usage | + | {{#if: {{#var: policy}}|==Usage Policy== |
WRITE USAGE POLICY HERE (perhaps templates for a couple of main licensing schemes can be used) | WRITE USAGE POLICY HERE (perhaps templates for a couple of main licensing schemes can be used) | ||
|}} | |}} | ||
| Line 55: | Line 63: | ||
{{#if: {{#var: citation}}|==Citation== | {{#if: {{#var: citation}}|==Citation== | ||
If you publish research that uses {{#var:app}} you have to cite it as follows: | If you publish research that uses {{#var:app}} you have to cite it as follows: | ||
| − | + | ||
| + | J Gregory Caporaso, Justin Kuczynski, Jesse Stombaugh, Kyle Bittinger, Frederic D Bushman, Elizabeth K Costello, Noah Fierer, Antonio Gonzalez Pena, Julia K Goodrich, Jeffrey I Gordon, Gavin A Huttley, Scott T Kelley, Dan Knights, Jeremy E Koenig, Ruth E Ley, Catherine A Lozupone, Daniel McDonald, Brian D Muegge, Meg Pirrung, Jens Reeder, Joel R Sevinsky, Peter J Turnbaugh, William A Walters, Jeremy Widmann, Tanya Yatsunenko, Jesse Zaneveld and Rob Knight; Nature Methods, 2010; doi:10.1038/nmeth.f.303 | ||
|}} | |}} | ||
<!--Installation--> | <!--Installation--> | ||
Latest revision as of 15:15, 19 August 2022
Description
QIIME (pronounced "chime") stands for Quantitative Insights Into Microbial Ecology. QIIME is an open source software package for comparison and analysis of microbial communities, primarily based on high-throughput amplicon sequencing data (such as SSU rRNA) generated on a variety of platforms, but also supporting analysis of other types of data (such as shotgun metagenomic data). QIIME takes users from their raw sequencing output through initial analyses such as OTU picking, taxonomic assignment, and construction of phylogenetic trees from representative sequences of OTUs, and through downstream statistical analysis, visualization, and production of publication-quality graphics. QIIME has been applied to single studies based on billions of sequences from thousands of samples.
Environment Modules
Run module spider qiime to find out what environment modules are available for this application.
System Variables
- HPC_QIIME_DIR - installation directory
How To Run
- TMPDIR
QIIME will use /tmp by default, which will fill up memory disks on HPG2 nodes and cause node and job failures. Our qiime modules will automatically create a 'tmp' directory inside the current working directory and set the variable TMPDIR to point to it. The tmp directory will be removed on the qiime module unload.
- Tasks vs Cores for parallel runs
Python threads in a parallel QIIME job will be bound to the same CPU core even if multiple ntasks are specified in the job script. Use cpus-per-task to parallelize QIIME jobs correctly. For example, for an 8-thread parallel QIIME job use the following resource request in your job script:
#SBATCH --ntasks=1 #SBATCH --cpus-per-task=8
See the single-threaded and multi-threaded examples on the Sample SLURM Scripts page for more details.
Citation
If you publish research that uses qiime you have to cite it as follows:
J Gregory Caporaso, Justin Kuczynski, Jesse Stombaugh, Kyle Bittinger, Frederic D Bushman, Elizabeth K Costello, Noah Fierer, Antonio Gonzalez Pena, Julia K Goodrich, Jeffrey I Gordon, Gavin A Huttley, Scott T Kelley, Dan Knights, Jeremy E Koenig, Ruth E Ley, Catherine A Lozupone, Daniel McDonald, Brian D Muegge, Meg Pirrung, Jens Reeder, Joel R Sevinsky, Peter J Turnbaugh, William A Walters, Jeremy Widmann, Tanya Yatsunenko, Jesse Zaneveld and Rob Knight; Nature Methods, 2010; doi:10.1038/nmeth.f.303