Difference between revisions of "NLP"
Jump to navigation
Jump to search
| Line 18: | Line 18: | ||
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}} | {{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}} | ||
| − | + | This page describes the collection of natural language processing software on HiPerGator. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
<!--Modules--> | <!--Modules--> | ||
| − | ==Environment Modules== | + | ==Environment Modules for NLP== |
| − | + | *'''nlp:''' module load nlp will provide a Python environment with spacy, pytorch, torchtext, nltk, sentencepiece, transformers, sentence-transformers, bertopic, RAPIDSai, gensim, scikit-learn, and more. | |
| − | + | *'''nemo:''' module load nemo will provide a singularity container environment with Python and Nvidia NeMo. NeMo has NLP task training, plus speech-to-text and text-to-speech models. | |
| − | * | + | *'''pytorch:''' Note, use module spider pytorch to list the version we have available. Beyond stock pytorch versions, we have the Nvidia pytorch singularity container with the Apex optimizers required for Megatron-LM. Use module load ngc-pytorch to access this container, and you can run Megatron from source code. |
| − | * | ||
<!--Configuration--> | <!--Configuration--> | ||
Revision as of 17:25, 26 July 2021
This page describes the collection of natural language processing software on HiPerGator.
Environment Modules for NLP
- nlp: module load nlp will provide a Python environment with spacy, pytorch, torchtext, nltk, sentencepiece, transformers, sentence-transformers, bertopic, RAPIDSai, gensim, scikit-learn, and more.
- nemo: module load nemo will provide a singularity container environment with Python and Nvidia NeMo. NeMo has NLP task training, plus speech-to-text and text-to-speech models.
- pytorch: Note, use module spider pytorch to list the version we have available. Beyond stock pytorch versions, we have the Nvidia pytorch singularity container with the Apex optimizers required for Megatron-LM. Use module load ngc-pytorch to access this container, and you can run Megatron from source code.