NLP

Description

This page describes the collection of natural language processing software on HiperGator. NLP is involved in many other fields of AI, such as image recognition. Research computing will help with language modeling for conversational AI, measurement, classification tasks, etc. via support requests or consulting. NVIDIA Megatron and NeMo are open-source software using transformer neural networks that can scale to multiple nodes of GPUs. See the directory /data/ai for more information.

Environment Modules for NLP

• nlp: module load nlp will provide a Python environment with pytorch, torchtext, nltk, spacy, transformers, sentence-transformers, Flair, BERTopic for topic modeling, sentencepiece, RAPIDSai for data processing and machine learning algorithms, gensim, scikit-learn, and more.

• nemo: module load nemo will provide a singularity container environment with Python and Nvidia NeMo. NeMo has NLP task training, plus speech-to-text and text-to-speech models.

• pytorch: Note, use module spider pytorch to list the version we have available. Beyond stock pytorch versions, we have a Nvidia pytorch singularity container with the requirements for Megatron-LM. Use module load ngc-pytorch to access this container, and you can run Megatron from source code.

• spark-nlp: See our Spark help doc to start a Spark cluster. Spark-nlp Python module is available in tensorflow/2.4.1.

• parlai: Conversational AI framework by Facebook, includes a wide variety of models from 110M to 9B parameters.

Examples

Please see our /data/ai folder for examples, reference data, and pretrained language models. Notebooks and batch scripts cover everything from pretraining and inferencing to summarization, information extraction, and topic modeling. Some of the reference data is listed on the reference data page on this help site.