NLP

Description

This page describes the collection of natural language processing software on HiPerGator. Research computing can provide help with language modeling for conversational AI, measurement, or classification tasks via support requests or consultations. NVIDIA Megatron and NeMo is open-source software using transformer neural networks that we suggest for applications and research. Example notebooks are in /data/ai/examples/text.

Environment Modules for NLP

• nlp: module load nlp will provide a Python environment with pytorch, torchtext, nltk, spacy, sentencepiece for vocabulary training, transformers, sentence-transformers, Flair, BERTopic for topic modeling, RAPIDSai for data processing and machine learning algorithms, gensim, scikit-learn, and more.

• nemo: module load nemo will provide a singularity container environment with Python and Nvidia NeMo. NeMo has NLP task training, plus speech-to-text and text-to-speech models.

• pytorch: Note, use module spider pytorch to list the version we have available. Beyond stock pytorch versions, we have the Nvidia pytorch singularity container with the Apex optimizers required for Megatron-LM. Use module load ngc-pytorch to access this container, and you can run Megatron from source code.

• spark-nlp: See our Spark help doc to start a Spark cluster. Spark-nlp Python module is available in tensorflow/2.4.1.

• parlai: Conversational AI framework by Facebook, includes a wide variety of models from 110M to 9B parameters.

Models and Examples

Please see our /data/ai folder for examples, reference data, and pretrained language models.