TensorFlow

Description

TensorFlow is an open-source software library that is commonly used for implementing artificial neural networks and for deep learning. TensorFlow's numerical computation model is based on data flow graphs. Nodes in the graph represent mathematical operations and the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes Tensorboard, a data visualization toolkit, and Keras, a high-level (and easier-to-use) neural networks API for Python.

Environment Modules

To use TensorFlow on HiPerGator, you first need to load one of the TensorFlow environment modules. Run module spider tensorflow to find out which environment modules are available for TensorFlow.

Once you determine which module you want to use, load it as follows (using tensorflow/2.7 as an example):

module load tensorflow/2.7

Additional Information

As of version 2.0, Keras is packaged with TensorFlow as the tensorflow.keras module. This is the module that you should use. Previously, Keras was developed and distributed separately from TensorFlow; see keras vs. tf.keras for details.

To use TensorFlow with a GPU or GPUs on HiPerGator, you must request the --gpus or --gpus-per-task resource and specify the gpu partition in your job script or on the command line as described in the GPU Access help page. For example, to start an interactive session with access to a single GPU, you might run the following command.

 srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i

Job Script Examples

To help you get started, here is an example SLURM script for running a Python TensorFlow application on a single GPU on HiPerGator. If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on using SLURM and writing SLURM scripts. For information about using GPUs on HiPerGator, please see GPU Access.

Note that lines beginning with #SBATCH are instructions to the SLURM scheduler. Lines beginning with # are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.

Expand this section to view example.

#!/bin/sh
# The job name: you can choose whatever you want for this.
#SBATCH --job-name=my_tensorflow_job

# Your email address and the events for which you want to receive email
# notification (NONE, BEGIN, END, FAIL, ALL).
#SBATCH --mail-user=username@ufl.edu
#SBATCH --mail-type=ALL

# The compute configuration for the job. For a job that uses GPUs, the
# partition must be set to "gpu". This example script requests access
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --ntasks=1
#SBATCH --gpus-per-task=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=30gb

# Specifies how long the job will be allowed to run in HH:MM:SS.
#SBATCH --time=05:05:05

# The log file for all job output. Note the special string "%j", which
# represents the job number.
#SBATCH --output=job_output_%j.out

# Prints the working directory, name of the assigned node, and
# date/time at the top of the output log.
pwd; hostname; date

module load tensorflow/2.7

# This should be the command you would use if you were running your TensorFlow application from the terminal.
python my_tensorflow_script.py

date

Citation

If you publish research that uses tensorflow you must cite it as follows:

https://www.tensorflow.org/versions/r0.11/resources/bib.html