Difference between revisions of "TensorFlow"

From UFRC
Jump to navigation Jump to search
 
(10 intermediate revisions by 5 users not shown)
Line 1: Line 1:
[[Category:Software]][[Category:Library]]
+
[[Category:Software]][[Category:Library]][[Category:Math]]
 
{|<!--CONFIGURATION: REQUIRED-->
 
{|<!--CONFIGURATION: REQUIRED-->
 
|{{#vardefine:app|tensorflow}}
 
|{{#vardefine:app|tensorflow}}
|{{#vardefine:url|https://github.com/tensorflow/tensorflow}}
+
|{{#vardefine:url|https://www.tensorflow.org/}}
 
<!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
 
<!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
 
|{{#vardefine:conf|}}          <!--CONFIGURATION-->
 
|{{#vardefine:conf|}}          <!--CONFIGURATION-->
 
|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
 
|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
|{{#vardefine:job|}}            <!--JOB SCRIPTS-->
+
|{{#vardefine:job|1}}            <!--JOB SCRIPTS-->
 
|{{#vardefine:policy|}}        <!--POLICY-->
 
|{{#vardefine:policy|}}        <!--POLICY-->
 
|{{#vardefine:testing|}}      <!--PROFILING-->
 
|{{#vardefine:testing|}}      <!--PROFILING-->
Line 18: Line 18:
 
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}}
 
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}}
  
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes TensorBoard, a data visualization toolkit.
+
TensorFlow is an open-source software library that is commonly used for implementing artificial neural networks and for deep learning. TensorFlow's numerical computation model is based on data flow graphs. Nodes in the graph represent mathematical operations and the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes [[Tensorboard]], a data visualization toolkit, and [https://keras.io/ Keras], a high-level (and easier-to-use) neural networks API for Python.
  
 
<!--Modules-->
 
<!--Modules-->
==Required Modules==
+
==Environment Modules==
Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for this application.
+
To use TensorFlow on HiPerGator, you first need to load one of the TensorFlow [[Modules_Basic_Usage|environment modules]].
==System Variables==
+
Run <code>module spider {{#var:app}}</code> to find out which environment modules are available for TensorFlow.
* HPC_{{#uppercase:{{#var:app}}}}_DIR - installation directory
+
 
 +
Once you determine which module you want to use, load it as follows (using <code>tensorflow/2.7</code> as an example):
 +
<nowiki>
 +
module load tensorflow/2.7</nowiki>
 +
 
 
<!--Configuration-->
 
<!--Configuration-->
 
{{#if: {{#var: conf}}|==Configuration==
 
{{#if: {{#var: conf}}|==Configuration==
Line 31: Line 35:
 
<!--Run-->
 
<!--Run-->
 
{{#if: {{#var: exe}}|==Additional Information==
 
{{#if: {{#var: exe}}|==Additional Information==
 +
As of version 2.0, Keras is packaged with TensorFlow as the <code>tensorflow.keras</code> module. This is the module that you should use. Previously, Keras was developed and distributed separately from TensorFlow; see [https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/ keras vs. tf.keras] for details.
  
TensorFlow is built from source and optimized for GPGPUs we have. There are two modules. The default module e.g. 'tensorflow/1.7.0' is python3 based. There is a separate py2 moduel e.g. 'tensorflow/1.7.0py2' for legacy scripts.
+
To use TensorFlow with a GPU or GPUs on HiPerGator, you must request the <code>--gpus</code> or <code>--gpus-per-task</code> resource and specify the <code>gpu</code> partition in your job script or on the command line as described in the [[GPU_Access|GPU Access]] help page. For example, to start an interactive session with access to a single GPU, you might run the following command.
  $ python3
+
  <nowiki>
  >>> import tensorflow as tf
+
  srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i</nowiki>
 
 
A number of useful python modules have been installed within the tensorflow python virtual environment on request e.g. keras.
 
  
$ python3 -c 'import keras'
 
 
|}}
 
|}}
 
<!--Job Scripts-->
 
<!--Job Scripts-->
 
{{#if: {{#var: job}}|==Job Script Examples==
 
{{#if: {{#var: job}}|==Job Script Examples==
See the [[{{PAGENAME}}_Job_Scripts]] page for {{#var: app}} Job script examples.
+
To help you get started, here is an example SLURM script for running a Python TensorFlow application on a single GPU on HiPerGator.  If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on [[Getting_Started#Scheduling_jobs_using_SLURM|using SLURM and writing SLURM scripts]]. For information about using GPUs on HiPerGator, please see [[GPU Access]].
 +
 
 +
Note that lines beginning with <code>#SBATCH</code> are instructions to the SLURM scheduler.  Lines beginning with <code># </code> are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.
 +
<div class="mw-collapsible mw-collapsed" style="width:70%; padding: 5px; border: 1px solid gray;">
 +
''Expand this section to view example.''
 +
<div class="mw-collapsible-content" style="padding: 5px;">
 +
<nowiki>
 +
#!/bin/sh
 +
# The job name: you can choose whatever you want for this.
 +
#SBATCH --job-name=my_tensorflow_job
 +
 
 +
# Your email address and the events for which you want to receive email
 +
# notification (NONE, BEGIN, END, FAIL, ALL).
 +
#SBATCH --mail-user=username@ufl.edu
 +
#SBATCH --mail-type=ALL
 +
 
 +
# The compute configuration for the job. For a job that uses GPUs, the
 +
# partition must be set to "gpu". This example script requests access
 +
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
 +
#SBATCH --nodes=1
 +
#SBATCH --partition=gpu
 +
#SBATCH --ntasks=1
 +
#SBATCH --gpus-per-task=1
 +
#SBATCH --cpus-per-task=16
 +
#SBATCH --mem=30gb
 +
 
 +
# Specifies how long the job will be allowed to run in HH:MM:SS.
 +
#SBATCH --time=05:05:05
 +
 
 +
# The log file for all job output. Note the special string "%j", which
 +
# represents the job number.
 +
#SBATCH --output=job_output_%j.out
 +
 
 +
# Prints the working directory, name of the assigned node, and
 +
# date/time at the top of the output log.
 +
pwd; hostname; date
 +
 
 +
module load tensorflow/2.7
 +
 
 +
# This should be the command you would use if you were running your TensorFlow application from the terminal.
 +
python my_tensorflow_script.py
 +
 
 +
date</nowiki>
 +
</div>
 +
</div>
 
|}}
 
|}}
 
<!--Policy-->
 
<!--Policy-->
Line 61: Line 107:
 
<!--Citation-->
 
<!--Citation-->
 
{{#if: {{#var: citation}}|==Citation==
 
{{#if: {{#var: citation}}|==Citation==
If you publish research that uses {{#var:app}} you have to cite it as follows:
+
If you publish research that uses {{#var:app}} you must cite it as follows:
  
 
https://www.tensorflow.org/versions/r0.11/resources/bib.html
 
https://www.tensorflow.org/versions/r0.11/resources/bib.html

Latest revision as of 15:25, 14 December 2022

Description

tensorflow website  

TensorFlow is an open-source software library that is commonly used for implementing artificial neural networks and for deep learning. TensorFlow's numerical computation model is based on data flow graphs. Nodes in the graph represent mathematical operations and the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes Tensorboard, a data visualization toolkit, and Keras, a high-level (and easier-to-use) neural networks API for Python.

Environment Modules

To use TensorFlow on HiPerGator, you first need to load one of the TensorFlow environment modules. Run module spider tensorflow to find out which environment modules are available for TensorFlow.

Once you determine which module you want to use, load it as follows (using tensorflow/2.7 as an example):

module load tensorflow/2.7


Additional Information

As of version 2.0, Keras is packaged with TensorFlow as the tensorflow.keras module. This is the module that you should use. Previously, Keras was developed and distributed separately from TensorFlow; see keras vs. tf.keras for details.

To use TensorFlow with a GPU or GPUs on HiPerGator, you must request the --gpus or --gpus-per-task resource and specify the gpu partition in your job script or on the command line as described in the GPU Access help page. For example, to start an interactive session with access to a single GPU, you might run the following command.

 srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i

Job Script Examples

To help you get started, here is an example SLURM script for running a Python TensorFlow application on a single GPU on HiPerGator. If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on using SLURM and writing SLURM scripts. For information about using GPUs on HiPerGator, please see GPU Access.

Note that lines beginning with #SBATCH are instructions to the SLURM scheduler. Lines beginning with # are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.

Expand this section to view example.

#!/bin/sh
# The job name: you can choose whatever you want for this.
#SBATCH --job-name=my_tensorflow_job

# Your email address and the events for which you want to receive email
# notification (NONE, BEGIN, END, FAIL, ALL).
#SBATCH --mail-user=username@ufl.edu
#SBATCH --mail-type=ALL

# The compute configuration for the job. For a job that uses GPUs, the
# partition must be set to "gpu". This example script requests access
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --ntasks=1
#SBATCH --gpus-per-task=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=30gb

# Specifies how long the job will be allowed to run in HH:MM:SS.
#SBATCH --time=05:05:05

# The log file for all job output. Note the special string "%j", which
# represents the job number.
#SBATCH --output=job_output_%j.out

# Prints the working directory, name of the assigned node, and
# date/time at the top of the output log.
pwd; hostname; date

module load tensorflow/2.7

# This should be the command you would use if you were running your TensorFlow application from the terminal.
python my_tensorflow_script.py

date


Citation

If you publish research that uses tensorflow you must cite it as follows:

https://www.tensorflow.org/versions/r0.11/resources/bib.html