Difference between revisions of "MOOSE Configuration"
| Line 130: | Line 130: | ||
</source> | </source> | ||
| − | ===Update and | + | ===Update and Build ''libmesh''=== |
Run the update and build libmesh script again and let it complete. It should finish without errors. | Run the update and build libmesh script again and let it complete. It should finish without errors. | ||
<source lang=bash> | <source lang=bash> | ||
Revision as of 15:39, 13 August 2020
Configure and test your own "MOOSE" framework
Basic Steps
- mkdir projects
- cd projects
- module load moose/12-aug-20
- git clone https://github.com/idaholab/moose.git
- cd moose
- git checkout master
- export VTKLIB_DIR=${HPC_VTK_LIB} VTKINCLUDE_DIR=${HPC_VTK_INC}
- Run "./scripts/update_and_rebuild_libmesh.sh --enable-vtk-required"
- Interrupt (ctrl-c) the above configure/build just after the git clone of libmesh is completed.
- Patch the "configure" script using the accompanying patch file (see below).
- Rerun "./scripts/update_and_rebuild_libmesh.sh --enable-vtk-required" and allow it to complete. It should finish without errors.
- cd test; run_tests -j 4
- Build and test the "phase field" module:
- cd moose/modules/phase_field
- make -j 4
- ./run_tests -j 4
- Build and test the "combined" module:
- cd moose/modules/combined
- make -j 4
- ./run_tests -j 4
Patch File (configure.patch)
The patch file below can be applied via the patch utility.
cd libmesh
patch -p1 < configure.patch
======================================configure.patch==========================================================
--- a/configure 2020-08-07 07:56:18.955696774 -0400
+++ b/configure 2020-08-07 08:00:45.329361095 -0400
@@ -38452,24 +38452,26 @@
VTK_LIBRARY_WITH_VERSION="-L$VTK_LIB -lvtkIOCore-$vtkmajorminor -lvtkCommonCore-$vtkmajorminor -lvtkCommonDataModel-$vtkmajorminor \
-lvtkFiltersCore-$vtkmajorminor -lvtkIOXML-$vtkmajorminor -lvtkImagingCore-$vtkmajorminor \
-lvtkIOImage-$vtkmajorminor -lvtkImagingMath-$vtkmajorminor \
- -lvtkParallelMPI-$vtkmajorminor -lvtkParallelCore-$vtkmajorminor"
+ -lvtkParallelMPI-$vtkmajorminor -lvtkParallelCore-$vtkmajorminor \
+ -lvtkCommonExecutionModel-$vtkmajorminor"
VTK_LIBRARY_NO_VERSION="-L$VTK_LIB -lvtkIOCore -lvtkCommonCore -lvtkCommonDataModel \
-lvtkFiltersCore -lvtkIOXML -lvtkImagingCore \
-lvtkIOImage -lvtkImagingMath \
- -lvtkParallelMPI -lvtkParallelCore"
+ -lvtkParallelMPI -lvtkParallelCore -lvtkCommonExecutionModel"
else
VTK_LIBRARY_WITH_VERSION="-L$VTK_LIB -lvtkIOCore-$vtkmajorminor -lvtkCommonCore-$vtkmajorminor -lvtkCommonDataModel-$vtkmajorminor \
-lvtkFiltersCore-$vtkmajorminor -lvtkIOXML-$vtkmajorminor -lvtkImagingCore-$vtkmajorminor \
-lvtkIOImage-$vtkmajorminor -lvtkImagingMath-$vtkmajorminor -lvtkIOParallelXML-$vtkmajorminor \
- -lvtkParallelMPI-$vtkmajorminor -lvtkParallelCore-$vtkmajorminor"
+ -lvtkParallelMPI-$vtkmajorminor -lvtkParallelCore-$vtkmajorminor \
+ -lvtkCommonExecutionModel-$vtkmajorminor"
VTK_LIBRARY_NO_VERSION="-L$VTK_LIB -lvtkIOCore -lvtkCommonCore -lvtkCommonDataModel \
-lvtkFiltersCore -lvtkIOXML -lvtkImagingCore \
-lvtkIOImage -lvtkImagingMath -lvtkIOParallelXML \
- -lvtkParallelMPI -lvtkParallelCore"
+ -lvtkParallelMPI -lvtkParallelCore -lvtkCommonExecutionModel"
fi
======================================================================================================
More Detailed Instructions
Load your dev session
Build libmesh and the MOOSE executables in the dev session, but run the tests on the login node. This is an alias you can add to your ~/.bash_profile:
vim ~/.bash_profile
alias devsess =’srundev --mem=32gb --ntasks=4 --cpus-per-task=4 --time=12:00:00’
Load the MOOSE module
There are multiple versions. Each version was built with different versions of the required libraries. In general, the latest version of the libraries will be used in the latest version of the module. The module listed below is good in general, but you may have specific reasons to use a different version.
module load moose/12-aug-20
Check the module list, you should see something like:
module list
Currently Loaded Modules:
1) ufrc
2) gui/2.0.0
3) python/3.8
4) gcc/8.2.0
5) openmpi/4.0.3
6) mkl/2020.0.166
7) hypre/2.15.1
8) petsc/3.13.4
9) vtk/8.2.0
10) moose/12-aug-20
Clone MOOSE
mkdir ~/projects
cd ~/projects
git clone https://github.com/idaholab/moose.git
cd moose
git checkout master
Export VTK paths
Make sure you export after the MOOSE module is loaded, otherwise the path may be empty.
export VTKLIB_DIR=${HPC_VTK_LIB} VTKINCLUDE_DIR=${HPC_VTK_INC}
Update and Build libmesh – But Interrupt It
Run the update and build libmesh script but interrupt (ctrl-c) the configure/build just after the git clone of libmesh is completed. The goal here is to simply populate the libmesh directory so that the associated configure script can be patched.
cd ~/projects/moose/scripts
./scripts/update_and_rebuild_libmesh.sh --enable-vtk-required
(ctrl-c)
Patch the libmesh configure script
Patch the "configure" script according to the following command. The configure.patch file is available in the folder. Note: in the future, this patch will be merged and distributed with MOOSE/libmesh.
cd ~/projects/moose/libmesh
patch -p1 < configure.patch
Update and Build libmesh
Run the update and build libmesh script again and let it complete. It should finish without errors.
cd ~/projects/moose/scripts
./scripts/update_and_rebuild_libmesh.sh --enable-vtk-required
Compile MOOSE Tests
Build the MOOSE test executable.
cd ~/projects/moose/test
make -j 4
Run the Test Suites
Now run the tests. The test suite in moose/test is important to check if all libraries are correct. You need to exit the dev session to run the tests. If the build was successful, I would recommend building the module you are interested in and then exit and run both test scripts. Don’t forget that you may need to reload the moose module on the login node.
cd ~/projects/moose/test
./run_tests -j 4 (login node)
Compile and Test Your Own Module
Build the executable and run the specific tests.
cd ~/projects/moose/modules/combined
make -j16 (dev session)
./run_tests -j10 (login node)
General suggestions
If you are not getting a fresh clone of MOOSE, it’s recommended to do a clean-up before you recompile your executables.
cd ~/projects/moose/modules/combined
make clobberall
make -j16
You may consider adding the module load and export steps to your ~/.bash_profile:
vim ~/.bash_profile
module purge
module load moose/12-aug-20
export VTKLIB_DIR=${HPC_VTK_LIB} VTKINCLUDE_DIR=${HPC_VTK_INC}
MOOSE-based Applications
Perform all the previous steps related to the MOOSE installation. Then build your application and test it.
cd ~/projects/your_app
make clobberall
make -j 4
./run_tests -j 4 (login node)
Slurm script
This is an example of a slurm script:
#!/bin/sh
#SBATCH --job-name=moose #Job name
#SBATCH --nodes=1 #Number of nodes (servers, 32 proc/node)
#SBATCH --ntasks=16 #Number of tasks/MPI RankS
#SBATCH --ntasks-per-node=16 #Tasks per node
#SBATCH --ntasks-per-socket=8 #Tasks per socket
#SBATCH --cpus-per-task=1 #Number of CPU per task
#SBATCH --mem-per-cpu=3600mb #Memory (120 gig/server)
#SBATCH --distribution=cyclic:cyclic #Distribute tasks cyclically
#SBATCH --time=12:00:00 #Walltime days-hh:mm:ss
#SBATCH --output=moose-%j.out #Output and error log
#SBATCH --mail-type=END,FAIL #When to email user
#SBATCH --mail-user=your-email@ufl.edu #Email address to send mail to
#SBATCH --account=michael.tonks #Allocation group name, add -b for burst job
srun --mpi=pmix_v3 ~/projects/moose/modules/combined-opt -i moose_input_file.i