3. Module System

The main purpose of the module system is to assist in both running application software on Svante and/or configuring your enviroment to compile and run any code you have written.

To accomplish this, the module system loads proper path names for a specific task, instead of forcing the user to write ugly-looking statements in their .bashrc (or .cshrc for legacy C shell). In fact, setting software-related environment variables or hard-coded paths in your .bashrc is likely to cause headaches due to possible conflicts with the module system. Using modules, you can load and unload software on the fly, allowing the user to quickly swap versions of different software and/or change to different software entirely.

The most basic command is module load  «module name», e.g. module load matlab , typed from a terminal window prior to running an application or compiling code. A module load matlab command would load the default matlab pathways (currently set to matlab/2017a). You could also give a more specific matlab request, e.g. module load matlab/2018a which would load this newer installation.

Other useful module commands:

  • module unload «module name» – unloads a specific module

  • module avail – lists all available/installed modules on the the current node

  • module purge – clears all loaded modules

  • module show «module name» – shows paths/environment variables set and other sub-commands executed by loading this module

  • module swap «module1 module2» - swap out module1 for new module2

  • module help – gets help on module system, lists additional commands

Our standard application-based modules include MATLAB, IDL, GrADS, R, Python, GAMS, and NCL, among other installed software. (Python on Svante is particularly complicated, and discussed in detail in Section 5). There is also a Svante module which is automatically loaded when you log in (ergo, you will lose some basic cluster functionality after a module purge; do a module load svante following any purge).

If you need software that is not available as a module on Svante (as can be seen with the command module avail), you can request this software become available as a module by reaching out to svante-admin@mit.edu.

“Environment-dependent” modules: The pgi, intel, and gcc modules (for compiling user code, typically Fortran or C) are unique in that loading these modules not only sets the pgi, intel, or gcc paths respectively, but also allows for additional environment-dependent modules to become available. After loading one of these modules, additional modules now appear after a module avail command. At present, openmpi (and other MPI support libraries) and netcdf are the notable environment-dependent modules installed on our system; these cannot be loaded without first loading pgi, intel, or gcc. Basic netCDF analyses tools do NOT require one to load a netcdf module; there is a default, pre-loaded, non-compiler specific “system netCDF” functionality which successfully handles many command netCDF terminal commands and functions. Note however one can load different versions of netcdf modules (as displayed from the list of available modules), which overrides the default system netCDF version. For compiling and running Fortran 90 (F90), a compiler-specific module load netcdf is necessary. Codes which use Fortran 77, e.g. MITgcm, do NOT require a netcdf module to compile or run. If in doubt about using a netcdf module vs. falling back to the system netCDF, doing a module load netcdf for non-F90 applications generally won’t cause any problems.

Note

Often modules are OS-dependent, i.e., we maintain modules with similar basic functionality but only run properly on a specific operating system. Modules built for EDR, FDR nodes and svante-login.mit.edu (and, at this time, a few of the oldest file servers awaiting OS update) are located in /home/software/modulefiles (i.e. Fedora Core-based modules); modules that are NOT OS-dependent also reside here. Modules built for HDR, file servers and svante.mit.edu (i.e. Red Hat aka rhel–based) are located in /home/software/rhel/8/modulefiles; note these modules will not generally appear when you do a module avail on a EDR or FDR node. An exception is module intel/2021.4.0_rhel8, located in /home/software/modulefiles, which resides here for historical reasons.

Loading modules in SLURM scripts: Typically, you’ll want to load modules in scripts such as for SLURM submissions (see Section 4) and you’ll need this command prior to any module load statement:

source /etc/profile.d/modules.sh

(legacy C shell script users, substitute .csh instead of .sh above). Do NOT load pgi or intel modules (or their netcdf or MPI-based extensions) in your .bashrc (.cshrc for C shell) startup script, although if desired for convenience, one can load commonly used analysis modules such as matlab or ncl.

Many other HPC clusters employ a similar module system (e.g. MIT Engaging Cluster); there is often more complete module system documentation in their cluster documentation. It is considered bad scripting to have hard-coded support software pathway references in your build scripts, as one tends to forget about them and they crash when software is upgraded or moved, forcing one to debug such problems. But in some instances, such hard links may be necessary; if you tell us however about a required environment variable in your scripts, we can add this variable to those set during a module load.