Environment Modules

# Environment Modules

## The Problem

HPC clusters host many software packages, often with multiple versions. Without a management system, you'd have conflicting PATH entries, incompatible library versions, and chaos.

---

## What Are Environment Modules?

Environment modules dynamically modify your shell environment -- setting `PATH`, `LD_LIBRARY_PATH`, and other variables -- so the right version of software is available when you need it.

Two implementations are common:

- **Environment Modules** (TCL-based, the original)
- **Lmod** (Lua-based, more features, increasingly standard)

The commands are nearly identical for daily use.

---

## Essential Commands

```bash
# List available modules
$ module avail

# Search for a specific package
$ module avail blast
--- /opt/modules ---
blast/2.13
blast/2.14
blast/2.15 (D)

# Load a module
$ module load blast/2.15

# Load default version (marked with D)
$ module load blast

# List currently loaded modules
$ module list
Currently Loaded Modules:
  1) blast/2.15

# Show what a module does
$ module show blast/2.15

# Unload a module
$ module unload blast

# Swap versions
$ module swap blast/2.15 blast/2.14

# Unload everything
$ module purge

# Search by keyword (Lmod)
$ module spider python
```

---

## Using Modules in Job Scripts

Always load your modules in the job script. Don't rely on your login environment being inherited:

```bash
#!/bin/bash
#SBATCH --job-name=blast_job
#SBATCH --time=02:00:00

# Clean environment first (recommended)
module purge

# Load required software
module load blast/2.15
module load python/3.11

# Verify
which blastn
python --version

# Run your work
blastn -query input.fasta -db /data/nt -out results.txt
```

> **Why `module purge` first?** It ensures a clean, reproducible environment regardless of what was loaded in your shell before submission.

---

## Module Hierarchies (Lmod)

Lmod supports hierarchical modules where loading a compiler or MPI implementation reveals additional modules:

```bash
$ module avail          # Shows core modules only
$ module load gcc/12.3  # Load a compiler
$ module avail          # Now shows modules built with gcc/12.3
$ module load openmpi/4.1
$ module avail          # Now shows MPI-dependent modules too
```

This prevents loading incompatible combinations (e.g., a library built with GCC used with Intel-compiled code).

---

## Module Collections (Lmod)

Save a set of modules for reuse:

```bash
# Set up your environment
$ module purge
$ module load gcc/12.3 openmpi/4.1 python/3.11 blast/2.15

# Save as a named collection
$ module save myenv

# Later, restore it
$ module restore myenv

# List saved collections
$ module savelist
```

> **ParallelCluster / PCS Note:** Because compute nodes are launched dynamically, software modules must be installed on **shared storage** (EFS or FSx for Lustre) so they are available on every node. A common pattern is to mount a shared filesystem at `/opt/modules` or `/shared/modules` and point `MODULEPATH` there. Modules installed only on the head node's local disk will not be visible to compute nodes.

---

## Common Pitfalls

**Module not found in job script:**
Your login node and compute nodes must have the same module paths. If a module works interactively but fails in a job, check that the module system is initialized:

```bash
#!/bin/bash
#SBATCH ...

# This is usually automatic, but if modules fail:
source /etc/profile.d/modules.sh    # or lmod.sh
module load blast/2.15
```

**Wrong version loaded:**
Be explicit about versions. `module load blast` loads the default, which may change when admins update it.

**Conflicting modules:**
Lmod will warn or error if you try to load incompatible modules. Use `module swap` to switch versions.

Exercises¶

Find and load a module

Use module avail to browse the software installed on your cluster. Pick a tool (e.g., BLAST, Python, GROMACS), load it, and verify the correct binary is active. Then unload it and confirm it is gone.

Hint / Solution

# Browse available modules
module avail

# Search for a specific tool
module avail blast

# Load it
module load blast/2.15

# Verify the binary is on your PATH
which blastn
blastn -version

# Unload and confirm
module unload blast
which blastn    # should return "not found" or point elsewhere

Expected output: Validated 2026-04-15 on slurm-training-val (Slurm 25.11.4, ParallelCluster 3.15.0).

$ module avail testapp 2>&1

----------------------------- /shared/modulefiles ------------------------------
   testapp/1.0    testapp/2.0 (D)

----------------------------- /shared/modulefiles ------------------------------
   testapp/1.0    testapp/2.0 (D)

  Where:
   D:  Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".

$ module load testapp/1.0 && module list 2>&1

Currently Loaded Modules:
  1) testapp/1.0



$ module unload testapp && module list 2>&1
No modules loaded

Compare module versions

Find a tool that has multiple versions installed. Load one version, check its version output, then swap to a different version and check again. This demonstrates why being explicit about versions matters in job scripts.

Hint / Solution

# See what versions are available
module avail python

# Load one version
module load python/3.10
python3 --version    # Python 3.10.x

# Swap to another version
module swap python/3.10 python/3.12
python3 --version    # Python 3.12.x

# Show what you have loaded
module list

Expected output: Validated 2026-04-15 on slurm-training-val (Slurm 25.11.4, ParallelCluster 3.15.0).

$ module purge && module load testapp/1.0 && module list 2>&1 && echo '---' && module swap testapp/1.0 testapp/2.0 && module list 2>&1

Currently Loaded Modules:
  1) testapp/1.0



---

The following have been reloaded with a version change:
  1) testapp/1.0 => testapp/2.0


Currently Loaded Modules:
  1) testapp/2.0

Save a module collection

Load the set of modules you would use for a typical analysis session, save them as a named collection, then restore them in a clean environment. This is useful for reproducible job scripts.

Hint / Solution

# Start clean
module purge

# Load your working set
module load python/3.12
module load blast/2.15
module load samtools/1.19

# Save as a named collection
module save my_analysis

# Verify the collection
module savelist

# Purge and restore
module purge
module list              # nothing loaded
module restore my_analysis
module list              # all three modules back

In a job script, you can put `module restore my_analysis` at the top instead of multiple `module load` lines. Expected output: Validated 2026-04-15 on slurm-training-val (Slurm 25.11.4, ParallelCluster 3.15.0).

$ module purge && module load testapp/2.0 && module save val_em3_collection 2>&1
Saved current collection of modules to: "val_em3_collection"

$ module purge && module list 2>&1
No modules loaded

$ module restore val_em3_collection && module list 2>&1
Restoring modules from user's val_em3_collection

Currently Loaded Modules:
  1) testapp/2.0

Environment Modules

Exercises¶

References¶