Environment Modules

# Environment Modules

## The Problem

HPC clusters host many software packages, often with multiple versions. Without a management system, you'd have conflicting PATH entries, incompatible library versions, and chaos.

---

## What Are Environment Modules?

Environment modules dynamically modify your shell environment -- setting `PATH`, `LD_LIBRARY_PATH`, and other variables -- so the right version of software is available when you need it.

Two implementations are common:

- **Environment Modules** (TCL-based, the original)
- **Lmod** (Lua-based, more features, increasingly standard)

The commands are nearly identical for daily use.

---

## Essential Commands

```bash
# List available modules
$ module avail

# Search for a specific package
$ module avail blast
--- /opt/modules ---
blast/2.13
blast/2.14
blast/2.15 (D)

# Load a module
$ module load blast/2.15

# Load default version (marked with D)
$ module load blast

# List currently loaded modules
$ module list
Currently Loaded Modules:
  1) blast/2.15

# Show what a module does
$ module show blast/2.15

# Unload a module
$ module unload blast

# Swap versions
$ module swap blast/2.15 blast/2.14

# Unload everything
$ module purge

# Search by keyword (Lmod)
$ module spider python
```

---

## Using Modules in Job Scripts

Always load your modules in the job script. Don't rely on your login environment being inherited:

```bash
#!/bin/bash
#SBATCH --job-name=blast_job
#SBATCH --time=02:00:00

# Clean environment first (recommended)
module purge

# Load required software
module load blast/2.15
module load python/3.11

# Verify
which blastn
python --version

# Run your work
blastn -query input.fasta -db /data/nt -out results.txt
```

> **Why `module purge` first?** It ensures a clean, reproducible environment regardless of what was loaded in your shell before submission.

---

## Module Hierarchies (Lmod)

Lmod supports hierarchical modules where loading a compiler or MPI implementation reveals additional modules:

```bash
$ module avail          # Shows core modules only
$ module load gcc/12.3  # Load a compiler
$ module avail          # Now shows modules built with gcc/12.3
$ module load openmpi/4.1
$ module avail          # Now shows MPI-dependent modules too
```

This prevents loading incompatible combinations (e.g., a library built with GCC used with Intel-compiled code).

---

## Module Collections (Lmod)

Save a set of modules for reuse:

```bash
# Set up your environment
$ module purge
$ module load gcc/12.3 openmpi/4.1 python/3.11 blast/2.15

# Save as a named collection
$ module save myenv

# Later, restore it
$ module restore myenv

# List saved collections
$ module savelist
```

> **ParallelCluster / PCS Note:** Because compute nodes are launched dynamically, software modules must be installed on **shared storage** (EFS or FSx for Lustre) so they are available on every node. A common pattern is to mount a shared filesystem at `/opt/modules` or `/shared/modules` and point `MODULEPATH` there. Modules installed only on the head node's local disk will not be visible to compute nodes.

---

## Common Pitfalls

**Module not found in job script:**
Your login node and compute nodes must have the same module paths. If a module works interactively but fails in a job, check that the module system is initialized:

```bash
#!/bin/bash
#SBATCH ...

# This is usually automatic, but if modules fail:
source /etc/profile.d/modules.sh    # or lmod.sh
module load blast/2.15
```

**Wrong version loaded:**
Be explicit about versions. `module load blast` loads the default, which may change when admins update it.

**Conflicting modules:**
Lmod will warn or error if you try to load incompatible modules. Use `module swap` to switch versions.

Environment Modules

References¶