BioTeam Slurm Training¶
Training for the Slurm workload manager, built for life science HPC. 47 modules organized into progressive tracks — from your first job submission through production cluster administration. Every module includes working command examples validated against Slurm 25.11 (with 26.05 changes called out where they matter), and most include hands-on exercises with hints and solutions.
Who This Is For¶
The materials are organized by audience level so you can jump in at the right depth:
"What is this scheduler thing and why should I care?"
New to HPC or schedulers. You want to understand what Slurm does and why your organization uses it. Start with: Getting Started track
"I need to submit jobs and get work done."
You submit jobs regularly and need to be effective on the cluster — resource requests, interactive sessions, containers, best practices. Start with: End-User Essentials track
"I want to optimize my workflows and use advanced features."
Running complex workflows — job arrays, dependency pipelines, MPI, GPU workloads, scheduled automation. Start with: Power User track
"I need to install, configure, and manage Slurm."
Cluster administrators responsible for Slurm deployment, configuration, monitoring, and operations. Start with: Administrator track
"I need to understand capacity, cost, and strategy."
IT directors, research leads, and business stakeholders evaluating or overseeing HPC infrastructure. No command-line required. Start with: Leadership track
What's Covered¶
47 modules organized into six tracks, plus standalone guides. Click a tab to learn more:
Six progressive tracks build on each other:
- User tracks (L1-L3): 16 modules from "what is a scheduler" through GPU jobs, MPI, containers, and automated pipelines
- Administrator track (L4): 11 modules covering architecture, installation, configuration, partitions, fairshare, monitoring, troubleshooting, and HA
- Leadership track (L5): Strategy, capacity planning, and cost allocation for decision-makers — no command-line required
See the Training Tracks Guide for detailed reading paths, role-based learning paths by discipline, and prerequisites.
Dedicated guides for running life science applications on Slurm. Each has an admin module (setup/configuration) and a user module (daily workflows).
- Schrodinger Suite: SLM licensing, hosts file config, GPU setup, Job Server, -HOST flag, Maestro submission
- CryoSPARC: Cluster lanes, cluster_script.sh, GPU management, CryoSPARC Live, resource selection, failure diagnosis
Every module covers concepts that apply across all platforms. Deployment-specific differences are highlighted in callout blocks throughout:
ParallelCluster Note: AWS ParallelCluster-specific behavior
PCS Note: AWS PCS-specific behavior
On-Prem Note: Traditional bare-metal/VM considerations
Deep-dive deployment modules: On-Premise | ParallelCluster | PCS
Transitioning from another scheduler? Each guide provides side-by-side command mapping, job script translation, and behavioral differences.
- SGE to Slurm — qsub/sbatch, PEs/--ntasks, queues/partitions
- PBS/Torque to Slurm — PBS directives to #SBATCH
- LSF to Slurm — bsub/sbatch, LSF queues to Slurm partitions
Exercises in most modules provide practical tasks you can try on a real cluster, with expandable hint and solution blocks.
Quick reference:
- Command Cheatsheet — sbatch, squeue, sacct, sinfo at a glance
- Environment Variables — SLURM_* variables in job scripts
- Job State Codes — Decoding PD, R, CG, F, etc.
- Node State Codes — Decoding idle, alloc, drain, down
- Why Is It Pending / Why Did It Fail — Decision trees for stuck and failed jobs
- Glossary — Slurm and cloud-HPC terms, one line each
Choose Your Track¶
New to HPC or Slurm? Start here. Learn what a scheduler does, understand the key concepts, and submit your first job.
- What is HPC Scheduling? — Why clusters need a scheduler
- Slurm Overview — Nodes, partitions, jobs, steps
- Getting Started — Write and submit your first job
- Submitting Jobs — sbatch options, directives, output files
- Monitoring Jobs — squeue, sacct, sinfo
- Managing Jobs — scancel, hold/release, modify jobs
Next step: End-User Essentials for resource management and best practices. Full track details →
For working users who submit jobs regularly. Builds on Getting Started with resource requests, interactive sessions, containers, and cluster etiquette.
- Resource Requests — --mem, --cpus-per-task, --time, GRES
- Interactive Jobs — srun, salloc, Jupyter on compute nodes
- Environment Modules — module load/unload, Lmod
- Containers on Slurm — Apptainer/Singularity, GPU containers
- Best Practices — Resource estimation, being a good cluster citizen
Prerequisite: Getting Started track. Full track details →
Advanced workflows for experienced users. Job arrays, pipelines, MPI, GPU workloads, and scheduled automation.
- Job Arrays — Parameter sweeps, file-list processing, throttling
- Job Dependencies — afterok/afterany, building pipelines
- Parallel & MPI Jobs — srun as MPI launcher, hybrid jobs
- GPU Jobs — Multi-GPU, CUDA_VISIBLE_DEVICES, life science examples
- Recurring Jobs — Slurm's built-in cron (scrontab)
Prerequisite: End-User Essentials track. Full track details →
Install, configure, and manage Slurm clusters. 11 modules from architecture through high availability, plus deployment-specific guides.
- Architecture — Daemons, communication, authentication
- Installation — Packages, Munge, MariaDB, slurmdbd
- Configuration — slurm.conf, cgroup.conf, node definitions
- Partitions & QOS — Partition design, limits, preemption
- Accounts & Fairshare — sacctmgr, fairshare algorithm
- Resource Management — GPUs, licenses, cgroups
- Monitoring & Accounting — sreport, sdiag, Prometheus
- Policies & Priority — Multifactor priority, reservations
- Troubleshooting — Pending jobs, node states, log files
- Maintenance — Draining, upgrades, backups
- High Availability — slurmctld failover, slurmdbd HA
Strategy and cost management for decision-makers. No command-line required.
- Why Slurm — Market position, cloud integration, talent pool
- Capacity Planning — Utilization metrics, planning strategies
- Cost Allocation — Chargeback/showback, QOS budgets
Dedicated guides for life science applications on Slurm. Each has an admin setup module and an end-user workflow module.
Schrodinger Suite: Admin Setup — Licensing, hosts file, GPU config, Job Server | User Guide — -HOST flag, Maestro submission, monitoring
CryoSPARC: Admin Setup — Cluster lanes, GPU management, CryoSPARC Live | User Guide — Lane selection, resources, failure diagnosis
Developed by BioTeam Inc. -- HPC consulting for life science research.