BioTeam Slurm Training¶
Training for the Slurm workload manager, built for life science HPC. 44 modules organized into progressive tracks — from your first job submission through production cluster administration. Every module includes working command examples validated against Slurm 25.11, and most include hands-on exercises with hints and solutions.
Who This Is For¶
The materials are organized by audience level so you can jump in at the right depth:
"What is this scheduler thing and why should I care?"
New to HPC or schedulers. You want to understand what Slurm does and why your organization uses it. Start with: Getting Started track
"I need to submit jobs and get work done."
You submit jobs regularly and need to be effective on the cluster — resource requests, interactive sessions, containers, best practices. Start with: End-User Essentials track
"I want to optimize my workflows and use advanced features."
Running complex workflows — job arrays, dependency pipelines, MPI, GPU workloads, scheduled automation. Start with: Power User track
"I need to install, configure, and manage Slurm."
Cluster administrators responsible for Slurm deployment, configuration, monitoring, and operations. Start with: Administrator track
"I need to understand capacity, cost, and strategy."
IT directors, research leads, and business stakeholders evaluating or overseeing HPC infrastructure. No command-line required. Start with: Leadership track
What's Covered¶
44 modules organized into six tracks, plus standalone guides. Click a tab to learn more:
Six progressive tracks build on each other:
- User tracks (L1-L3): 16 modules from "what is a scheduler" through GPU jobs, MPI, containers, and automated pipelines
- Administrator track (L4): 11 modules covering architecture, installation, configuration, partitions, fairshare, monitoring, troubleshooting, and HA
- Leadership track (L5): Strategy, capacity planning, and cost allocation for decision-makers — no command-line required
See the Training Tracks Guide for detailed reading paths, role-based learning paths by discipline, and prerequisites.
Dedicated guides for running life science applications on Slurm. Each has an admin module (setup/configuration) and a user module (daily workflows).
- Schrodinger Suite: SLM licensing, hosts file config, GPU setup, Job Server, -HOST flag, Maestro submission
- CryoSPARC: Cluster lanes, cluster_script.sh, GPU management, CryoSPARC Live, resource selection, failure diagnosis
Every module covers concepts that apply across all platforms. Deployment-specific differences are highlighted in callout blocks throughout:
ParallelCluster Note: AWS ParallelCluster-specific behavior
PCS Note: AWS PCS-specific behavior
On-Prem Note: Traditional bare-metal/VM considerations
Deep-dive deployment modules: On-Premise | ParallelCluster | PCS
Transitioning from another scheduler? Each guide provides side-by-side command mapping, job script translation, and behavioral differences.
- SGE to Slurm — qsub/sbatch, PEs/--ntasks, queues/partitions
- PBS/Torque to Slurm — PBS directives to #SBATCH
- LSF to Slurm — bsub/sbatch, LSF queues to Slurm partitions
Exercises in most modules provide practical tasks you can try on a real cluster, with expandable hint and solution blocks.
Quick reference:
- Command Cheatsheet — sbatch, squeue, sacct, sinfo at a glance
- Environment Variables — SLURM_* variables in job scripts
- Job State Codes — Decoding PD, R, CG, F, etc.
- Node State Codes — Decoding idle, alloc, drain, down
Choose Your Track¶
New to HPC or Slurm? Start here. Learn what a scheduler does, understand the key concepts, and submit your first job.
- What is HPC Scheduling? — Why clusters need a scheduler
- Slurm Overview — Nodes, partitions, jobs, steps
- Getting Started — Write and submit your first job
- Submitting Jobs — sbatch options, directives, output files
- Monitoring Jobs — squeue, sacct, sinfo
- Managing Jobs — scancel, hold/release, modify jobs
Next step: End-User Essentials for resource management and best practices. Full track details →
For working users who submit jobs regularly. Builds on Getting Started with resource requests, interactive sessions, containers, and cluster etiquette.
- Resource Requests — --mem, --cpus-per-task, --time, GRES
- Interactive Jobs — srun, salloc, Jupyter on compute nodes
- Environment Modules — module load/unload, Lmod
- Containers on Slurm — Apptainer/Singularity, GPU containers
- Best Practices — Resource estimation, being a good cluster citizen
Prerequisite: Getting Started track. Full track details →
Advanced workflows for experienced users. Job arrays, pipelines, MPI, GPU workloads, and scheduled automation.
- Job Arrays — Parameter sweeps, file-list processing, throttling
- Job Dependencies — afterok/afterany, building pipelines
- Parallel & MPI Jobs — srun as MPI launcher, hybrid jobs
- GPU Jobs — Multi-GPU, CUDA_VISIBLE_DEVICES, life science examples
- Recurring Jobs — Slurm's built-in cron (scrontab)
Prerequisite: End-User Essentials track. Full track details →
Install, configure, and manage Slurm clusters. 11 modules from architecture through high availability, plus deployment-specific guides.
- Architecture — Daemons, communication, authentication
- Installation — Packages, Munge, MariaDB, slurmdbd
- Configuration — slurm.conf, cgroup.conf, node definitions
- Partitions & QOS — Partition design, limits, preemption
- Accounts & Fairshare — sacctmgr, fairshare algorithm
- Resource Management — GPUs, licenses, cgroups
- Monitoring & Accounting — sreport, sdiag, Prometheus
- Policies & Priority — Multifactor priority, reservations
- Troubleshooting — Pending jobs, node states, log files
- Maintenance — Draining, upgrades, backups
- High Availability — slurmctld failover, slurmdbd HA
Strategy and cost management for decision-makers. No command-line required.
- Why Slurm — Market position, cloud integration, talent pool
- Capacity Planning — Utilization metrics, planning strategies
- Cost Allocation — Chargeback/showback, QOS budgets
Dedicated guides for life science applications on Slurm. Each has an admin setup module and an end-user workflow module.
Schrodinger Suite: Admin Setup — Licensing, hosts file, GPU config, Job Server | User Guide — -HOST flag, Maestro submission, monitoring
CryoSPARC: Admin Setup — Cluster lanes, GPU management, CryoSPARC Live | User Guide — Lane selection, resources, failure diagnosis
Developed by BioTeam Inc. -- HPC consulting for life science research.