In addition to CCR's large production cluster and which are freely available to UB and affiliated researchers, CCR maintains a wide variety of project-specific storage systems and compute clusters. Faculty (PI) partitions are resources purchased by faculty for use by a specific research group. These nodes are not available to all CCR users. If you are interested in purchasing your own equipment, please see this page for more details.
The faculty partitions are grouped in separate cluster than the ub-hpc or industry clusters. When issuing a SLURM command, users must specify the cluster name "faculty." A partition must be specified as well. If a cluster and partition are not specified, then the default will be the CCR academic cluster, UB-HPC, and the general-compute partition.
The following instructions assume a basic familiarity with using SLURM on the CCR cluster. If you are a new user, then please read material on the Introductory SLURM page first.
To display a list of all partition names:
sinfo -M faculty
Specifying a Cluster and Partition
Use the following SLURM flags to specify a cluster:
Use the following SLURM flags to specify a partition and qos within a cluster:
To view list of what access you have, use the slimits command:
Submitting Jobs to a PI partition
Use a cluster flag and a partition flag when submitting a job.
#SBATCH --partition=partition_name --qos=qos_name
Viewing the Status of Jobs in a Partition or Cluster
Use a cluster flag to view jobs running in any partition within the cluster. The partition flag with show the jobs in that specific partition.
squeue -M cluster_name
squeue -M cluster_name -p partition_name -q qos_name
squeue --clusters=cluster_name --partition=partition_name --qos=qos_name
The following will show jobs on all clusters and partitions:
squeue -M all
If I wanted to view the status of my jobs running on any cluster and partition, then I would use the following command:
squeue -M all -u ccrgst
The graphical monitor slurmjobvis can be used to view jobs running on a faculty partition:
slurmjobvis jobid cluster
slurmjobvis 14527 faculty
Cancelling a Job
Use the cluster flag when canceling a job:
scancel -M cluster_name jobid
scancel --clusters jobid
Status of Nodes
Use the sinfo or snodes commands to check that status of nodes in a cluster or partition.:
sinfo -M cluster_name
sinfo -M cluster_name -p partition_name
snodes all cluster_name/all
snodes all cluster_name/partition_name
snodes all cluster_name/partition_name idle
Access a PI Partition using salloc
The salloc and srun commands do not accept any cluster flags. The default will be to use the CCR cluster, UB-HPC. Here are instructions to use salloc on the faculty cluster as an example:
This will set the default SLURM cluster to the faculty cluster. Now both salloc and srun will use the faculty cluster, rather than the CCR UB-HPC cluster. Once SLURM_CONF is set to a PI cluster, then the -M ub-hpc flag or --clusters=ub-hpc flag must be used to submit jobs to the CCR cluster. You can return to the original cluster default by unsetting the variable.