Job statistics website:

Use the UB Metrics on Demand website to view job statistics as of the previous day.  More details on UBMoD can be found here




Command line options for viewing job history and accounting information:


Show SLURM Associations:

Each user has SLURM associations that enable access to the SLURM partitions and clusters.  The slimits command lists all associations the user has access to and any restrictions associated with those associations.  If the user has access to the 'supporters' quality of service (QOS) boost, it will be listed under QOS as normal,supporters



[ccrgstuser@rush:~]$ slimits

..................!

   Cluster Account               User        Def QOS             QOS

---------- -------------------- ---------- -------------------- --------------------------------------

   physics ccrstaff             ccruser                          scavenger

 chemistry ccrstaff             ccruser                          scavenger

       mae ccrstaff             ccruser                          scavenger

  industry ccrstaff             ccruser     industry             scavenger

    ub-hpc ccrstaff             ccruser     general-compute      debug,general-compute,gpu,largemem,viz


Show Job History and Accounting:

The SLURM job accounting records show job details, including job steps and memory usage.   The memory usage is accurate for jobs that use srun to launch tasks.


Show job account information for a specific job:
sacct -j jobid --format=User,JobID,Jobname,partition,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus,nodelist


or

slist jobid (if job is running, you will get node information.  If has completed, you will not)



Show information for jobs currently in queue:

sacct --format=User,JobID,Jobname,partition,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus,nodelist


Show all job information starting from a specific date:

sacct --starttime 2014-07-01 --format=User,JobID,Jobname,partition,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus,nodelist



Show all job information starting from a specific date for a specific user (nicer formatting than above command):

sacct -S start-date -u user-name


Example:

sacct 2015-01-01 ccrgst3
JobID        User      Partition       NNodes NCPUS  Start               Elapsed      State      Priority  
------------ --------- --------------- ------ ------ ------------------- ------------ ---------- -----------
3166812      ccrgst3   general-compute 1      2      2015-01-06T10:57:19 00:00:59     COMPLETED  50165      
3167316      ccrgst3   general-compute 1      2      2015-01-06T12:39:50 00:00:38     COMPLETED  50165      
3167850      ccrgst3   general-compute 1      2      2015-01-06T18:12:36 00:01:52     COMPLETED  50165      
3167852      ccrgst3   general-compute 1      2      2015-01-06T18:18:50 00:00:28     COMPLETED  50165      
3169338      ccrgst3   general-compute 1      2      2015-01-07T13:07:46 00:01:07     COMPLETED  50165      
3169477      ccrgst3   general-compute 1      2      2015-01-07T15:10:29 00:00:17     COMPLETED  50165      
3169519      ccrgst3   general-compute 1      2      2015-01-07T15:29:49 00:01:10     COMPLETED  50165      
3169841      ccrgst3   general-compute 1      2      2015-01-07T17:31:54 00:00:38     COMPLETED  50165      
3169893      ccrgst3   general-compute 1      2      2015-01-07T18:04:47 00:01:03     COMPLETED  50165      
3169906      ccrgst3   general-compute 1      2      2015-01-07T18:15:45 00:01:11     COMPLETED  50165      
3169920      ccrgst3   general-compute 1      2      2015-01-07T18:26:43 00:00:13     COMPLETED  50165      
3169924      ccrgst3   general-compute 1      2      2015-01-07T18:31:07 00:00:23     COMPLETED  50165      
3171870      ccrgst3   general-compute 1      12     2015-01-08T11:15:26 00:00:59     COMPLETED  50289      
3173721      ccrgst3   general-compute 1      12     2015-01-09T12:37:09 00:00:55     COMPLETED  50289      
3174384      ccrgst3   general-compute 1      12     2015-01-09T15:03:21 00:01:10     COMPLETED  50289      
3200062      ccrgst3   general-compute 1      12     2015-01-30T19:12:09 00:00:44     COMPLETED  50289      
3200079      ccrgst3   general-compute 1      12     2015-01-30T19:22:29 00:00:20     COMPLETED  50289      
3200081      ccrgst3   general-compute 1      12     2015-01-30T19:26:37 00:00:12     COMPLETED  50289      
3200107      ccrgst3   general-compute 1      12     2015-01-30T19:58:53 00:00:14     COMPLETED  50289      
3200111      ccrgst3   general-compute 2      24     2015-01-30T20:01:46 00:00:32     COMPLETED  50578      
3200117      ccrgst3   general-compute 2      24     2015-01-30T20:08:16 00:00:13     COMPLETED  50578      
3200119      ccrgst3   general-compute 1      12     2015-01-30T20:12:18 00:00:10     COMPLETED  50289      
3203497      ccrgst3   general-compute 1      12     2015-02-02T10:51:56 00:00:27     COMPLETED  50289      
3203507      ccrgst3   general-compute 1      12     2015-02-02T10:59:04 00:00:16     COMPLETED  50289      
3203509      ccrgst3   general-compute 2      24     2015-02-02T11:04:40 00:00:32     COMPLETED  50578      
3203554      ccrgst3   general-compute 1      12     2015-02-02T11:09:18 00:00:15     COMPLETED  50289      
3203558      ccrgst3   general-compute 1      12     2015-02-02T11:18:26 00:00:18     FAILED     50288      
3203563      ccrgst3   general-compute 2      24     2015-02-02T11:27:27 00:01:45     COMPLETED  50578      
3203566      ccrgst3   general-compute 1      12     2015-02-02T11:33:31 00:00:13     COMPLETED  50288      
3203568      ccrgst3   general-compute 1      12     2015-02-02T11:35:55 00:00:12     COMPLETED  50288      
3203579      ccrgst3   general-compute 2      24     2015-02-02T11:57:53 00:01:27     COMPLETED  50578      
4687745      ccrgst3   viz             1      2      2015-11-09T15:06:35 00:04:18     COMPLETED  50166