Skip to content

Latest commit

 

History

History
94 lines (71 loc) · 10.3 KB

README.md

File metadata and controls

94 lines (71 loc) · 10.3 KB
Raw

mjobs

A little tool to make it easier to inspect IBM Spectrum LSF (or just LSF) and Slurm jobs.

The program will auto detect bjobs under LSF, or squeue in Slurm.

IBM LSF

mjobs doesn't not support all the options of bjobs, only a subset. It also adds a the -f FILTER, --bkill, -ts and -nh options.

[mbc@codon-login-02 ~]$ ./mjobs -h
Usage: mjobs [-h] [-f FILTER] [-ts] [-nh] [-q QUEUE] [-u USER] [-r] [-a] [-d] [-G USER_GROUP] [-g GROUP] [-m HOSTS] [-p] [-e] [--bkill] [job_id ...]

Just like bjobs but a bit nicer

Positional Arguments:
  job_id         Specifies the jobs or job arrays that bjobs displays.

Optional Arguments:
  -h, --help     show this help message and exit
  -f FILTER      Filter the jobs using the specified regex on the job name or pending reason.
  -ts, --tsv     No fancy table, a good ol' tsv
  -nh            Don't print the table header, useful to pipe the tsv output
  -q QUEUE       Displays jobs in the specified queue
  -u USER        Displays jobs in the specified user
  -r             Displays running jobs.
  -a             Displays information about jobs in all states, including jobs that finished recently.
  -d             Displays information about jobs that finished recently.
  -G USER_GROUP  Displays jobs associated with the specified user group.
  -g GROUP       Displays information about jobs attached to the specified job group.
  -m HOSTS       Displays jobs dispatched to the specified hosts.
  -p             Displays pending jobs, together with the pending reasons that caused each job not to be dispatched during the last dispatch turn.
  -e             Add the execution hosts, output file and error file to the table.
  --bkill        Terminate found or filtered jobs with bkill.

Example

[mbc@codon-login-02 ~]$ mjobs -u emgpr_wgs -f SRR12480298_mgnify_analysis
                                                                LSF jobs for emgpr_wgs                                                                 
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃    JobId ┃ Status ┃ JobName                     ┃ JobGroup ┃ User      ┃ Queue      ┃ Submit Time  ┃ Start Time   ┃ Finish Time    ┃ Pending reason ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 87346050 │ RUN    │ SRR12480298_mgnify_analysis │   ----   │ emgpr_wgs │ production │ Feb 25 08:00 │ Feb 25 08:03 │ Feb 28 08:03 L │      ----      │
└──────────┴────────┴─────────────────────────────┴──────────┴───────────┴────────────┴──────────────┴──────────────┴────────────────┴────────────────┘

Slurm

mjobs doesn't not support all the options of squeue, only a subset. It also adds a the -f FILTER, -ts and -nh options.

[mbc@codon-slurm-login-01 ~]$ ./mjobs -h
Usage: mjobs [-h] [-f FILTER] [-ts] [-nh] [-p PARTITION] [-u USER]
             [-t {pending,running,suspended,completed,cancelled,failed,timeout,node_fail,preempted,boot_fail,deadline,out_of_memory,completing,configuring,resizing,resv_del_hold,requeued,requeue_fed,requeue_hold,revoked,signaling,special_exit,stage_out,stopped} [{pending,running,suspended,completed,cancelled,failed,timeout,node_fail,preempted,boot_fail,deadline,out_of_memory,completing,configuring,resizing,resv_del_hold,requeued,requeue_fed,requeue_hold,revoked,signaling,special_exit,stage_out,stopped} ...]]
             [-w NODELIST [NODELIST ...]] [-e]
             [job_id ...]

Just like squeue but a bit nicer

Positional Arguments:
  job_id                Specifies the jobs or job arrays that squeue displays.

Optional Arguments:
  -h, --help            show this help message and exit
  -f FILTER             Filter the jobs using the specified regex on the job name or pending reason.
  -ts, --tsv            No fancy table, a good ol' tsv
  -nh                   Don't print the table header, useful to pipe the tsv output
  -p, --partition PARTITION
                        Specify the partitions of the jobs or steps to view. Accepts a comma separated list of partition names.
  -u, --user USER       Request jobs or job steps from a comma separated list of users. The list can consist of user names or user id numbers. Performance of the command can be measurably improved for systems with large numbers of jobs when a
                        single user is specified.
  -t, --states {pending,running,suspended,completed,cancelled,failed,timeout,node_fail,preempted,boot_fail,deadline,out_of_memory,completing,configuring,resizing,resv_del_hold,requeued,requeue_fed,requeue_hold,revoked,signaling,special_exit,stage_out,stopped} [{pending,running,suspended,completed,cancelled,failed,timeout,node_fail,preempted,boot_fail,deadline,out_of_memory,completing,configuring,resizing,resv_del_hold,requeued,requeue_fed,requeue_hold,revoked,signaling,special_exit,stage_out,stopped} ...]
                        Specify the states of jobs to view. Accepts a comma separated list of state names or 'all'. If 'all' is specified then jobs of all states will be reported. If no state is specified then pending, running, and completing jobs
                        are reported. See the JOB STATE CODES section below for a list of valid states. Both extended and compact forms are valid. Note the <state_list> supplied is case insensitive ('pending' and 'PENDING' are equivalent).
  -w, --nodelist NODELIST [NODELIST ...]
                        Report only on jobs allocated to the specified node or list of nodes. This may either be the NodeName or NodeHostname as defined in slurm.conf(5) in the event that they differ. A node_name of localhost is mapped to the
                        current host name.
  -e                    Add the execution nodes, stdoutput file and stderror file to the table.

Installation - bundle the app

Create an executable with pyinstaller.

$ pyinstaller mjobs/main.py --onefile --clean --name mjobs
$ ./dist/main --help

GH Actions binary

This binary is build the python 3.9 buster docker image. This is done to maintain compatibility with the libc version use at EMBl-EBI. For more information -> https://pyinstaller.org/en/stable/usage.html#making-gnu-linux-apps-forward-compatible

Download

Get the executable from the releases tab.