Plugin for direct execution.
aiida.schedulers.plugins.direct.
DirectJobResource
Bases: aiida.schedulers.datastructures.NodeNumberJobResource
aiida.schedulers.datastructures.NodeNumberJobResource
__abstractmethods__
__module__
_abc_impl
DirectScheduler
Bases: aiida.schedulers.scheduler.Scheduler
aiida.schedulers.scheduler.Scheduler
Support for the direct execution bypassing schedulers.
_convert_time
Convert a string in the format HH:MM:SS to a number of seconds.
_features
_get_joblist_command
The command to report full information on existing jobs.
to pass the -t options to list each subjob).
_get_kill_command
Return the command to kill the job with specified jobid.
_get_submit_command
Return the string to execute to submit a given script.
Note
One needs to redirect stdout and stderr to /dev/null otherwise the daemon remains hanging for the script to run
submit_script – the path of the submit script relative to the working directory. IMPORTANT: submit_script should be already escaped.
_get_submit_script_header
Return the submit script header, using the parameters from the job_tmpl.
job_tmpl: an JobTemplate instance with relevant parameters set.
_job_resource_class
alias of DirectJobResource
_logger
_parse_joblist_output
Parse the queue output string, as returned by executing the command returned by _get_joblist_command command (qstat -f).
Return a list of JobInfo objects, one of each job, each relevant parameters implemented.
depending on the scheduler configuration, finished jobs may either appear here, or not. This function will only return one element for each job find in the qstat output; missing jobs (for whatever reason) simply will not appear here.
_parse_kill_output
Parse the output of the kill command.
To be implemented by the plugin.
True if everything seems ok, False otherwise.
_parse_submit_output
Parse the output of the submit command, as returned by executing the command returned by _get_submit_command command.
Return a string with the JobID.
get_jobs
Overrides original method from DirectScheduler in order to list missing processes as DONE.
Plugin for LSF. This has been tested on the CERN lxplus cluster (LSF 9.1.3)
aiida.schedulers.plugins.lsf.
LsfJobResource
Bases: aiida.schedulers.datastructures.JobResource
aiida.schedulers.datastructures.JobResource
An implementation of JobResource for LSF, that supports the OPTIONAL specification of a parallel environment (a string) + the total number of processors.
‘parallel_env’ should contain a string of the form “host1 host2! hostgroupA! host3 host4” where the “!” symbol indicates the first execution host candidates. Other hosts are added only if the number of processors asked is more than those of the first execution host. See https://www-01.ibm.com/support/knowledgecenter/SSETD4_9.1.2/lsf_command_ref/bsub.1.dita?lang=en for more details about the parallel environment definition (the -m option of bsub).
__init__
Initialize the job resources from the passed arguments (the valid keys can be obtained with the function self.get_valid_keys()).
ValueError – on invalid parameters.
TypeError – on invalid parameters.
aiida.common.ConfigurationError – if default_mpiprocs_per_machine was set for this computer, since LsfJobResource cannot accept this parameter.
_default_fields
accepts_default_mpiprocs_per_machine
Return True if this JobResource accepts a ‘default_mpiprocs_per_machine’ key, False otherwise.
get_tot_num_mpiprocs
Return the total number of cpus of this job resource.
LsfScheduler
Support for the IBM LSF scheduler ‘https://www-01.ibm.com/support/knowledgecenter/SSETD4_9.1.2/lsf_welcome.html’
_get_detailed_job_info_command
Return the command to run to get the detailed information on a job, even after the job has finished.
The output text is just retrieved, and returned for logging purposes.
Separates the fields with the _field_separator string order: jobnum, state, walltime, queue[=partition], user, numnodes, numcores, title
_get_submit_script_footer
Return the submit script final part, using the parameters from the job_tmpl.
job_tmpl – a JobTemplate instance with relevant parameters set.
Return the submit script header, using the parameters from the job_tmpl. See the following manual https://www-01.ibm.com/support/knowledgecenter/SSETD4_9.1.2/lsf_command_ref/bsub.1.dita?lang=en for more details about the possible options to bsub, in particular for the parallel environment definition (with the -m option).
job_tmpl – an JobTemplate instance with relevant parameters set.
alias of LsfJobResource
_joblist_fields
Parse the queue output string, as returned by executing the command returned by _get_joblist_command command, that is here implemented as a list of lines, one for each job, with _field_separator as separator. The order is described in the _get_joblist_command function.
either appear here, or not. This function will only return one element for each job find in the qstat output; missing jobs (for whatever reason) simply will not appear here.
_parse_time_string
Parse a time string and returns a datetime object. Example format: ‘Feb 2 07:39’ or ‘Feb 2 07:39 L’
Base classes for PBSPro and PBS/Torque plugins.
aiida.schedulers.plugins.pbsbaseclasses.
PbsBaseClass
Base class with support for the PBSPro scheduler (http://www.pbsworks.com/) and for PBS and Torque (http://www.adaptivecomputing.com/products/open-source/torque/).
Only a few properties need to be redefined, see examples of the pbspro and torque plugins
_get_resource_lines
Return a set a list of lines (possibly empty) with the header lines relative to:
num_machines
num_mpiprocs_per_machine
num_cores_per_machine
max_memory_kb
max_wallclock_seconds
This is done in an external function because it may change in different subclasses.
directory. IMPORTANT: submit_script should be already escaped.
TODO: truncate the title if too long
alias of PbsJobResource
PbsJobResource
_map_status
Parse a time string in the format returned from qstat -f and returns a datetime object.
Class for PBS job resources.
validate_resources
Validate the resources against the job resource class of this scheduler.
This extends the base class validator and calculates the num_cores_per_machine fields to pass to PBSlike schedulers. Checks that num_cores_per_machine is a multiple of num_cores_per_mpiproc and/or num_mpiprocs_per_machine.
kwargs – dictionary of values to define the job resources
attribute dictionary with the parsed parameters populated
ValueError – if the resources are invalid or incomplete
Plugin for PBSPro. This has been tested on PBSPro v. 12.
aiida.schedulers.plugins.pbspro.
PbsproScheduler
Bases: aiida.schedulers.plugins.pbsbaseclasses.PbsBaseClass
aiida.schedulers.plugins.pbsbaseclasses.PbsBaseClass
Subclass to support the PBSPro scheduler (http://www.pbsworks.com/).
I redefine only what needs to change from the base class
Return the lines for machines, memory and wallclock relative to pbspro.
Plugin for SGE. This has been tested on GE 6.2u3.
Plugin originally written by Marco Dorigo. Email: marco(DOT)dorigo(AT)rub(DOT)de
aiida.schedulers.plugins.sge.
SgeJobResource
Bases: aiida.schedulers.datastructures.ParEnvJobResource
aiida.schedulers.datastructures.ParEnvJobResource
SgeScheduler
Support for the Sun Grid Engine scheduler and its variants/forks (Son of Grid Engine, Oracle Grid Engine, …)
Return the command to run to get detailed information for a given job.
This is typically called after the job has finished, to retrieve the most detailed information possible about the job. This is done because most schedulers just make finished jobs disappear from the qstat command, and instead sometimes it is useful to know some more detailed information about the job exit status, etc.
aiida.common.exceptions.FeatureNotAvailable
!!!ALL COPIED FROM PBSPRO!!! TODO: understand if it is worth escaping the username, or rather leave it unescaped to allow to pass $USER
alias of SgeJobResource
Parse the joblist output as returned by executing the command returned by _get_joblist_command method.
list of JobInfo objects, one of each job each with at least its default params implemented.
Parse a time string in the format returned from qstat -xml -ext and returns a datetime object. Example format: 2013-06-13T11:53:11
Plugin for SLURM. This has been tested on SLURM 14.03.7 on the CSCS.ch machines.
aiida.schedulers.plugins.slurm.
SlurmJobResource
Class for SLURM job resources.
This extends the base class validator to check that the num_cores_per_machine are a multiple of num_cores_per_mpiproc and/or num_mpiprocs_per_machine.
SlurmScheduler
Support for the SLURM scheduler (http://slurm.schedmd.com/).
Convert a string in the format DD-HH:MM:SS to a number of seconds.
_detailed_job_info_fields
The output text is just retrieved, and returned for logging purposes. –parsable split the fields with a pipe (|), adding a pipe also at the end.
Separate the fields with the _field_separator string order: jobnum, state, walltime, queue[=partition], user, numnodes, numcores, title
alias of SlurmJobResource
fields
Plugin for PBS/Torque. This has been tested on Torque v.2.4.16 (from Ubuntu).
aiida.schedulers.plugins.torque.
TorqueScheduler
Subclass to support the Torque scheduler..