aiida.engine package

Module with all the internals that make up the engine of aiida-core.

aiida.engine.run(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

the outputs of the process

aiida.engine.run_get_pid(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and process pid

aiida.engine.run_get_node(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and the calculation node

aiida.engine.submit(process, **inputs)

Submit the process with the supplied inputs to the daemon runners immediately returning control to the interpreter. The return value will be the calculation node of the submitted process.

Parameters:

• process – the process class to submit
• inputs – the inputs to be passed to the process

Returns:

the calculation node of the process

class aiida.engine.CalcJob(*args, **kwargs)

Bases: aiida.engine.processes.process.Process

Implementation of the CalcJob process.

__abstractmethods__ = frozenset([])

__init__(*args, **kwargs)

Construct the instance only if it is a sub class of CalcJob otherwise raise InvalidOperation.

__module__ = 'aiida.engine.processes.calcjobs.calcjob'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 102

_abc_registry = <_weakrefset.WeakSet object>

_node_class

alias of aiida.orm.nodes.process.calculation.calcjob.CalcJobNode

_spec_type

alias of aiida.engine.processes.process_spec.CalcJobProcessSpec

classmethod define(spec)

classmethod get_state_classes()

link_label_retrieved = 'retrieved'

on_terminated()

Cleanup the node by deleting the calulation job state.

Note

This has to be done before calling the super because that will seal the node after we cannot change it

parse(retrieved_temporary_folder=None)

Parse a retrieved job calculation.

This is called once it’s finished waiting for the calculation to be finished and the data has been retrieved.

prepare_for_submission(folder)

Docs.

presubmit(folder)

Prepares the calculation folder with all inputs, ready to be copied to the cluster.

Parameters:folder – a SandboxFolder, empty in input, that will be filled with calculation input files and the scheduling script. Return calcinfo:  the CalcInfo object containing the information needed by the daemon to handle operations.

run()

Run the calculation, we put it in the TOSUBMIT state and then wait for it to be completed.

class aiida.engine.ExitCode(status, message)

Bases: tuple

__dict__ = dict_proxy({'status': <property object>, '__module__': 'aiida.engine.processes.exit_code', '__getstate__': <function __getstate__>, '__new__': <staticmethod object>, '_make': <classmethod object>, '_fields': ('status', 'message'), '_replace': <function _replace>, '__slots__': (), '_asdict': <function _asdict>, '__repr__': <function __repr__>, '__dict__': <property object>, 'message': <property object>, '__getnewargs__': <function __getnewargs__>, '__doc__': 'ExitCode(status, message)'})

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__getstate__()

Exclude the OrderedDict from pickling

__module__ = 'aiida.engine.processes.exit_code'

static __new__(_cls, status=0, message=None)

Create new instance of ExitCode(status, message)

__repr__()

Return a nicely formatted representation string

__slots__ = ()

_asdict()

Return a new OrderedDict which maps field names to their values

_fields = ('status', 'message')

classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new ExitCode object from a sequence or iterable

_replace(**kwds)

Return a new ExitCode object replacing specified fields with new values

message

Alias for field number 1

status

Alias for field number 0

class aiida.engine.ExitCodesNamespace

Bases: aiida.common.extendeddicts.AttributeDict

A namespace of ExitCode tuples that can be accessed through getattr as well as getitem. Additionally, the collection can be called with an identifier, that can either reference the integer status of the ExitCode that needs to be retrieved or the key in the collection

__call__(identifier)

Return a specific exit code identified by either its exit status or label

Parameters:identifier – the identifier of the exit code. If the type is integer, it will be interpreted as the exit code status, otherwise it be interpreted as the exit code label Returns:an ExitCode named tuple Raises:ValueError – if no exit code with the given label is defined for this process

__module__ = 'aiida.engine.processes.exit_code'

aiida.engine.calcfunction(function)

A decorator to turn a standard python function into a calcfunction. Example usage:

>>> from aiida.orm import Int
>>>
>>> # Define the calcfunction
>>> @calcfunction
>>> def sum(a, b):
>>>    return a + b
>>> # Run it with some input
>>> r = sum(Int(4), Int(5))
>>> print(r)
9
>>> r.get_incoming().all() # doctest: +SKIP
[Neighbor(link_type='', link_label='result',
node=<CalcFunctionNode: uuid: ce0c63b3-1c84-4bb8-ba64-7b70a36adf34 (pk: 3567)>)]
>>> r.get_incoming().get_node_by_label('result').get_incoming().all_nodes()
[4, 5]

aiida.engine.workfunction(function)

A decorator to turn a standard python function into a workfunction. Example usage:

>>> from aiida.orm import Int
>>>
>>> # Define the workfunction
>>> @workfunction
>>> def select(a, b):
>>>    return a
>>> # Run it with some input
>>> r = select(Int(4), Int(5))
>>> print(r)
4
>>> r.get_incoming().all() # doctest: +SKIP
[Neighbor(link_type='', link_label='result',
node=<WorkFunctionNode: uuid: ce0c63b3-1c84-4bb8-ba64-7b70a36adf34 (pk: 3567)>)]
>>> r.get_incoming().get_node_by_label('result').get_incoming().all_nodes()
[4, 5]

class aiida.engine.Process(inputs=None, logger=None, runner=None, parent_pid=None, enable_persistence=True)

Bases: plumpy.processes.Process

This class represents an AiiDA process which can be executed and will have full provenance saved in the database.

SINGLE_OUTPUT_LINKNAME = 'result'

class SaveKeys

Bases: enum.Enum

Keys used to identify things in the saved instance state bundle.

CALC_ID = 'calc_id'

__module__ = 'aiida.engine.processes.process'

__abstractmethods__ = frozenset([])

__init__(inputs=None, logger=None, runner=None, parent_pid=None, enable_persistence=True)

The signature of the constructor should not be changed by subclassing processes.

Parameters:

• inputs (dict) – A dictionary of the process inputs
• pid – The process ID, can be manually set, if not a unique pid will be chosen
• logger (logging.Logger) – An optional logger for the process to use
• loop – The event loop
• communicator (plumpy.Communicator) – The (optional) communicator

__module__ = 'aiida.engine.processes.process'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 102

_abc_registry = <_weakrefset.WeakSet object>

_auto_persist = set(['_CREATION_TIME', '_enable_persistence', '_future', '_parent_pid', '_paused', '_pid', '_pre_paused_status', '_status'])

_create_and_setup_db_record()

Create and setup the database record for this process

Returns:the uuid of the process

_flat_inputs()

Return a flattened version of the parsed inputs dictionary.

The eventual keys will be a concatenation of the nested keys. Note that the metadata dictionary, if present, is not passed, as those are dealt with separately in _setup_metadata.

Returns:flat dictionary of parsed inputs

_flatten_inputs(port, port_value, parent_name='', separator='_')

Function that will recursively flatten the inputs dictionary, omitting inputs for ports that are marked as being non database storable

Parameters:

• port – port against which to map the port value, can be InputPort or PortNamespace
• port_value – value for the current port, can be a Mapping
• parent_name – the parent key with which to prefix the keys
• separator – character to use for the concatenation of keys

static _get_namespace_list(namespace=None, agglomerate=True)

Get the list of namespaces in a given namespace

_node_class

alias of aiida.orm.nodes.process.process.ProcessNode

_save_checkpoint()

Save the current state in a chechpoint if persistence is enabled and the process state is not terminal

If the persistence call excepts with a PersistenceError, it will be caught and a warning will be logged.

_setup_db_record()

Create the database record for this process and the links with respect to its inputs

This function will set various attributes on the node that serve as a proxy for attributes of the Process. This is essential as otherwise this information could only be introspected through the Process itself, which is only available to the interpreter that has it in memory. To make this data introspectable from any interpreter, for example for the command line interface, certain Process attributes are proxied through the calculation node.

In addition, the parent calculation will be setup with a CALL link if applicable and all inputs will be linked up as well.

_setup_inputs()

Create the links between the input nodes and the ProcessNode that represents this process.

_setup_metadata()

Store the metadata on the ProcessNode.

_spec_type

alias of aiida.engine.processes.process_spec.ProcessSpec

classmethod build_process_type()

The process type.

Returns:string of the process type Return type:str

Note: This could be made into a property ‘process_type’ but in order to have it be a property of the class it would need to be defined in the metaclass, see https://bugs.python.org/issue20659

decode_input_args(encoded)

Decode saved input arguments as they came from the saved instance state Bundle

Parameters:encoded – Returns:The decoded input args

classmethod define(spec)

encode_input_args(inputs)

Encode input arguments such that they may be saved in a Bundle

Parameters:inputs – A mapping of the inputs as passed to the process Returns:The encoded inputs

exit_codes = {}

exposed_inputs(process_class, namespace=None, agglomerate=True)

Gather a dictionary of the inputs that were exposed for a given Process class under an optional namespace.

Parameters:

• process_class – Process class whose inputs to try and retrieve
• namespace (str) – PortNamespace in which to look for the inputs
• agglomerate (bool) – If set to true, all parent namespaces of the given namespace will also be searched for inputs. Inputs in lower-lying namespaces take precedence.

exposed_outputs(process_instance, process_class, namespace=None, agglomerate=True)

Gather the outputs which were exposed from the process_class and emitted by the specific process_instance in a dictionary.

Parameters:

• namespace (str) – Namespace in which to search for exposed outputs.
• agglomerate (bool) – If set to true, all parent namespaces of the given namespace will also be searched for outputs. Outputs in lower-lying namespaces take precedence.

classmethod get_builder()

classmethod get_or_create_db_record()

Create a database calculation node that represents what happened in this process. :return: A calculation

get_parent_calc()

Get the parent process node

Returns:the parent process node if there is one Return type:aiida.orm.nodes.process.process.ProcessNode

get_provenance_inputs_iterator()

init()

kill(msg=None)

Kill the process and all the children calculations it called

load_instance_state(saved_state, load_context)

metadata

Return the metadata passed when launching this process.

Returns:metadata dictionary

node

Return the ProcessNode used by this process to represent itself in the database.

Returns:instance of sub class of ProcessNode

on_create()

on_entered(from_state)

on_entering(state)

on_except(exc_info)

Log the exception by calling the report method with formatted stack trace from exception info object and store the exception string as a node attribute

Parameters:exc_info – the sys.exc_info() object

on_finish(result, successful)

Set the finish status on the process node

on_output_emitting(output_port, value)

The process has emitted a value on the given output port.

Parameters:

• output_port – The output port name the value was emitted on
• value – The value emitted

on_paused(msg=None)

The Process was paused so set the paused attribute on the process node

on_playing()

The Process was unpaused so remove the paused attribute on the process node

on_terminated()

Called when a Process enters a terminal state.

options

Return the options of the metadata passed when launching this process.

Returns:options dictionary

out(output_port, value=None)

Record an output value for a specific output port. If the output port matches an explicitly defined Port it will be validated against that. If not it will be validated against the PortNamespace, which means it will be checked for dynamicity and whether the type of the value is valid

Parameters:

• output_port (str) – the name of the output port, can be namespaced
• value – the value for the output port

Raises:

TypeError if the output value is not validated against the port

out_many(out_dict)

Add all values given in out_dict to the outputs. The keys of the dictionary will be used as output names.

report(msg, *args, **kwargs)

Log a message to the logger, which should get saved to the database through the attached DbLogHandler.

The pk, class name and function name of the caller are prepended to the given message

Parameters:

• msg – message to log
• args – args to pass to the log call
• kwargs – kwargs to pass to the log call

runner

save_instance_state(out_state, save_context)

Ask the process to save its current instance state.

Parameters:

• out_state (plumpy.Bundle) – A bundle to save the state to
• save_context – The save context

set_status(status)

The status of the Process is about to be changed, so we reflect this is in node’s attribute proxy.

Parameters:status – the status message

submit(process, *args, **kwargs)

update_node_state(state)

update_outputs()

Attach any new outputs to the node since the last time this was called, if store provenance is True.

uuid

Return the UUID of the process which corresponds to the UUID of its associated ProcessNode.

Returns:the UUID associated to this process instance

class aiida.engine.ProcessState

Bases: enum.Enum

The possible states that a Process can be in.

CREATED = 'created'

EXCEPTED = 'excepted'

FINISHED = 'finished'

KILLED = 'killed'

RUNNING = 'running'

WAITING = 'waiting'

__module__ = 'plumpy.process_states'

aiida.engine.ToContext

alias of __builtin__.dict

aiida.engine.assign_(target)

Convenience function that will construct an Awaitable for a given class instance with the context action set to ASSIGN. When the awaitable target is completed it will be assigned to the context for a key that is to be defined later

Parameters:target – an instance of a Process or Awaitable Returns:the awaitable Return type:Awaitable

aiida.engine.append_(target)

Convenience function that will construct an Awaitable for a given class instance with the context action set to APPEND. When the awaitable target is completed it will be appended to a list in the context for a key that is to be defined later

Parameters:target – an instance of a Process or Awaitable Returns:the awaitable Return type:Awaitable

class aiida.engine.WorkChain(inputs=None, logger=None, runner=None, enable_persistence=True)

Bases: aiida.engine.processes.process.Process

A WorkChain, the base class for AiiDA workflows.

_CONTEXT = 'CONTEXT'

_STEPPER_STATE = 'stepper_state'

__abstractmethods__ = frozenset([])

__init__(inputs=None, logger=None, runner=None, enable_persistence=True)

Construct the instance only if it is a sub class of WorkChain otherwise raise InvalidOperation.

__module__ = 'aiida.engine.processes.workchains.workchain'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 102

_abc_registry = <_weakrefset.WeakSet object>

_auto_persist = set(['_CREATION_TIME', '_awaitables', '_enable_persistence', '_future', '_parent_pid', '_paused', '_pid', '_pre_paused_status', '_status'])

_do_step()

Execute the next step in the outline and return the result.

If the stepper returns a non-finished status and the return value is of type ToContext, the contents of the ToContext container will be turned into awaitables if necessary. If any awaitables were created, the process will enter in the Wait state, otherwise it will go to Continue. When the stepper returns that it is done, the stepper result will be converted to None and returned, unless it is an integer or instance of ExitCode.

_node_class

alias of aiida.orm.nodes.process.workflow.workchain.WorkChainNode

_spec_type

alias of WorkChainSpec

_store_nodes(data)

Recurse through a data structure and store any unstored nodes that are found along the way

Parameters:data – a data structure potentially containing unstored nodes

action_awaitables()

Handle the awaitables that are currently registered with the workchain

Depending on the class type of the awaitable’s target a different callback function will be bound with the awaitable and the runner will be asked to call it when the target is completed

ctx

classmethod define(spec)

insert_awaitable(awaitable)

Insert a awaitable that will cause the workchain to wait until the wait on is finished before continuing to the next step.

Parameters:awaitable (aiida.engine.processes.workchains.awaitable.Awaitable) – The thing to await

load_instance_state(saved_state, load_context)

on_exiting()

Ensure that any unstored nodes in the context are stored, before the state is exited

After the state is exited the next state will be entered and if persistence is enabled, a checkpoint will be saved. If the context contains unstored nodes, the serialization necessary for checkpointing will fail.

on_process_finished(awaitable, pk)

Callback function called by the runner when the process instance identified by pk is completed. The awaitable will be effectuated on the context of the workchain and removed from the internal list. If all awaitables have been dealt with, the workchain process is resumed

Parameters:

• awaitable – an Awaitable instance
• pk – the pk of the awaitable’s target

on_run()

on_wait(awaitables)

remove_awaitable(awaitable)

Remove a awaitable.

Precondition: must be a awaitable that was previously inserted

Parameters:awaitable – The awaitable to remove

run()

save_instance_state(out_state, save_context)

Ask the process to save its current instance state.

Parameters:

• out_state (plumpy.Bundle) – A bundle to save the state to
• save_context – The save context

to_context(**kwargs)

This is a convenience method that provides syntactic sugar, for a user to add multiple intersteps that will assign a certain value to the corresponding key in the context of the workchain

aiida.engine.if_(condition)

A conditional that can be used in a workchain outline.

Use as:

if_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters:condition – The workchain method that will return True or False

aiida.engine.while_(condition)

A while loop that can be used in a workchain outline.

Use as:

while_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters:condition – The workchain method that will return True or False

Subpackages

• aiida.engine.daemon package
  • Submodules
• aiida.engine.processes package
  • Subpackages
    • aiida.engine.processes.calcjobs package
      • Submodules
    • aiida.engine.processes.workchains package
      • Submodules
  • Submodules

Submodules

Exceptions that can be thrown by parts of the workflow engine.

exception aiida.engine.exceptions.PastException

Bases: aiida.common.exceptions.AiidaException

Raised when an attempt is made to continue a Process that has already excepted before.

__module__ = 'aiida.engine.exceptions'

Top level functions that can be used to launch a Process.

aiida.engine.launch.run(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

the outputs of the process

aiida.engine.launch.run_get_pid(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and process pid

aiida.engine.launch.run_get_node(process, *args, **inputs)

Run the process with the supplied inputs in a local runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and the calculation node

aiida.engine.launch.submit(process, **inputs)

Submit the process with the supplied inputs to the daemon runners immediately returning control to the interpreter. The return value will be the calculation node of the submitted process.

Parameters:

• process – the process class to submit
• inputs – the inputs to be passed to the process

Returns:

the calculation node of the process

Definition of AiiDA’s process persister and the necessary object loaders.

class aiida.engine.persistence.AiiDAPersister

Bases: plumpy.persistence.Persister

This node is responsible to taking saved process instance states and persisting them to the database.

__abstractmethods__ = frozenset([])

__module__ = 'aiida.engine.persistence'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 102

_abc_registry = <_weakrefset.WeakSet object>

delete_checkpoint(pid, tag=None)

Delete a persisted process checkpoint, where no error will be raised if the checkpoint does not exist

Parameters:

• pid – the process id of the plumpy.Process
• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint

delete_process_checkpoints(pid)

Delete all persisted checkpoints related to the given process id

Parameters:pid – the process id of the aiida.engine.processes.process.Process

get_checkpoints()

Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag

Returns:list of PersistedCheckpoint tuples

get_process_checkpoints(pid)

Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag

Parameters:pid – the process pid Returns:list of PersistedCheckpoint tuples

load_checkpoint(pid, tag=None)

Load a process from a persisted checkpoint by its process id

Parameters:

• pid – the process id of the plumpy.Process
• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint

Returns:

a bundle with the process state

Return type:

plumpy.Bundle

Raises:

plumpy.PersistenceError Raised if there was a problem loading the checkpoint

save_checkpoint(process, tag=None)

Persist a Process instance

Parameters:

• process – aiida.engine.Process
• tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process

Raises:

plumpy.PersistenceError Raised if there was a problem saving the checkpoint

aiida.engine.persistence.ObjectLoader

alias of plumpy.loaders.DefaultObjectLoader

aiida.engine.persistence.get_object_loader()

Get the global AiiDA object loader

Returns:The global object loader Return type:plumpy.ObjectLoader

Runners that can run and submit processes.

class aiida.engine.runners.Runner(poll_interval=0, loop=None, communicator=None, rmq_submit=False, persister=None)

Bases: object

Class that can launch processes by running in the current interpreter or by submitting them to the daemon.

__dict__ = dict_proxy({'__exit__': <function __exit__>, '__module__': 'aiida.engine.runners', 'run': <function run>, 'run_until_complete': <function run_until_complete>, 'schedule': <function schedule>, '_closed': False, '_poll_calculation': <function _poll_calculation>, 'stop': <function stop>, '_persister': None, 'controller': <property object>, 'run_get_pid': <function run_get_pid>, 'get_calculation_future': <function get_calculation_future>, 'communicator': <property object>, '__dict__': <attribute '__dict__' of 'Runner' objects>, 'close': <function close>, '__weakref__': <attribute '__weakref__' of 'Runner' objects>, 'transport': <property object>, 'job_manager': <property object>, 'persister': <property object>, 'run_get_node': <function run_get_node>, '__enter__': <function __enter__>, 'submit': <function submit>, '_communicator': None, 'start': <function start>, '__init__': <function __init__>, 'is_closed': <function is_closed>, 'call_on_calculation_finish': <function call_on_calculation_finish>, '_run': <function _run>, '_controller': None, '__doc__': 'Class that can launch processes by running in the current interpreter or by submitting them to the daemon.', 'loop': <property object>})

__enter__()

__exit__(exc_type, exc_val, exc_tb)

__init__(poll_interval=0, loop=None, communicator=None, rmq_submit=False, persister=None)

Construct a new runner

Parameters:

• poll_interval – interval in seconds between polling for status of active calculations
• loop (tornado.ioloop.IOLoop) – an event loop to use, if none is suppled a new one will be created
• communicator (kiwipy.Communicator) – the communicator to use
• rmq_submit – if True, processes will be submitted to RabbitMQ, otherwise they will be scheduled here
• persister (plumpy.Persister) – the persister to use to persist processes

__module__ = 'aiida.engine.runners'

__weakref__

list of weak references to the object (if defined)

_closed = False

_communicator = None

_controller = None

_persister = None

_poll_calculation(calc_node, callback)

_run(process, *args, **inputs)

Run the process with the supplied inputs in this runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and the calculation node

call_on_calculation_finish(pk, callback)

Callback to be called when the calculation of the given pk is terminated

Parameters:

• pk – the pk of the calculation
• callback – the function to be called upon calculation termination

close()

Close the runner by stopping the loop.

communicator

Get the communicator used by this runner

Returns:the communicator Return type:kiwipy.Communicator

controller

get_calculation_future(pk)

Get a future for an orm Calculation. The future will have the calculation node as the result when finished.

Returns:A future representing the completion of the calculation node

is_closed()

job_manager

loop

Get the event loop of this runner

Returns:the event loop Return type:tornado.ioloop.IOLoop

persister

run(process, *args, **inputs)

Run the process with the supplied inputs in this runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

the outputs of the process

run_get_node(process, *args, **inputs)

Run the process with the supplied inputs in this runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and the calculation node

run_get_pid(process, *args, **inputs)

Run the process with the supplied inputs in this runner that will block until the process is completed. The return value will be the results of the completed process

Parameters:

• process – the process class or process function to run
• inputs – the inputs to be passed to the process

Returns:

tuple of the outputs of the process and process pid

run_until_complete(future)

Run the loop until the future has finished and return the result.

schedule(process, *args, **inputs)

Schedule a process to be executed by this runner

Parameters:

• process – the process class to submit
• inputs – the inputs to be passed to the process

Returns:

the calculation node of the process

start()

Start the internal event loop.

stop()

Stop the internal event loop.

submit(process, *args, **inputs)

Submit the process with the supplied inputs to this runner immediately returning control to the interpreter. The return value will be the calculation node of the submitted process

Parameters:

• process – the process class to submit
• inputs – the inputs to be passed to the process

Returns:

the calculation node of the process

transport

A transport queue to batch process multiple tasks that require a Transport.

class aiida.engine.transports.TransportQueue(loop=None)

Bases: object

A queue to get transport objects from authinfo. This class allows clients to register their interest in a transport object which will be provided at some point in the future.

Internally the class will wait for a specific interval at the end of which it will open the transport and give it to all the clients that asked for it up to that point. This way opening of transports (a costly operation) can be minimised.

class AuthInfoEntry(authinfo, transport, callbacks, callback_handle)

Bases: tuple

__dict__ = dict_proxy({'__module__': 'aiida.engine.transports', '_make': <classmethod object>, 'callback_handle': <property object>, 'callbacks': <property object>, '_asdict': <function _asdict>, '__dict__': <property object>, '__getnewargs__': <function __getnewargs__>, 'transport': <property object>, '_fields': ('authinfo', 'transport', 'callbacks', 'callback_handle'), '__new__': <staticmethod object>, '_replace': <function _replace>, 'authinfo': <property object>, '__slots__': (), '__repr__': <function __repr__>, '__getstate__': <function __getstate__>, '__doc__': 'AuthInfoEntry(authinfo, transport, callbacks, callback_handle)'})

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__getstate__()

Exclude the OrderedDict from pickling

__module__ = 'aiida.engine.transports'

static __new__(_cls, authinfo, transport, callbacks, callback_handle)

Create new instance of AuthInfoEntry(authinfo, transport, callbacks, callback_handle)

__repr__()

Return a nicely formatted representation string

__slots__ = ()

_asdict()

Return a new OrderedDict which maps field names to their values

_fields = ('authinfo', 'transport', 'callbacks', 'callback_handle')

classmethod _make(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)

Make a new AuthInfoEntry object from a sequence or iterable

_replace(**kwds)

Return a new AuthInfoEntry object replacing specified fields with new values

authinfo

Alias for field number 0

callback_handle

Alias for field number 3

callbacks

Alias for field number 2

transport

Alias for field number 1

__dict__ = dict_proxy({'__module__': 'aiida.engine.transports', 'request_transport': <function request_transport>, 'loop': <function loop>, '__dict__': <attribute '__dict__' of 'TransportQueue' objects>, '__weakref__': <attribute '__weakref__' of 'TransportQueue' objects>, '__doc__': '\n A queue to get transport objects from authinfo. This class allows clients\n to register their interest in a transport object which will be provided at\n some point in the future.\n\n Internally the class will wait for a specific interval at the end of which\n it will open the transport and give it to all the clients that asked for it\n up to that point. This way opening of transports (a costly operation) can\n be minimised.\n ', '__init__': <function __init__>, 'AuthInfoEntry': <class 'aiida.engine.transports.AuthInfoEntry'>})

__init__(loop=None)

Parameters:loop (tornado.ioloop.IOLoop) – The event loop to use, will use tornado.ioloop.IOLoop.current() if not supplied

__module__ = 'aiida.engine.transports'

__weakref__

list of weak references to the object (if defined)

loop()

Get the loop being used by this transport queue

request_transport(**kwds)

Request a transport from an authinfo. Because the client is not allowed to request a transport immediately they will instead be given back a future that can be yielded to get the transport:

@tornado.gen.coroutine
def transport_task(transport_queue, authinfo):
    with transport_queue.request_transport(authinfo) as request:
        transport = yield request
        # Do some work with the transport

Parameters:authinfo – The authinfo to be used to get transport Returns:A future that can be yielded to give the transport

class aiida.engine.transports.TransportRequest

Bases: object

Information kept about request for a transport object

__dict__ = dict_proxy({'__dict__': <attribute '__dict__' of 'TransportRequest' objects>, '__module__': 'aiida.engine.transports', '__weakref__': <attribute '__weakref__' of 'TransportRequest' objects>, '__doc__': ' Information kept about request for a transport object ', '__init__': <function __init__>})

__init__()

x.__init__(…) initializes x; see help(type(x)) for signature

__module__ = 'aiida.engine.transports'

__weakref__

list of weak references to the object (if defined)

Utilities for the workflow engine.

class aiida.engine.utils.RefObjectStore

Bases: object

An object store that has a reference count based on a context manager. Basic usage:

store = RefObjectStore()
with store.get('Martin', lambda: 'martin.uhrin@epfl.ch') as email:
    with store.get('Martin') as email2:
        email is email2  # True

The use case for this store is when you have an object can be used by multiple parts of the code simultaneously (nested or async code) and where there should be one instance that exists for the lifetime of these contexts. Once noone is using the object, it should be removed from the store (and therefore eventually garbage collected).

class Reference(obj)

Bases: object

A reference to store the context reference count and the object itself.

__dict__ = dict_proxy({'count': <property object>, '__module__': 'aiida.engine.utils', 'get': <function get>, '__dict__': <attribute '__dict__' of 'Reference' objects>, '__weakref__': <attribute '__weakref__' of 'Reference' objects>, '__doc__': 'A reference to store the context reference count and the object itself.', '__init__': <function __init__>})

__init__(obj)

x.__init__(…) initializes x; see help(type(x)) for signature

__module__ = 'aiida.engine.utils'

__weakref__

list of weak references to the object (if defined)

count

Get the reference count for the object

Returns:The reference count Return type:int

get(**kwds)

Get the object itself, which will up the reference count for the duration of the context.

Returns:The object

__dict__ = dict_proxy({'__module__': 'aiida.engine.utils', 'Reference': <class 'aiida.engine.utils.Reference'>, 'get': <function get>, '__dict__': <attribute '__dict__' of 'RefObjectStore' objects>, '__weakref__': <attribute '__weakref__' of 'RefObjectStore' objects>, '__doc__': "\n An object store that has a reference count based on a context manager.\n Basic usage::\n\n store = RefObjectStore()\n with store.get('Martin', lambda: 'martin.uhrin@epfl.ch') as email:\n with store.get('Martin') as email2:\n email is email2 # True\n\n The use case for this store is when you have an object can be used by\n multiple parts of the code simultaneously (nested or async code) and\n where there should be one instance that exists for the lifetime of these\n contexts. Once noone is using the object, it should be removed from the\n store (and therefore eventually garbage collected).\n ", '__init__': <function __init__>})

__init__()

x.__init__(…) initializes x; see help(type(x)) for signature

__module__ = 'aiida.engine.utils'

__weakref__

list of weak references to the object (if defined)

get(**kwds)

Get or create an object. The internal reference count will be upped for the duration of the context. If the reference count drops to 0 the object will be automatically removed from the list.

Parameters:

• identifier – The key identifying the object
• constructor – An optional constructor that is called with no arguments if the object doesn’t already exist in the store

Returns:

The object corresponding to the identifier

aiida.engine.utils.interruptable_task(coro, loop=None)

Turn the given coroutine into an interruptable task by turning it into an InterruptableFuture and returning it.

Parameters:

• coro – the coroutine that should be made interruptable
• loop – the event loop in which to run the coroutine, by default uses tornado.ioloop.IOLoop.current()

Returns:

an InterruptableFuture

class aiida.engine.utils.InterruptableFuture

Bases: tornado.concurrent.Future

A future that can be interrupted by calling interrupt.

__module__ = 'aiida.engine.utils'

interrupt(reason)

This method should be called to interrupt the coroutine represented by this InterruptableFuture.

with_interrupt(**kwargs)

Yield a yieldable which will be interrupted if this future is interrupted

from tornado import ioloop, gen
loop = ioloop.IOLoop.current()

interruptable = InterutableFuture()
loop.add_callback(interruptable.interrupt, RuntimeError("STOP"))
loop.run_sync(lambda: interruptable.with_interrupt(gen.sleep(2)))
>>> RuntimeError: STOP

Parameters:yieldable – The yieldable Returns:The result of the yieldable