aiida.engine.daemon package¶
Submodules¶
Controls the daemon
-
class
aiida.engine.daemon.client.
ControllerProtocol
[source]¶ Bases:
enum.Enum
The protocol to use to for the controller of the Circus daemon
-
IPC
= 0¶
-
TCP
= 1¶
-
__module__
= 'aiida.engine.daemon.client'¶
-
-
class
aiida.engine.daemon.client.
DaemonClient
(profile)[source]¶ Bases:
object
Extension of the Profile which also provides handles to retrieve profile specific properties related to the daemon client
-
DAEMON_ERROR_NOT_RUNNING
= 'daemon-error-not-running'¶
-
DAEMON_ERROR_TIMEOUT
= 'daemon-error-timeout'¶
-
_DAEMON_NAME
= 'aiida-{name}'¶
-
_DEFAULT_LOGLEVEL
= 'INFO'¶
-
_ENDPOINT_PROTOCOL
= 0¶
-
__dict__
= dict_proxy({'__module__': 'aiida.engine.daemon.client', 'loglevel': <property object>, 'get_stats_endpoint': <function get_stats_endpoint>, 'get_worker_info': <function get_worker_info>, 'get_tcp_endpoint': <function get_tcp_endpoint>, 'get_ipc_endpoint': <function get_ipc_endpoint>, '__dict__': <attribute '__dict__' of 'DaemonClient' objects>, 'get_status': <function get_status>, 'daemon_log_file': <property object>, '_DEFAULT_LOGLEVEL': 'INFO', 'get_circus_port': <function get_circus_port>, '__weakref__': <attribute '__weakref__' of 'DaemonClient' objects>, '__init__': <function __init__>, 'virtualenv': <property object>, 'circus_port_file': <property object>, 'circus_pid_file': <property object>, 'decrease_workers': <function decrease_workers>, 'get_controller_endpoint': <function get_controller_endpoint>, 'DAEMON_ERROR_TIMEOUT': 'daemon-error-timeout', 'increase_workers': <function increase_workers>, 'circus_socket_file': <property object>, 'circus_socket_endpoints': <property object>, 'stop_daemon': <function stop_daemon>, 'get_circus_socket_directory': <function get_circus_socket_directory>, 'profile': <property object>, 'delete_circus_socket_directory': <function delete_circus_socket_directory>, 'get_pubsub_endpoint': <function get_pubsub_endpoint>, 'call_client': <function call_client>, 'daemon_name': <property object>, 'cmd_string': <property object>, 'circus_log_file': <property object>, 'DAEMON_ERROR_NOT_RUNNING': 'daemon-error-not-running', '_DAEMON_NAME': 'aiida-{name}', 'is_daemon_running': <property object>, 'get_available_port': <classmethod object>, 'daemon_pid_file': <property object>, '__doc__': '\n Extension of the Profile which also provides handles to retrieve profile specific\n properties related to the daemon client\n ', '_ENDPOINT_PROTOCOL': <ControllerProtocol.IPC: 0>, 'client': <property object>, 'get_daemon_info': <function get_daemon_info>, 'restart_daemon': <function restart_daemon>, 'get_daemon_pid': <function get_daemon_pid>})¶
-
__init__
(profile)[source]¶ Construct a DaemonClient instance for a given profile
Parameters: profile – the profile instance aiida.manage.configuration.profile.Profile
-
__module__
= 'aiida.engine.daemon.client'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
call_client
(command)[source]¶ Call the client with a specific command. Will check whether the daemon is running first by checking for the pid file. When the pid is found yet the call still fails with a timeout, this means the daemon was actually not running and it was terminated unexpectedly causing the pid file to not be cleaned up properly
Parameters: command – command to call the circus client with Returns: the result of the circus client call
-
circus_log_file
¶
-
circus_pid_file
¶
-
circus_port_file
¶
-
circus_socket_endpoints
¶
-
circus_socket_file
¶
-
client
¶ Return an instance of the CircusClient with the endpoint defined by the controller endpoint, which used the port that was written to the port file upon starting of the daemon
Returns: CircucClient instance
-
cmd_string
¶ Return the command string to start the AiiDA daemon
-
daemon_log_file
¶
-
daemon_name
¶ Get the daemon name which is tied to the profile name
-
daemon_pid_file
¶
-
decrease_workers
(number)[source]¶ Decrease the number of workers
Parameters: number – the number of workers to remove Returns: the client call response
-
delete_circus_socket_directory
()[source]¶ Attempt to delete the directory used to store the circus endpoint sockets. Will not raise if the directory does not exist
-
classmethod
get_available_port
()[source]¶ Get an available port from the operating system
Returns: a currently available port
-
get_circus_port
()[source]¶ Retrieve the port for the circus controller, which should be written to the circus port file. If the daemon is running, the port file should exist and contain the port to which the controller is connected. If it cannot be read, a RuntimeError will be thrown. If the daemon is not running, an available port will be requested from the operating system, written to the port file and returned
Returns: the port for the circus controller
-
get_circus_socket_directory
()[source]¶ Retrieve the absolute path of the directory where the circus sockets are stored if the IPC protocol is used and the daemon is running. If the daemon is running, the sockets file should exist and contain the absolute path of the directory that contains the sockets of the circus endpoints. If it cannot be read, a RuntimeError will be thrown. If the daemon is not running, a temporary directory will be created and its path will be written to the sockets file and returned.
Note
A temporary folder needs to be used for the sockets because UNIX limits the filepath length to 107 bytes. Placing the socket files in the AiiDA config folder might seem like the more logical choice but that folder can be placed in an arbitrarily nested directory, the socket filename will exceed the limit. The solution is therefore to always store them in the temporary directory of the operation system whose base path is typically short enough as to not exceed the limit
Returns: the absolute path of directory to write the sockets to
-
get_controller_endpoint
()[source]¶ Get the endpoint string for the circus controller. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol an available port will be found and saved in the profile specific port file
Returns: the endpoint string
-
get_daemon_pid
()[source]¶ Get the daemon pid which should be written in the daemon pid file specific to the profile
Returns: the pid of the circus daemon process or None if not found
-
get_ipc_endpoint
(endpoint)[source]¶ Get the ipc endpoint string for a circus daemon endpoint for a given socket
Parameters: endpoint – the circus endpoint for which to return a socket Returns: the ipc endpoint string
-
get_pubsub_endpoint
()[source]¶ Get the endpoint string for the circus pubsub endpoint. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol any available port will be used
Returns: the endpoint string
-
get_stats_endpoint
()[source]¶ Get the endpoint string for the circus stats endpoint. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol any available port will be used
Returns: the endpoint string
-
get_tcp_endpoint
(port=None)[source]¶ Get the tcp endpoint string for a circus daemon endpoint. If the port is unspecified, the operating system will be asked for a currently available port.
Parameters: port – a port to use for the endpoint Returns: the tcp endpoint string
-
increase_workers
(number)[source]¶ Increase the number of workers
Parameters: number – the number of workers to add Returns: the client call response
-
is_daemon_running
¶ Return whether the daemon is running, which is determined by seeing if the daemon pid file is present
Returns: True if daemon is running, False otherwise
-
loglevel
¶
-
profile
¶
-
restart_daemon
(wait)[source]¶ Restart the daemon
Parameters: wait – boolean to indicate whether to wait for the result of the command Returns: the client call response
-
stop_daemon
(wait)[source]¶ Stop the daemon
Parameters: wait – boolean to indicate whether to wait for the result of the command Returns: the client call response
-
virtualenv
¶
-
-
aiida.engine.daemon.client.
get_daemon_client
(profile_name=None)[source]¶ Return the daemon client for the given profile or the current profile if not specified.
Parameters: profile_name – the profile name, will use the current profile if None
Returns: the daemon client
Return type: Raises: - aiida.common.MissingConfigurationError – if the configuration file cannot be found
- aiida.common.ProfileConfigurationError – if the given profile does not exist
This file contains the main routines to submit, check and retrieve calculation results. These are general and contain only the main logic; where appropriate, the routines make reference to the suitable plugins for all plugin-specific operations.
-
aiida.engine.daemon.execmanager.
_retrieve_singlefiles
(job, transport, folder, retrieve_file_list, logger_extra=None)[source]¶
-
aiida.engine.daemon.execmanager.
kill_calculation
(calculation, transport)[source]¶ Kill the calculation through the scheduler
Parameters: - calculation – the instance of CalcJobNode to kill.
- transport – an already opened transport to use to address the scheduler
-
aiida.engine.daemon.execmanager.
parse_results
(process, retrieved_temporary_folder=None)[source]¶ Parse the results for a given CalcJobNode (job)
Returns: integer exit code, where 0 indicates success and non-zero failure
-
aiida.engine.daemon.execmanager.
retrieve_calculation
(calculation, transport, retrieved_temporary_folder)[source]¶ Retrieve all the files of a completed job calculation using the given transport.
If the job defined anything in the retrieve_temporary_list, those entries will be stored in the retrieved_temporary_folder. The caller is responsible for creating and destroying this folder.
Parameters: - calculation – the instance of CalcJobNode to update.
- transport – an already opened transport to use for the retrieval.
- retrieved_temporary_folder – the absolute path to a directory in which to store the files listed, if any, in the retrieved_temporary_folder of the jobs CalcInfo
-
aiida.engine.daemon.execmanager.
retrieve_files_from_list
(calculation, transport, folder, retrieve_list)[source]¶ Retrieve all the files in the retrieve_list from the remote into the local folder instance through the transport. The entries in the retrieve_list can be of two types:
- a string
- a list
If it is a string, it represents the remote absolute filepath of the file. If the item is a list, the elements will correspond to the following:
- remotepath
- localpath
- depth
If the remotepath contains file patterns with wildcards, the localpath will be treated as the work directory of the folder and the depth integer determines upto what level of the original remotepath nesting the files will be copied.
Parameters: - transport – the Transport instance
- folder – an absolute path to a folder to copy files in
- retrieve_list – the list of files to retrieve
-
aiida.engine.daemon.execmanager.
submit_calculation
(calculation, transport, calc_info, script_filename)[source]¶ Submit a calculation
Parameters: - calculation – the instance of CalcJobNode to submit.
- transport – an already opened transport to use to submit the calculation.
- calc_info – the calculation info datastructure returned by CalcJobNode._presubmit
- script_filename – the job launch script returned by CalcJobNode._presubmit
-
aiida.engine.daemon.execmanager.
upload_calculation
(node, transport, calc_info, script_filename)[source]¶ Upload a CalcJob instance
Parameters: - node – the CalcJobNode.
- transport – an already opened transport to use to submit the calculation.
- calc_info – the calculation info datastructure returned by CalcJobNode.presubmit
- script_filename – the job launch script returned by CalcJobNode.presubmit
Function that starts a daemon runner.