aiida.manage package#
Managing an AiiDA instance:
configuration file
profiles
databases
repositories
external components (such as Postgres, RabbitMQ)
Note
Modules in this sub package may require the database environment to be loaded
Subpackages#
Submodules#
Definition of caching mechanism and configuration for calculations.
- class aiida.manage.caching.ConfigKeys(value)[source]#
Bases:
enum.Enum
Valid keys for caching configuration.
- DEFAULT = 'caching.default_enabled'#
- DISABLED = 'caching.disabled_for'#
- ENABLED = 'caching.enabled_for'#
- __module__ = 'aiida.manage.caching'#
- class aiida.manage.caching._ContextCache[source]#
Bases:
object
Cache options, accounting for when in enable_caching or disable_caching contexts.
- __dict__ = mappingproxy({'__module__': 'aiida.manage.caching', '__doc__': 'Cache options, accounting for when in enable_caching or disable_caching contexts.', '__init__': <function _ContextCache.__init__>, 'clear': <function _ContextCache.clear>, 'enable_all': <function _ContextCache.enable_all>, 'disable_all': <function _ContextCache.disable_all>, 'enable': <function _ContextCache.enable>, 'disable': <function _ContextCache.disable>, 'get_options': <function _ContextCache.get_options>, '__dict__': <attribute '__dict__' of '_ContextCache' objects>, '__weakref__': <attribute '__weakref__' of '_ContextCache' objects>, '__annotations__': {}})#
- __module__ = 'aiida.manage.caching'#
- __weakref__#
list of weak references to the object (if defined)
- aiida.manage.caching._match_wildcard(*, string: str, pattern: str) bool [source]#
Return whether a given name matches a pattern which can contain ‘*’ wildcards.
- Parameters
string – The string to check.
pattern – The patter to match for.
- Returns
True
ifstring
matches thepattern
,False
otherwise.
- aiida.manage.caching._validate_identifier_pattern(*, identifier: str, strict: bool = False)[source]#
Validate an caching identifier pattern.
The identifier (without wildcards) can have one of two forms:
<group_name><ENTRY_POINT_STRING_SEPARATOR><tail>
where group_name is one of the keys in ENTRY_POINT_GROUP_TO_MODULE_PATH_MAP and tail can be anything _except_ ENTRY_POINT_STRING_SEPARATOR.
a fully qualified Python name
this is a colon-separated string, where each part satisfies part.isidentifier() and not keyword.iskeyword(part)
This function checks if an identifier _with_ wildcards can possibly match one of these two forms. If it can not, a
ValueError
is raised.- Parameters
identifier – Process type string, or a pattern with ‘*’ wildcard that matches it.
strict – When set to
True
, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.
- Raises
ValueError – If the identifier is an invalid identifier.
ValueError – If
strict=True
and the identifier cannot be successfully loaded.
- aiida.manage.caching.disable_caching(*, identifier: str | None = None, strict: bool = False)[source]#
Context manager to disable caching, either for a specific node class, or globally.
Warning
this does not affect the behavior of the daemon, only the local Python interpreter.
- Parameters
identifier (str) – Process type string of the node, or a pattern with ‘*’ wildcard that matches it. If not provided, caching is disabled for all classes.
strict – When set to
True
, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.
- aiida.manage.caching.enable_caching(*, identifier: str | None = None, strict: bool = False)[source]#
Context manager to enable caching, either for a specific node class, or globally.
Warning
this does not affect the behavior of the daemon, only the local Python interpreter.
- Parameters
identifier (str) – Process type string of the node, or a pattern with ‘*’ wildcard that matches it. If not provided, caching is enabled for all classes.
strict – When set to
True
, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.
- aiida.manage.caching.get_use_cache(*, identifier: Optional[str] = None, strict: bool = False) bool [source]#
Return whether the caching mechanism should be used for the given process type according to the configuration.
- Parameters
identifier – Process type string of the node
strict – When set to
True
, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.
- Returns
boolean, True if caching is enabled, False otherwise
- Raises
~aiida.common.exceptions.ConfigurationError if the configuration is invalid, either due to a general configuration error, or by defining the class both enabled and disabled
AiiDA manager for global settings
- class aiida.manage.manager.Manager[source]#
Bases:
object
Manager singleton for globally loaded resources.
AiiDA can have the following global resources loaded:
A single configuration object that contains:
Global options overrides
The name of a default profile
A mapping of profile names to their configuration and option overrides
A single profile object that contains:
The name of the profile
The UUID of the profile
The configuration of the profile, for connecting to storage and processing resources
The option overrides for the profile
3. A single storage backend object for the profile, to connect to data storage resources 5. A single daemon client object for the profile, to connect to the AiiDA daemon 4. A single communicator object for the profile, to connect to the process control resources 6. A single process controller object for the profile, which uses the communicator to control process tasks 7. A single runner object for the profile, which uses the process controller to start and stop processes 8. A single persister object for the profile, which can persist running processes to the profile storage
- __dict__ = mappingproxy({'__module__': 'aiida.manage.manager', '__doc__': 'Manager singleton for globally loaded resources.\n\n AiiDA can have the following global resources loaded:\n\n 1. A single configuration object that contains:\n\n - Global options overrides\n - The name of a default profile\n - A mapping of profile names to their configuration and option overrides\n\n 2. A single profile object that contains:\n\n - The name of the profile\n - The UUID of the profile\n - The configuration of the profile, for connecting to storage and processing resources\n - The option overrides for the profile\n\n 3. A single storage backend object for the profile, to connect to data storage resources\n 5. A single daemon client object for the profile, to connect to the AiiDA daemon\n 4. A single communicator object for the profile, to connect to the process control resources\n 6. A single process controller object for the profile, which uses the communicator to control process tasks\n 7. A single runner object for the profile, which uses the process controller to start and stop processes\n 8. A single persister object for the profile, which can persist running processes to the profile storage\n\n ', '__init__': <function Manager.__init__>, 'get_config': <staticmethod(<function Manager.get_config>)>, 'get_profile': <function Manager.get_profile>, 'load_profile': <function Manager.load_profile>, 'reset_profile': <function Manager.reset_profile>, 'reset_profile_storage': <function Manager.reset_profile_storage>, 'reset_communicator': <function Manager.reset_communicator>, 'reset_runner': <function Manager.reset_runner>, 'unload_profile': <function Manager.unload_profile>, 'profile_storage_loaded': <property object>, 'get_option': <function Manager.get_option>, 'get_backend': <function Manager.get_backend>, 'get_profile_storage': <function Manager.get_profile_storage>, 'get_persister': <function Manager.get_persister>, 'get_communicator': <function Manager.get_communicator>, 'create_communicator': <function Manager.create_communicator>, 'get_daemon_client': <function Manager.get_daemon_client>, 'get_process_controller': <function Manager.get_process_controller>, 'get_runner': <function Manager.get_runner>, 'set_runner': <function Manager.set_runner>, 'create_runner': <function Manager.create_runner>, 'create_daemon_runner': <function Manager.create_daemon_runner>, 'check_rabbitmq_version': <function Manager.check_rabbitmq_version>, 'check_version': <function Manager.check_version>, '__dict__': <attribute '__dict__' of 'Manager' objects>, '__weakref__': <attribute '__weakref__' of 'Manager' objects>, '__annotations__': {'_profile': "Optional['Profile']", '_profile_storage': "Optional['StorageBackend']", '_daemon_client': "Optional['DaemonClient']", '_communicator': "Optional['RmqThreadCommunicator']", '_process_controller': "Optional['RemoteProcessThreadController']", '_persister': "Optional['AiiDAPersister']", '_runner': "Optional['Runner']"}})#
- __module__ = 'aiida.manage.manager'#
- __weakref__#
list of weak references to the object (if defined)
- check_rabbitmq_version(communicator: RmqThreadCommunicator)[source]#
Check the version of RabbitMQ that is being connected to and emit warning if it is not compatible.
- check_version()[source]#
Check the currently installed version of
aiida-core
and warn if it is a post release development version.The
aiida-core
package maintains the protocol that themain
branch will use a post release version number. This means it will always append .post0 to the version of the latest release. This should mean that if this protocol is maintained properly, this method will print a warning if the currently installed version is a post release development branch and not an actual release.
- create_communicator(task_prefetch_count: Optional[int] = None) RmqThreadCommunicator [source]#
Create a Communicator.
- Parameters
task_prefetch_count – optional specify how many tasks this communicator take simultaneously
- Returns
the communicator instance
- create_daemon_runner(loop: Optional[asyncio.AbstractEventLoop] = None) Runner [source]#
Create and return a new daemon runner.
This is used by workers when the daemon is running and in testing.
- Parameters
loop – the (optional) asyncio event loop to use
- Returns
a runner configured to work in the daemon configuration
- create_runner(with_persistence: bool = True, **kwargs: Any) Runner [source]#
Create and return a new runner
- Parameters
with_persistence – create a runner with persistence enabled
- Returns
a new runner instance
- get_backend() StorageBackend [source]#
Return the current profile’s storage backend, loading it if necessary.
Deprecated: use get_profile_storage instead.
- get_communicator() RmqThreadCommunicator [source]#
Return the communicator
- Returns
a global communicator instance
- static get_config(create=False) Config [source]#
Return the current config.
- Returns
current loaded config instance
- Raises
aiida.common.ConfigurationError – if the configuration file could not be found, read or deserialized
- get_daemon_client() DaemonClient [source]#
Return the daemon client for the current profile.
- Returns
the daemon client
- Raises
aiida.common.MissingConfigurationError – if the configuration file cannot be found
aiida.common.ProfileConfigurationError – if the given profile does not exist
- get_option(option_name: str) Any [source]#
Return the value of a configuration option.
In order of priority, the option is returned from:
The current profile, if loaded and the option specified
The current configuration, if loaded and the option specified
The default value for the option
- Parameters
option_name – the name of the option to return
- Returns
the value of the option
- Raises
aiida.common.exceptions.ConfigurationError – if the option is not found
- get_persister() AiiDAPersister [source]#
Return the persister
- Returns
the current persister instance
- get_process_controller() RemoteProcessThreadController [source]#
Return the process controller
- Returns
the process controller instance
- get_profile() Optional[Profile] [source]#
Return the current loaded profile, if any
- Returns
current loaded profile instance
- get_profile_storage() StorageBackend [source]#
Return the current profile’s storage backend, loading it if necessary.
- get_runner(**kwargs) Runner [source]#
Return a runner that is based on the current profile settings and can be used globally by the code.
- Returns
the global runner
- load_profile(profile: Union[None, str, Profile] = None, allow_switch=False) Profile [source]#
Load a global profile, unloading any previously loaded profile.
Note
If a profile is already loaded and no explicit profile is specified, nothing will be done.
- Parameters
profile – the name of the profile to load, by default will use the one marked as default in the config
allow_switch – if True, will allow switching to a different profile when storage is already loaded
- Returns
the loaded Profile instance
- Raises
aiida.common.exceptions.InvalidOperation – if another profile has already been loaded and allow_switch is False
- property profile_storage_loaded: bool#
Return whether a storage backend has been loaded.
- Returns
boolean, True if database backend is currently loaded, False otherwise
- reset_profile_storage() None [source]#
Reset the profile storage.
This will close any connections to the services used by the storage, such as database connections.
- aiida.manage.manager.get_manager() aiida.manage.manager.Manager [source]#
Return the AiiDA global manager instance.
- aiida.manage.manager.get_rabbitmq_version(communicator: RmqThreadCommunicator)[source]#
Return the version of the RabbitMQ server that the current profile connects to.
- Returns
packaging.version.Version
- aiida.manage.manager.is_rabbitmq_version_supported(communicator: RmqThreadCommunicator) bool [source]#
Return whether the version of RabbitMQ configured for the current profile is supported.
Versions 3.5 and below are not supported at all, whereas versions 3.8.15 and above are not compatible with a default configuration of the RabbitMQ server.
- Returns
boolean whether the current RabbitMQ version is supported.
Module for the ProfileAccessManager that tracks process access to the profile.
- class aiida.manage.profile_access.ProfileAccessManager(profile: aiida.manage.configuration.profile.Profile)[source]#
Bases:
object
Class to manage access to a profile.
Any process that wants to request access for a given profile, should first call:
ProfileAccessManager(profile).request_access()
If this returns normally, the profile can be used safely. It will raise if it is locked, in which case the profile should not be used. If a process wants to request exclusive access to the profile, it should use
lock
:- with ProfileAccessManager(profile).lock():
pass
If the profile is already locked or is currently in use, an exception is raised.
Access and locks of the profile will be recorded in a directory with files with a
.pid
and.lock
extension, respectively. In principle then, at any one time, there can either be a number of pid files, or just a single lock file. If there is a mixture or there are more than one lock files, we are in an inconsistent state.- __dict__ = mappingproxy({'__module__': 'aiida.manage.profile_access', '__doc__': 'Class to manage access to a profile.\n\n Any process that wants to request access for a given profile, should first call:\n\n ProfileAccessManager(profile).request_access()\n\n If this returns normally, the profile can be used safely. It will raise if it is locked, in which case the profile\n should not be used. If a process wants to request exclusive access to the profile, it should use ``lock``:\n\n with ProfileAccessManager(profile).lock():\n pass\n\n If the profile is already locked or is currently in use, an exception is raised.\n\n Access and locks of the profile will be recorded in a directory with files with a ``.pid`` and ``.lock`` extension,\n respectively. In principle then, at any one time, there can either be a number of pid files, or just a single lock\n file. If there is a mixture or there are more than one lock files, we are in an inconsistent state.\n ', '__init__': <function ProfileAccessManager.__init__>, 'request_access': <function ProfileAccessManager.request_access>, 'lock': <function ProfileAccessManager.lock>, 'is_locked': <function ProfileAccessManager.is_locked>, 'is_active': <function ProfileAccessManager.is_active>, 'clear_locks': <function ProfileAccessManager.clear_locks>, '_clear_stale_pid_files': <function ProfileAccessManager._clear_stale_pid_files>, '_get_tracking_files': <function ProfileAccessManager._get_tracking_files>, '_raise_if_locked': <function ProfileAccessManager._raise_if_locked>, '_raise_if_active': <function ProfileAccessManager._raise_if_active>, '__dict__': <attribute '__dict__' of 'ProfileAccessManager' objects>, '__weakref__': <attribute '__weakref__' of 'ProfileAccessManager' objects>, '__annotations__': {}})#
- __init__(profile: aiida.manage.configuration.profile.Profile)[source]#
Class that manages access and locks to the given profile.
- Parameters
profile – the profile whose access to manage.
- __module__ = 'aiida.manage.profile_access'#
- __weakref__#
list of weak references to the object (if defined)
- _get_tracking_files(ext_string: str, exclude_self: bool = False) List[pathlib.Path] [source]#
Return a list of all files that track the accessing and locking of the profile.
- Parameters
ext_string – To get the files that track locking use .lock, to get the files that track access use .pid.
exclude_self – If true removes from the returned list any tracking to the current process.
- _raise_if_active(message_start)[source]#
Raise a
LockingProfileError
if the profile is being accessed.- Parameters
message_start – Text to use as the start of the exception message.
- Raises
LockingProfileError – if the profile is active.
- _raise_if_locked(message_start)[source]#
Raise a
LockedProfileError
if the profile is locked.- Parameters
message_start – Text to use as the start of the exception message.
- Raises
LockedProfileError – if the profile is locked.
- clear_locks() None [source]#
Clear all locks on this profile.
Warning
This should only be used if the profile is currently still incorrectly locked because the lock was not automatically released after the
lock
contextmanager exited its scope.
- lock()[source]#
Request a lock on the profile for exclusive access.
This context manager should be used if exclusive access to the profile is required. Access will be granted if the profile is currently not in use, nor locked by another process. During the context, the profile will be locked, which will be lifted automatically as soon as the context exits.
- Raises
LockingProfileError – if there are currently active processes using the profile.
LockedProfileError – if there currently already is a lock on the profile.
- request_access() None [source]#
Request access to the profile.
If the profile is locked, a
LockedProfileError
is raised. Otherwise a PID file is created for this process and the function returnsNone
. The PID file contains the command of the process.- Raises
LockedProfileError – if the profile is locked.