aiida.manage package#

Managing an AiiDA instance:

configuration file

profiles

databases

repositories

external components (such as Postgres, RabbitMQ)

Note

Modules in this sub package may require the database environment to be loaded

Subpackages#

aiida.manage.configuration package
- Subpackages
  - aiida.manage.configuration.migrations package
    - Submodules
  - aiida.manage.configuration.schema package
- Submodules
aiida.manage.external package
- Subpackages
  - aiida.manage.external.rmq package
    - Submodules
- Submodules
aiida.manage.tests package
- Submodules

Submodules#

Definition of caching mechanism and configuration for calculations.

class aiida.manage.caching.ConfigKeys(value)[source]#

Bases: enum.Enum

Valid keys for caching configuration.

DEFAULT = 'caching.default_enabled'#

DISABLED = 'caching.disabled_for'#

ENABLED = 'caching.enabled_for'#

__module__ = 'aiida.manage.caching'#

class aiida.manage.caching._ContextCache[source]#

Bases: object

Cache options, accounting for when in enable_caching or disable_caching contexts.

__dict__ = mappingproxy({'__module__': 'aiida.manage.caching', '__doc__': 'Cache options, accounting for when in enable_caching or disable_caching contexts.', '__init__': <function _ContextCache.__init__>, 'clear': <function _ContextCache.clear>, 'enable_all': <function _ContextCache.enable_all>, 'disable_all': <function _ContextCache.disable_all>, 'enable': <function _ContextCache.enable>, 'disable': <function _ContextCache.disable>, 'get_options': <function _ContextCache.get_options>, '__dict__': <attribute '__dict__' of '_ContextCache' objects>, '__weakref__': <attribute '__weakref__' of '_ContextCache' objects>, '__annotations__': {}})#

__init__()[source]#

__module__ = 'aiida.manage.caching'#

__weakref__#: list of weak references to the object (if defined)

clear()[source]#: Clear caching overrides.

disable(identifier: str)[source]#

disable_all()[source]#

enable(identifier: str)[source]#

enable_all()[source]#

get_options(strict: bool = False)[source]#

Return the options, applying any context overrides.

Parameters: strict – When set to True, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.

aiida.manage.caching._match_wildcard(*, string: str, pattern: str) → bool[source]#

Return whether a given name matches a pattern which can contain ‘*’ wildcards.

Parameters

string – The string to check.
pattern – The patter to match for.

Returns

True if string matches the pattern, False otherwise.

aiida.manage.caching._validate_identifier_pattern(*, identifier: str, strict: bool = False)[source]#

Validate an caching identifier pattern.

The identifier (without wildcards) can have one of two forms:

<group_name><ENTRY_POINT_STRING_SEPARATOR><tail>

where group_name is one of the keys in ENTRY_POINT_GROUP_TO_MODULE_PATH_MAP and tail can be anything _except_ ENTRY_POINT_STRING_SEPARATOR.
a fully qualified Python name

this is a colon-separated string, where each part satisfies part.isidentifier() and not keyword.iskeyword(part)

This function checks if an identifier _with_ wildcards can possibly match one of these two forms. If it can not, a ValueError is raised.

Parameters

identifier – Process type string, or a pattern with ‘*’ wildcard that matches it.
strict – When set to True, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.

Raises

ValueError – If the identifier is an invalid identifier.
ValueError – If strict=True and the identifier cannot be successfully loaded.

aiida.manage.caching.disable_caching(*, identifier: str | None = None, strict: bool = False)[source]#

Context manager to disable caching, either for a specific node class, or globally.

Warning

this does not affect the behavior of the daemon, only the local Python interpreter.

Parameters

identifier (str) – Process type string of the node, or a pattern with ‘*’ wildcard that matches it. If not provided, caching is disabled for all classes.
strict – When set to True, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.

aiida.manage.caching.enable_caching(*, identifier: str | None = None, strict: bool = False)[source]#

Context manager to enable caching, either for a specific node class, or globally.

Warning

this does not affect the behavior of the daemon, only the local Python interpreter.

Parameters

identifier (str) – Process type string of the node, or a pattern with ‘*’ wildcard that matches it. If not provided, caching is enabled for all classes.
strict – When set to True, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.

aiida.manage.caching.get_use_cache(*, identifier: Optional[str] = None, strict: bool = False) → bool[source]#

Return whether the caching mechanism should be used for the given process type according to the configuration.

Parameters

identifier – Process type string of the node
strict – When set to True, the function will actually try to resolve the identifier by loading it and if it fails, an exception is raised.

Returns

boolean, True if caching is enabled, False otherwise

Raises

~aiida.common.exceptions.ConfigurationError if the configuration is invalid, either due to a general configuration error, or by defining the class both enabled and disabled

AiiDA manager for global settings

class aiida.manage.manager.Manager[source]#

Bases: object

Manager singleton for globally loaded resources.

AiiDA can have the following global resources loaded:

A single configuration object that contains:
- Global options overrides
- The name of a default profile
- A mapping of profile names to their configuration and option overrides
A single profile object that contains:
- The name of the profile
- The UUID of the profile
- The configuration of the profile, for connecting to storage and processing resources
- The option overrides for the profile

3. A single storage backend object for the profile, to connect to data storage resources 5. A single daemon client object for the profile, to connect to the AiiDA daemon 4. A single communicator object for the profile, to connect to the process control resources 6. A single process controller object for the profile, which uses the communicator to control process tasks 7. A single runner object for the profile, which uses the process controller to start and stop processes 8. A single persister object for the profile, which can persist running processes to the profile storage

__dict__ = mappingproxy({'__module__': 'aiida.manage.manager', '__doc__': 'Manager singleton for globally loaded resources.\n\n AiiDA can have the following global resources loaded:\n\n 1. A single configuration object that contains:\n\n - Global options overrides\n - The name of a default profile\n - A mapping of profile names to their configuration and option overrides\n\n 2. A single profile object that contains:\n\n - The name of the profile\n - The UUID of the profile\n - The configuration of the profile, for connecting to storage and processing resources\n - The option overrides for the profile\n\n 3. A single storage backend object for the profile, to connect to data storage resources\n 5. A single daemon client object for the profile, to connect to the AiiDA daemon\n 4. A single communicator object for the profile, to connect to the process control resources\n 6. A single process controller object for the profile, which uses the communicator to control process tasks\n 7. A single runner object for the profile, which uses the process controller to start and stop processes\n 8. A single persister object for the profile, which can persist running processes to the profile storage\n\n ', '__init__': <function Manager.__init__>, 'get_config': <staticmethod(<function Manager.get_config>)>, 'get_profile': <function Manager.get_profile>, 'load_profile': <function Manager.load_profile>, 'reset_profile': <function Manager.reset_profile>, 'reset_profile_storage': <function Manager.reset_profile_storage>, 'reset_communicator': <function Manager.reset_communicator>, 'reset_runner': <function Manager.reset_runner>, 'unload_profile': <function Manager.unload_profile>, 'profile_storage_loaded': <property object>, 'get_option': <function Manager.get_option>, 'get_backend': <function Manager.get_backend>, 'get_profile_storage': <function Manager.get_profile_storage>, 'get_persister': <function Manager.get_persister>, 'get_communicator': <function Manager.get_communicator>, 'create_communicator': <function Manager.create_communicator>, 'get_daemon_client': <function Manager.get_daemon_client>, 'get_process_controller': <function Manager.get_process_controller>, 'get_runner': <function Manager.get_runner>, 'set_runner': <function Manager.set_runner>, 'create_runner': <function Manager.create_runner>, 'create_daemon_runner': <function Manager.create_daemon_runner>, 'check_rabbitmq_version': <function Manager.check_rabbitmq_version>, 'check_version': <function Manager.check_version>, '__dict__': <attribute '__dict__' of 'Manager' objects>, '__weakref__': <attribute '__weakref__' of 'Manager' objects>, '__annotations__': {'_profile': "Optional['Profile']", '_profile_storage': "Optional['StorageBackend']", '_daemon_client': "Optional['DaemonClient']", '_communicator': "Optional['RmqThreadCommunicator']", '_process_controller': "Optional['RemoteProcessThreadController']", '_persister': "Optional['AiiDAPersister']", '_runner': "Optional['Runner']"}})#

__init__() → None[source]#: Construct a new instance.

__module__ = 'aiida.manage.manager'#

__weakref__#: list of weak references to the object (if defined)

check_rabbitmq_version(communicator: RmqThreadCommunicator)[source]#: Check the version of RabbitMQ that is being connected to and emit warning if it is not compatible.

check_version()[source]#

Check the currently installed version of aiida-core and warn if it is a post release development version.

The aiida-core package maintains the protocol that the main branch will use a post release version number. This means it will always append .post0 to the version of the latest release. This should mean that if this protocol is maintained properly, this method will print a warning if the currently installed version is a post release development branch and not an actual release.

create_communicator(task_prefetch_count: Optional[int] = None) → RmqThreadCommunicator[source]#

Create a Communicator.

Parameters: task_prefetch_count – optional specify how many tasks this communicator take simultaneously
Returns: the communicator instance

create_daemon_runner(loop: Optional[asyncio.AbstractEventLoop] = None) → Runner[source]#

Create and return a new daemon runner.

This is used by workers when the daemon is running and in testing.

Parameters: loop – the (optional) asyncio event loop to use
Returns: a runner configured to work in the daemon configuration

create_runner(with_persistence: bool = True, **kwargs: Any) → Runner[source]#

Create and return a new runner

Parameters: with_persistence – create a runner with persistence enabled
Returns: a new runner instance

get_backend() → StorageBackend[source]#

Return the current profile’s storage backend, loading it if necessary.

Deprecated: use get_profile_storage instead.

get_communicator() → RmqThreadCommunicator[source]#

Return the communicator

Returns: a global communicator instance

static get_config(create=False) → Config[source]#

Return the current config.

Returns: current loaded config instance
Raises: aiida.common.ConfigurationError – if the configuration file could not be found, read or deserialized

get_daemon_client() → DaemonClient[source]#

Return the daemon client for the current profile.

Returns

the daemon client

Raises

aiida.common.MissingConfigurationError – if the configuration file cannot be found
aiida.common.ProfileConfigurationError – if the given profile does not exist

get_option(option_name: str) → Any[source]#

Return the value of a configuration option.

In order of priority, the option is returned from:

The current profile, if loaded and the option specified
The current configuration, if loaded and the option specified
The default value for the option

Parameters: option_name – the name of the option to return
Returns: the value of the option
Raises: aiida.common.exceptions.ConfigurationError – if the option is not found

get_persister() → AiiDAPersister[source]#

Return the persister

Returns: the current persister instance

get_process_controller() → RemoteProcessThreadController[source]#

Return the process controller

Returns: the process controller instance

get_profile() → Optional[Profile][source]#

Return the current loaded profile, if any

Returns: current loaded profile instance

get_profile_storage() → StorageBackend[source]#: Return the current profile’s storage backend, loading it if necessary.

get_runner(**kwargs) → Runner[source]#

Return a runner that is based on the current profile settings and can be used globally by the code.

Returns: the global runner

load_profile(profile: Union[None, str, Profile] = None, allow_switch=False) → Profile[source]#

Load a global profile, unloading any previously loaded profile.

Note

If a profile is already loaded and no explicit profile is specified, nothing will be done.

Parameters

profile – the name of the profile to load, by default will use the one marked as default in the config
allow_switch – if True, will allow switching to a different profile when storage is already loaded

Returns

the loaded Profile instance

Raises

aiida.common.exceptions.InvalidOperation – if another profile has already been loaded and allow_switch is False

property profile_storage_loaded: bool#

Return whether a storage backend has been loaded.

Returns: boolean, True if database backend is currently loaded, False otherwise

reset_communicator() → None[source]#: Reset the communicator.

reset_profile() → None[source]#: Close and reset any associated resources for the current profile.

reset_profile_storage() → None[source]#

Reset the profile storage.

This will close any connections to the services used by the storage, such as database connections.

reset_runner() → None[source]#: Reset the process runner.

set_runner(new_runner: Runner) → None[source]#

Set the currently used runner

Parameters: new_runner – the new runner to use

unload_profile() → None[source]#: Unload the current profile, closing any associated resources.

aiida.manage.manager.get_manager() → aiida.manage.manager.Manager[source]#: Return the AiiDA global manager instance.

aiida.manage.manager.get_rabbitmq_version(communicator: RmqThreadCommunicator)[source]#

Return the version of the RabbitMQ server that the current profile connects to.

Returns: packaging.version.Version

aiida.manage.manager.is_rabbitmq_version_supported(communicator: RmqThreadCommunicator) → bool[source]#

Return whether the version of RabbitMQ configured for the current profile is supported.

Versions 3.5 and below are not supported at all, whereas versions 3.8.15 and above are not compatible with a default configuration of the RabbitMQ server.

Returns: boolean whether the current RabbitMQ version is supported.

Module for the ProfileAccessManager that tracks process access to the profile.

class aiida.manage.profile_access.ProfileAccessManager(profile: aiida.manage.configuration.profile.Profile)[source]#

Bases: object

Class to manage access to a profile.

Any process that wants to request access for a given profile, should first call:

ProfileAccessManager(profile).request_access()

If this returns normally, the profile can be used safely. It will raise if it is locked, in which case the profile should not be used. If a process wants to request exclusive access to the profile, it should use lock:

with ProfileAccessManager(profile).lock():
pass

If the profile is already locked or is currently in use, an exception is raised.

Access and locks of the profile will be recorded in a directory with files with a .pid and .lock extension, respectively. In principle then, at any one time, there can either be a number of pid files, or just a single lock file. If there is a mixture or there are more than one lock files, we are in an inconsistent state.

__dict__ = mappingproxy({'__module__': 'aiida.manage.profile_access', '__doc__': 'Class to manage access to a profile.\n\n Any process that wants to request access for a given profile, should first call:\n\n ProfileAccessManager(profile).request_access()\n\n If this returns normally, the profile can be used safely. It will raise if it is locked, in which case the profile\n should not be used. If a process wants to request exclusive access to the profile, it should use ``lock``:\n\n with ProfileAccessManager(profile).lock():\n pass\n\n If the profile is already locked or is currently in use, an exception is raised.\n\n Access and locks of the profile will be recorded in a directory with files with a ``.pid`` and ``.lock`` extension,\n respectively. In principle then, at any one time, there can either be a number of pid files, or just a single lock\n file. If there is a mixture or there are more than one lock files, we are in an inconsistent state.\n ', '__init__': <function ProfileAccessManager.__init__>, 'request_access': <function ProfileAccessManager.request_access>, 'lock': <function ProfileAccessManager.lock>, 'is_locked': <function ProfileAccessManager.is_locked>, 'is_active': <function ProfileAccessManager.is_active>, 'clear_locks': <function ProfileAccessManager.clear_locks>, '_clear_stale_pid_files': <function ProfileAccessManager._clear_stale_pid_files>, '_get_tracking_files': <function ProfileAccessManager._get_tracking_files>, '_raise_if_locked': <function ProfileAccessManager._raise_if_locked>, '_raise_if_active': <function ProfileAccessManager._raise_if_active>, '__dict__': <attribute '__dict__' of 'ProfileAccessManager' objects>, '__weakref__': <attribute '__weakref__' of 'ProfileAccessManager' objects>, '__annotations__': {}})#

__init__(profile: aiida.manage.configuration.profile.Profile)[source]#

Class that manages access and locks to the given profile.

Parameters: profile – the profile whose access to manage.

__module__ = 'aiida.manage.profile_access'#

__weakref__#: list of weak references to the object (if defined)

_clear_stale_pid_files() → None[source]#: Clear any stale PID files.

_get_tracking_files(ext_string: str, exclude_self: bool = False) → List[pathlib.Path][source]#

Return a list of all files that track the accessing and locking of the profile.

Parameters

ext_string – To get the files that track locking use .lock, to get the files that track access use .pid.
exclude_self – If true removes from the returned list any tracking to the current process.

_raise_if_active(message_start)[source]#

Raise a LockingProfileError if the profile is being accessed.

Parameters: message_start – Text to use as the start of the exception message.
Raises: LockingProfileError – if the profile is active.

_raise_if_locked(message_start)[source]#

Raise a LockedProfileError if the profile is locked.

Parameters: message_start – Text to use as the start of the exception message.
Raises: LockedProfileError – if the profile is locked.

clear_locks() → None[source]#: Clear all locks on this profile.

Warning

This should only be used if the profile is currently still incorrectly locked because the lock was not automatically released after the lock contextmanager exited its scope.

is_active() → bool[source]#: Return whether the profile is currently in use.

is_locked() → bool[source]#: Return whether the profile is locked.

lock()[source]#

Request a lock on the profile for exclusive access.

This context manager should be used if exclusive access to the profile is required. Access will be granted if the profile is currently not in use, nor locked by another process. During the context, the profile will be locked, which will be lifted automatically as soon as the context exits.

Raises

LockingProfileError – if there are currently active processes using the profile.
LockedProfileError – if there currently already is a lock on the profile.

request_access() → None[source]#

Request access to the profile.

If the profile is locked, a LockedProfileError is raised. Otherwise a PID file is created for this process and the function returns None. The PID file contains the command of the process.

Raises: LockedProfileError – if the profile is locked.