aiida.manage.external package

User facing APIs to control AiiDA from the verdi cli, scripts or plugins

Submodules

Provides an API for postgres database maintenance tasks.

This API creates and drops postgres users and databases used by the verdi quicksetup commandline tool. It allows convenient access to this functionality from within python without knowing details about how postgres is installed by default on various systems. If the postgres setup is not the default installation, additional information needs to be provided.

class aiida.manage.external.postgres.Postgres(interactive=False, quiet=True, dbinfo=None)

Bases: object

Provides postgres database manipulation assuming no prior setup

• Can be used to create the initial aiida database and database user.
• Works in every reasonable environment, provided the user can sudo

Tries to use psycopg2 with a fallback to psql subcommands (using sudo su to run as postgres user).

Parameters:

• port – (str) Assume the database server runs on this port
• interactive – (bool) Allow prompting the user for information Will also be passed to sudo (if using psycopg2 fails) and to the callback that can be set to be called when automatic setup detection fails
• quiet – (bool) Suppress messages

Simple Example:

postgres = Postgres()
postgres.determine_setup()
postgres.create_dbuser('username', 'password')
if not postgres.db_exists('dbname'):
    postgres.create_db('username', 'dbname')

Complex Example:

postgres = Postgres(interactive=True, dbinfo={'port': 5433})
postgres.setup_fail_callback = prompt_db_info
postgres.determine_setup()
if postgres.pg_execute:
    print('setup sucessful!')

Note: In postgresql

• you cannot drop databases you are currently connected to
• ‘template0’ is the unmodifiable template database (which you cannot connect to)
• ‘template1’ is the modifiable template database (which you can connect to)

__dict__ = dict_proxy({'__module__': 'aiida.manage.external.postgres', 'copy_db': <function copy_db>, 'determine_setup': <function determine_setup>, 'set_dbinfo': <function set_dbinfo>, 'create_dbuser': <function create_dbuser>, 'drop_db': <function drop_db>, 'drop_dbuser': <function drop_dbuser>, 'dbuser_exists': <function dbuser_exists>, '__dict__': <attribute '__dict__' of 'Postgres' objects>, 'db_exists': <function db_exists>, '__init__': <function __init__>, 'get_dbinfo': <function get_dbinfo>, '__weakref__': <attribute '__weakref__' of 'Postgres' objects>, 'from_profile': <classmethod object>, '_no_setup_detected': <function _no_setup_detected>, 'create_db': <function create_db>, 'set_setup_fail_callback': <function set_setup_fail_callback>, 'try_connect': <function try_connect>, '__doc__': "\n Provides postgres database manipulation assuming no prior setup\n\n * Can be used to create the initial aiida database and database user.\n * Works in every reasonable environment, provided the user can sudo\n\n Tries to use psycopg2 with a fallback to psql subcommands (using ``sudo su`` to run as postgres user).\n\n :param port: (str) Assume the database server runs on this port\n :param interactive: (bool) Allow prompting the user for information\n Will also be passed to ``sudo`` (if using ``psycopg2`` fails) and to\n the callback that can be set to be called when automatic setup detection fails\n :param quiet: (bool) Suppress messages\n\n Simple Example::\n\n postgres = Postgres()\n postgres.determine_setup()\n postgres.create_dbuser('username', 'password')\n if not postgres.db_exists('dbname'):\n postgres.create_db('username', 'dbname')\n\n Complex Example::\n\n postgres = Postgres(interactive=True, dbinfo={'port': 5433})\n postgres.setup_fail_callback = prompt_db_info\n postgres.determine_setup()\n if postgres.pg_execute:\n print('setup sucessful!')\n\n Note: In postgresql\n * you cannot drop databases you are currently connected to\n * 'template0' is the unmodifiable template database (which you cannot connect to)\n * 'template1' is the modifiable template database (which you can connect to)\n "})

__init__(interactive=False, quiet=True, dbinfo=None)

Store postgres connection info.

Parameters:

• interactive – use True for verdi commands
• quiet – use False to show warnings/exceptions
• dbinfo – psycopg dictionary containing keys like ‘host’, ‘user’, ‘port’, ‘database’

__module__ = 'aiida.manage.external.postgres'

__weakref__

list of weak references to the object (if defined)

_no_setup_detected()

Print a warning message and calls the failed setup callback

copy_db(src_db, dest_db, dbuser)

create_db(dbuser, dbname)

Create a database in postgres

Parameters:

• dbuser – (str), Name of the user which should own the db.
• dbname – (str), Name of the database.

create_dbuser(dbuser, dbpass)

Create a database user in postgres

Parameters:

• dbuser – (str), Name of the user to be created.
• dbpass – (str), Password the user should be given.

db_exists(dbname)

Check wether a postgres database with dbname exists

Parameters:dbname – Name of the database to check for Returns:(bool), True if database exists, False otherwise

dbuser_exists(dbuser)

Find out if postgres user with name dbuser exists

Parameters:dbuser – (str) database user to check for Returns:(bool) True if user exists, False otherwise

determine_setup()

Find out how postgres can be accessed.

Depending on how postgres is set up, psycopg2 can be used to create dbs and db users, otherwise a subprocess has to be used that executes psql as an os user with the right permissions.

Note: We aim for a setup that is can manipulate (create/drop) databases and database users.

We therefore do not want to connect to databases of AiiDA profiles and will connect to ‘template1’.

drop_db(dbname)

Drop a database in postgres

Parameters:dbname – (str), Name of the database.

drop_dbuser(dbuser)

Drop a database user in postgres

Parameters:dbuser – (str), Name of the user to be dropped.

classmethod from_profile(profile, **kwargs)

Create Postgres instance with dbinfo from profile data.

get_dbinfo()

set_dbinfo(dbinfo)

Set the dbinfo manually

set_setup_fail_callback(callback)

Set a callback to be called when setup cannot be determined automatically

Parameters:callback – a callable with signature callback(interactive, dbinfo)

try_connect()

Try connecting using stored dbinfo.

aiida.manage.external.postgres._pg_execute_not_connected(command, **kwargs)

A dummy implementation of a postgres command execution function.

Represents inability to execute postgres commands.

aiida.manage.external.postgres._pg_execute_psyco(command, **kwargs)

executes a postgres commandline through psycopg2

Parameters:

• command – A psql command line as a str
• kwargs – will be forwarded to psycopg2.connect

aiida.manage.external.postgres._pg_execute_sh(command, user='postgres', **kwargs)

executes a postgres command line as another system user in a subprocess.

Parameters:

• command – A psql command line as a str
• user – Name of a system user with postgres permissions
• kwargs – connection details to forward to psql, signature as in psycopg2.connect

To stop sudo from asking for a password and fail if one is required, pass non_interactive=True as a kwarg.

aiida.manage.external.postgres._sudo_exists()

Check that the sudo command can be found

Returns:True if successful, False otherwise

aiida.manage.external.postgres._try_connect_psycopg(**kwargs)

try to start a psycopg2 connection.

Returns:True if successful, False otherwise

aiida.manage.external.postgres._try_subcmd(**kwargs)

try to run psql in a subprocess.

Returns:True if successful, False otherwise

aiida.manage.external.postgres.manual_setup_instructions(dbuser, dbname)

Create a message with instructions for manually creating a database

aiida.manage.external.postgres.prompt_db_info(*args)

Prompt interactively for postgres database connecting details

Can be used as a setup fail callback for aiida.manage.external.postgres.Postgres

Returns:dictionary with the following keys: host, port, database, user

Components to communicate tasks to RabbitMQ.

exception aiida.manage.external.rmq.RemoteException

Bases: exceptions.Exception

An exception occurred at the remote end of the call

__module__ = 'kiwipy.communications'

__weakref__

list of weak references to the object (if defined)

aiida.manage.external.rmq.CommunicationTimeout

alias of concurrent.futures._base.TimeoutError

exception aiida.manage.external.rmq.DeliveryFailed

Bases: exceptions.Exception

Failed to deliver a message

__module__ = 'kiwipy.communications'

__weakref__

list of weak references to the object (if defined)

class aiida.manage.external.rmq.ProcessLauncher(loop=None, persister=None, load_context=None, loader=None)

Bases: plumpy.process_comms.ProcessLauncher

A sub class of plumpy.ProcessLauncher to launch a Process

It overrides the _continue method to make sure the node corresponding to the task can be loaded and that if it is already marked as terminated, it is not continued but the future is reconstructed and returned

__module__ = 'aiida.manage.external.rmq'

_continue(**kwargs)

Continue the task

Note that the task may already have been completed, as indicated from the corresponding the node, in which case it is not continued, but the corresponding future is reconstructed and returned. This scenario may occur when the Process was already completed by another worker that however failed to send the acknowledgment.

Parameters:

• communicator – the communicator that called this method
• pid – the pid of the process to continue
• nowait – if True don’t wait for the process to finish, just return the pid, otherwise wait and return the results
• tag – the tag of the checkpoint to continue from

Raises:

plumpy.TaskRejected – if the node corresponding to the task cannot be loaded