AiiDA is designed to run on Unix operating systems and requires the following software:
bash or zsh (The shell)
python >= 3.5 (The programming language used by AiiDA)
python3-pip (Python 3 package manager)
postgresql (Database software, version 9.4 or higher)
RabbitMQ (A message broker necessary for AiiDA to communicate between processes)
Depending on your set up, there are a few optional dependencies:
git (Version control system used for AiiDA development)
graphviz (For plotting AiiDA provenance graphs)
AiiDA has been tested on the following platforms:
Ubuntu 14.04, 16.04, 18.04
Mac OS X
We expect AiiDA to also run on:
Older and newer Ubuntu versions
Other Linux distributions
Windows Subsystem for Linux (WSL)
Below, we provide installation instructions for a number of operating systems.
$ sudo apt-get install \ postgresql postgresql-server-dev-all postgresql-client rabbitmq-server \ git python3-dev python3-pip $ pip install aiida-core
See Ubuntu instructions for details.
$ brew install postgresql rabbitmq git python $ brew services start postgresql $ brew services start rabbitmq $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py $ python get-pip.py $ pip install aiida-core
See MacOS X (Homebrew) instructions for details.
$ sudo port install postgresql postgresql-server rabbitmq-server $ pg_ctl -D /usr/local/var/postgres start $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py $ python get-pip.py $ pip install aiida-core
See MacOS X (MacPorts) instructions for details.
$ emerge -av git postgresql rabbitmq-server $ rc-update add rabbitmq $ emerge --ask dev-lang/python:3.7 $ emerge --ask dev-python/pip $ pip install aiida-core
See Gentoo Linux instructions for details.
$ sudo apt-get install \ postgresql postgresql-server-dev-all postgresql-client \ git python3-dev python-pip $ sudo service postgresql start $ pip install aiida-core
Then RabbitMQ must be installed directly on Windows. See WSL instructions for details.
To install the prerequisites on Ubuntu and any other Debian derived distribution, you can use the apt package manager. The following will install the basic Python requirements and the git source control manager:
$ sudo apt-get install git python3-dev python3-pip
To install the requirements for the postgres database run the following:
postgres
$ sudo apt-get install postgresql postgresql-server-dev-all postgresql-client
Install the RabbitMQ message broker:
$ sudo apt-get install rabbitmq-server
This installs and adds RabbitMQ as a system service. To check whether it is running:
$ sudo rabbitmqctl status
If it is not running already, it should after a reboot.
Finally install the aiida-core Python environment:
$ sudo git python3-dev python3-pip $ pip install aiida-core
Further Reading
For a more detailed description of database requirements and usage see the database topic.
For problems with installing RabbitMQ, refer to the detailed instructions provided on the RabbitMQ website for Debian based distributions.
For Mac OS we recommend using the Homebrew package manager. If you have not installed Homebrew yet, see their installation guide.
After you have installed Homebrew, you can install the basic requirements as follows:
$ brew install postgresql rabbitmq git python
To start the postgres database server and rabbitmq service, execute:
rabbitmq
$ brew services start postgresql $ brew services start rabbitmq
You can check whether it is running by checking the status through the command:
$ /usr/local/sbin/rabbitmqctl status
Finally install the aiida-core python environment:
$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py $ python get-pip.py $ pip install aiida-core
For problems with installing RabbitMQ, refer to the detailed instructions provided on the RabbitMQ website for Homebrew based distributions.
For details on the installation of the pip package manager, refer to their documentation
pip
Another package manager for MacOS is macports.
$ sudo port install postgresql postgresql-server rabbitmq-server git python
To start the postgres database server, run:
$ sudo su postgres $ pg_ctl -D /opt/local/var/db/postgresql96/defaultdb start
To start the rabbitmq server, run:
$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.rabbitmq-server.plist
You can check whether it is running as follows:
$ sudo rabbitmqctl status # this starts ``rabbitmq`` at system startup: $ sudo port load rabbitmq-server
Trouble Installing RabbitMQ?
Be sure to install rabbitmq-server 3.7.9 or later. If rabbitmqctl status returns an error “Hostname mismatch”, the easiest solution can be to simply sudo port uninstall the package and install it again.
rabbitmq-server 3.7.9
rabbitmqctl status
sudo port uninstall
To install RabbitMQ on a Gentoo distribution through the portage package manager run the following command:
portage
$ emerge -av rabbitmq-server
To make sure that RabbitMQ is started at system boot, execute:
rc-update add rabbitmq
If you want to manually start the RabbitMQ server you can use:
/etc/init.d/rabbitmq start
Make sure that RabbitMQ is running with:
If you have encounter the following error Argument '-smp enable' not supported." Remove the mentioned option from the file /usr/libexec/rabbitmq/rabbitmq-env and restart the server. If you still have trouble getting RabbitMQ to run, please refer to the detailed instructions provided on the website of RabbitMQ itself for generic Unix systems.
If you have encounter the following error
Argument '-smp enable' not supported."
Remove the mentioned option from the file /usr/libexec/rabbitmq/rabbitmq-env and restart the server. If you still have trouble getting RabbitMQ to run, please refer to the detailed instructions provided on the website of RabbitMQ itself for generic Unix systems.
/usr/libexec/rabbitmq/rabbitmq-env
The guide for Ubuntu above can generally be followed, but there are a few things to note:
Tip
Installing Ubuntu instead of the version specific applications, will let you have the latest LTS version.
The Windows native RabbitMQ should be installed and started. (For WSL 2, this should not be necessary.)
Linux services under WSL are not started automatically. To start the PostgreSQL and RabbitMQ-server services, type the commands below in the terminal:
sudo service postgresql start sudo service rabbitmq-server start
These services may be run at startup without passing a password in the following way:
Create a .sh file with the lines above, but without sudo. Make the file executeable, i.e., type:
.sh
sudo
$ chmod +x /path/to/file.sh $ sudo visudo
And add the line:
<username> ALL=(root) NOPASSWD: /path/to/file.sh
Replacing <username> with your Ubuntu username. This will allow you to run only this specific .sh file with root access (without password), without lowering security on the rest of your system.
<username>
root
There is a known issue in WSL Ubuntu 18.04 where the timezone is not configured correctly out-of-the-box, which may cause problem for the database. The following command can be used to re-configure the time zone:
$ sudo dpkg-reconfigure tzdata
The file open limit may need to be raised using ulimit -n 2048 (default is 1024), when running tests. You can check the limit by using ulimit -n.
ulimit -n 2048
ulimit -n
Hint
This may need to be run every time the system starts up.
It may be worth considering adding some of these commands to your ~/.bashrc file, since some of these settings may reset upon reboot.
~/.bashrc
For using WSL as a developer, please see the considerations made in our wiki-page for developers.
AiiDA depends on a number of third party python packages, and usually on specific versions of those packages. In order not to interfere with third party packages needed by other software on your system, we strongly recommend isolating AiiDA in a virtual python environment.
Additional Information
A very good tutorial on Python environments is provided by realpython.com.
venv is module included directly with python for creating virtual environments. To create a virtual environment, in a given directory, run:
$ python3 -m venv /path/to/new/virtual/environment/aiida
The command to activate the environment is shell specific (see the documentation. With bash the following command is used:
$ source /path/to/new/virtual/environment/aiida/bin/activate
To leave or deactivate the environment, simply run:
(aiida) $ deactivate
Update install tools
You may need to install pip and setuptools in your virtual environment in case the system or user version of these tools is old
setuptools
(aiida) $ pip install -U setuptools pip
If you have Conda installed then you can directly create a new environment with aiida-core and (optionally) Postgres and RabbitMQ installed.
aiida-core
$ conda create -n aiida -c conda-forge python=3.7 aiida-core aiida-core.services pip $ conda activate $ conda deactivate aiida
AiiDA can be installed either from the python package index PyPI, Anaconda Cloud (both good for general use) or directly from the aiida-core github repository (good for developers).
With your virtual environment active (see above), install the aiida-core python package from PyPI using:
$ pip install aiida-core
Installing AiiDA in your system environment
Consider adding the --user flag to avoid the need for administrator privileges.
--user
Or, if using Conda, from the Anaconda Cloud:
$ conda install -c conda-forge aiida-core aiida-core.services
Note
aiida-core.services is an optional package to also install PostgreSQL and RabbitMQ.
Alternatively, you can create a directory where to clone the AiiDA source code and install AiiDA from source:
$ mkdir <your_directory> $ cd <your_directory> $ git clone https://github.com/aiidateam/aiida-core $ pip install -e aiida-core
There are additional optional packages that you may want to install, which are grouped in the following categories:
atomic_tools: packages that allow importing and manipulating crystal structure from various formats ssh_kerberos: adds support for ssh transport authentication through Kerberos REST: allows a REST server to be ran locally to serve AiiDA data docs: tools to build the documentation notebook: jupyter notebook - to allow it to import AiiDA modules tests: python modules required to run the automatic unit tests pre-commit: pre-commit tools required for developers to enable automatic code linting and formatting
atomic_tools: packages that allow importing and manipulating crystal structure from various formats
atomic_tools
ssh_kerberos: adds support for ssh transport authentication through Kerberos
ssh_kerberos
REST: allows a REST server to be ran locally to serve AiiDA data
REST
docs: tools to build the documentation
docs
notebook: jupyter notebook - to allow it to import AiiDA modules
notebook
tests: python modules required to run the automatic unit tests
tests
pre-commit: pre-commit tools required for developers to enable automatic code linting and formatting
pre-commit
In order to install any of these package groups, simply append them as a comma separated list in the pip install command
$ pip install -e aiida-core[atomic_tools,docs]
Kerberos on Ubuntu
If you are installing the optional ssh_kerberos and you are on Ubuntu you might encounter an error related to the gss package. To fix this you need to install the libffi-dev and libkrb5-dev packages:
gss
libffi-dev
libkrb5-dev
$ sudo apt-get install libffi-dev libkrb5-dev
AiiDA uses the reentry package to keep a fast cache of entry points for a snappy verdi cli. After installing AiiDA packages, always remember to update the reentry cache using:
verdi
$ reentry scan
After successful installation, you need to create an AiiDA profile via AiiDA’s command line interface verdi.
Most users should use the interactive quicksetup:
$ verdi quicksetup
which leads through the installation process and takes care of creating the corresponding AiiDA database.
For maximum control and customizability, one can use verdi setup and set up the database manually as explained below.
verdi setup
Don’t forget to backup your data!
See the installation backup how-to.
Want to manage multiple profiles?
See the managing profiles how-to.
AiiDA uses a database to store the nodes, node attributes and other information, allowing the end user to perform fast queries of the results. Currently, the highly performant PostgreSQL database is supported as a database backend.
Find out more about the database
Creating a Database Cluster.
Starting the Database Server.
The database topic.
To manually create the database for AiiDA, you need to run the program psql to interact with postgres. On most operating systems, you need to do so as the postgres user that was created upon installing the software. To assume the role of postgres run as root:
psql
$ su - postgres
(or, equivalently, type sudo su - postgres, depending on your distribution) and launch the postgres program:
sudo su - postgres
$ psql
Create a new database user account for AiiDA by running:
CREATE USER aiida WITH PASSWORD '<password>';
replacing <password> with a password of your choice.
<password>
You will need to provide the password again when you configure AiiDA to use this database through verdi setup. If you want to change the password you just created use the command:
ALTER USER aiida PASSWORD '<password>';
Next, we create the database itself. We enforce the UTF-8 encoding and specific locales:
CREATE DATABASE aiidadb OWNER aiida ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE=template0;
and grant all privileges on this DB to the previously-created aiida user:
aiida
GRANT ALL PRIVILEGES ON DATABASE aiidadb to aiida;
You have now created a database for AiiDA and you can close the postgres shell by typing \q. To test if the database was created successfully, you can run the following command as a regular user in a bash terminal:
\q
$ psql -h localhost -d aiidadb -U aiida -W
and type the password you inserted before, when prompted. If everything worked well, you should get no error and see the prompt of the psql shell.
If you use the same names as in the example commands above, then during the verdi setup phase the following parameters will apply to the newly created database:
$ Database engine: postgresql_psycopg2 $ Database host: localhost $ Database port: 5432 $ AiiDA Database name: aiidadb $ AiiDA Database user: aiida $ AiiDA Database password: <password>
Don’t forget to backup your database!
See the Database backup how-to), and how to move your database.
On Ubuntu Linux, the default PostgreSQL setup is configured to use peer authentication, which allows password-less login via local Unix sockets. In this mode, PostgreSQL compares the Unix user connecting to the socket with its own database of users and allows a connection if a matching user exists.
peer
This is an alternative route to set up your database - the standard approach will work on Ubuntu just as well.
Below we are going to take advantage of the command-line utilities shipped on Ubuntu to simplify creating users and databases compared to issuing the SQL commands directly.
Assume the role of postgres:
$ sudo su postgres
Create a database user with the same name as the UNIX user who will be running AiiDA (usually your login name):
$ createuser <username>
replacing <username> with your username.
Next, create the database itself with your user as the owner:
$ createdb -O <username> aiidadb
Exit the shell to go back to your login user. To test if the database was created successfully, try:
$ psql aiidadb
During the verdi setup phase, use ! to leave host empty and specify your Unix user name as the AiiDA Database user.:
!
$ Database engine: postgresql_psycopg2 $ Database host: ! $ Database port: 5432 $ AiiDA Database name: aiidadb $ AiiDA Database user: <username> $ AiiDA Database password: ""
After the database has been created, do:
$ verdi setup --profile <profile_name>
where <profile_name> is a profile name of your choosing. The verdi setup command will guide you through the setup process through a series of prompts.
The first information asked is your email, which will be used to associate the calculations to you. In AiiDA, the email is your username, and acts as a unique identifier when importing/exporting data from AiiDA.
The password, in the current version of AiiDA, is not used (it will be used only in the REST API and in the web interface). If you leave the field empty, no password will be set and no access will be granted to the user via the REST API and the web interface.
Then, the following prompts will help you configure the database. Typical settings are:
$ Default user email: richard.wagner@leipzig.de $ Database engine: postgresql_psycopg2 $ PostgreSQL host: localhost $ PostgreSQL port: 5432 $ AiiDA Database name: aiida_dev $ AiiDA Database user: aiida $ AiiDA Database password: <password> $ AiiDA repository directory: /home/wagner/.aiida/repository/ [...] Configuring a new user with email 'richard.wagner@leipzig.de' $ First name: Richard $ Last name: Wagner $ Institution: BRUHL, LEIPZIG $ The user has no password, do you want to set one? [y/N] y $ Insert the new password: $ Insert the new password (again):
The AiiDA daemon process runs in the background and takes care of processing your submitted calculations and workflows, checking their status, retrieving their results once they are finished and storing them in the AiiDA database.
The AiiDA daemon is controlled using three simple commands:
verdi daemon start: start the daemon verdi daemon status: check the status of the daemon verdi daemon stop: stop the daemon
verdi daemon start: start the daemon
verdi daemon start
verdi daemon status: check the status of the daemon
verdi daemon status
verdi daemon stop: stop the daemon
verdi daemon stop
While operational, the daemon logs its activity to a file in ~/.aiida/daemon/log/ (or, more generally, $AIIDA_PATH/.aiida/daemon/log). Get the latest log messages via verdi daemon logshow.
~/.aiida/daemon/log/
$AIIDA_PATH/.aiida/daemon/log
verdi daemon logshow
Use the verdi status command to check that all services are up and running:
verdi status
$ verdi status ✓ profile: On profile quicksetup ✓ repository: /repo/aiida_dev/quicksetup ✓ postgres: Connected to aiida@localhost:5432 ✓ rabbitmq: Connected to amqp://127.0.0.1?heartbeat=600 ✓ daemon: Daemon is running as PID 2809 since 2019-03-15 16:27:52
In the example output, all service have a green check mark and so should be running as expected.
Are you in your virtual environment?
Remember that in order to work with AiiDA through for example the verdi command, you need to be in your virtual environment. If you open a new terminal for example, be sure to activate it first, e.g. for venv:
$ source ~/.virtualenvs/aiida/bin/activate
Want to manage your install?
See the Installation how-to
Configuring Updating Backing-up
What next?
Try out the Basic Tutorial.
Jupyter is an open-source web application that allows you to create in-browser notebooks containing live code, visualizations and formatted text.
Originally born out of the iPython project, it now supports code written in many languages and customized iPython kernels.
If you didn’t already install AiiDA with the [notebook] option (during pip install), run pip install jupyter inside the virtualenv, and then run from within the virtualenv:
[notebook]
pip install
pip install jupyter
$ jupyter notebook
This will open a tab in your browser. Click on New -> Python and type:
New -> Python
import aiida
followed by Shift-Enter. If no exception is thrown, you can use AiiDA in Jupyter.
Shift-Enter
If you want to set the same environment as in a verdi shell, add the following code to a .py file (create one if there isn’t any) in <home_folder>/.ipython/profile_default/startup/:
verdi shell
.py
<home_folder>/.ipython/profile_default/startup/
try: import aiida except ImportError: pass else: import IPython from aiida.tools.ipython.ipython_magics import load_ipython_extension # Get the current Ipython session ipython = IPython.get_ipython() # Register the line magic load_ipython_extension(ipython)
This file will be executed when the ipython kernel starts up and enable the line magic %aiida. Alternatively, if you have a aiida-core repository checked out locally, you can just copy the file <aiida-core>/aiida/tools/ipython/aiida_magic_register.py to the same folder. The current ipython profile folder can be located using:
%aiida
<aiida-core>/aiida/tools/ipython/aiida_magic_register.py
$ ipython locate profile
After this, if you open a Jupyter notebook as explained above and type in a cell:
followed by Shift-Enter. You should receive the message “Loaded AiiDA DB environment.” This line magic should also be enabled in standard ipython shells.
AiiDA maintains a Docker image on Docker Hub. This image contains a pre-configured AiiDA environment, and so is particularly useful for learning and testing purposes. It is a great way to quickly get started on the tutorials!
Follow Docker’s install guide to download Docker and start its daemon. Now you can pull the aiida-core image straight from Docker Hub, for a specific version.
$ docker pull aiidateam/aiida-core:1.2.1
We can start a container running by:
$ docker run -d --name aiida-container aiidateam/aiida-core:1.2.1
The container comes installed with all required services and, on start-up, will automatically start them and create an AiiDA profile (plus a localhost computer). To (optionally) wait for the services to start and inspect the start-up process, we can run:
$ docker exec -t aiida-container wait-for-services $ docker logs aiida-container
The profile is created under the aiida username, so to execute commands use:
$ docker exec -t --user aiida aiida-container /bin/bash -l -c 'verdi status' ✓ config dir: /home/aiida/.aiida ✓ profile: On profile default ✓ repository: /home/aiida/.aiida/repository/default ✓ postgres: Connected as aiida_qs_aiida_477d3dfc78a2042156110cb00ae3618f@localhost:5432 ✓ rabbitmq: Connected to amqp://127.0.0.1?heartbeat=600 ✓ daemon: Daemon is running as PID 1795 since 2020-05-20 02:54:00
Or to enter into the container interactively:
$ docker exec -it --user aiida aiida-container /bin/bash
If you stop the container and start it again, any data you created will persist.
$ docker stop aiida-container $ docker start aiida-container
But if you remove the container all data will be removed.
$ docker stop aiida-container $ docker rm aiida-container
To store data and even share it between containers, you may wish to add a volume:
$ docker run -d --name aiida-container --mount source=my_data,target=/tmp/my_data aiidateam/aiida-core:1.2.1
Now anything that you save to the /tmp/my_data folder will be saved to the volume persistently. You can even add files directly to the folder outside of the container, by finding its mounting point:
/tmp/my_data
$ docker volume inspect my_data $ echo "hallo" | sudo tee -a /var/lib/docker/volumes/my_data/_data/hallo.txt > /dev/null