aiida.backends.sqlalchemy.migrations.versions package

Submodules

This migration cleans the log records from non-Node entity records. It removes from the DbLog table the legacy workflow records and records that correspond to an unknown entity and places them to corresponding files.

This migration corresponds to the 0024_dblog_update Django migration.

Revision ID: 041a79fc615f Revises: 7ca08c391c49 Create Date: 2018-12-28 15:53:14.596810

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.downgrade()[source]

Downgrade function to the previous schema.

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.export_and_clean_workflow_logs(connection)[source]

Export the logs records that correspond to legacy workflows and to unknown entities (place them to files and remove them from the DbLog table).

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_legacy_workflow_log_number(connection)[source]

Get the number of the log records that correspond to legacy workflows

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_logs_with_no_nodes_number(connection)[source]

Get the number of the log records that correspond to nodes that were deleted

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_serialized_legacy_workflow_logs(connection)[source]

Get the serialized log records that correspond to legacy workflows

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_serialized_logs_with_no_nodes(connection)[source]

Get the serialized log records that correspond to nodes that were deleted

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_serialized_unknown_entity_logs(connection)[source]

Get the serialized log records that correspond to unknown entities

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.get_unknown_entity_log_number(connection)[source]

Get the number of the log records that correspond to unknown entities

aiida.backends.sqlalchemy.migrations.versions.041a79fc615f_dblog_cleaning.upgrade()[source]

Changing the log table columns to use uuid to reference remote objects and log entries. Upgrade function.

Drop the transport_params from the Computer database model.

Revision ID: 07fac78e6209 Revises: de2eaf6978b4 Create Date: 2019-02-16 15:32:42.745450

aiida.backends.sqlalchemy.migrations.versions.07fac78e6209_drop_computer_transport_params.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.07fac78e6209_drop_computer_transport_params.upgrade()[source]

Migrations for the upgrade.

Correct the type string for the base data types

Revision ID: 0aebbeab274d Revises: 7a6587e16f4c Create Date: 2018-02-24 20:12:44.731358

aiida.backends.sqlalchemy.migrations.versions.0aebbeab274d_base_data_plugin_type_string.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.0aebbeab274d_base_data_plugin_type_string.upgrade()[source]

Move trajectory symbols from repository array to attribute

Revision ID: 12536798d4d3 Revises: 37f3d4882837 Create Date: 2019-01-21 10:15:02.451308

aiida.backends.sqlalchemy.migrations.versions.12536798d4d3_trajectory_symbols_to_attribute.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.12536798d4d3_trajectory_symbols_to_attribute.upgrade()[source]

Migrations for the upgrade.

Migration to reflect the name change of the built in calculation entry points in the database.

Revision ID: 140c971ae0a3 Revises: 162b99bca4a2 Create Date: 2018-12-06 12:42:01.897037

aiida.backends.sqlalchemy.migrations.versions.140c971ae0a3_migrate_builtin_calculations.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.140c971ae0a3_migrate_builtin_calculations.upgrade()[source]

Migrations for the upgrade.

Drop the DbCalcState table

Revision ID: 162b99bca4a2 Revises: a603da2cc809 Create Date: 2018-11-14 08:37:13.719646

aiida.backends.sqlalchemy.migrations.versions.162b99bca4a2_drop_dbcalcstate.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.162b99bca4a2_drop_dbcalcstate.upgrade()[source]

Drop the columns nodeversion and public from the DbNode model.

Revision ID: 1830c8430131 Revises: 1b8ed3425af9 Create Date: 2019-05-27 15:35:37.404644

aiida.backends.sqlalchemy.migrations.versions.1830c8430131_drop_node_columns_nodeversion_public.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.1830c8430131_drop_node_columns_nodeversion_public.upgrade()[source]

Remove legacy workflows

Revision ID: 1b8ed3425af9 Revises: 3d6190594e19 Create Date: 2019-04-03 17:11:44.073582

aiida.backends.sqlalchemy.migrations.versions.1b8ed3425af9_remove_legacy_workflows.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.1b8ed3425af9_remove_legacy_workflows.export_workflow_data(connection)[source]

Export existing legacy workflow data to a JSON file.

aiida.backends.sqlalchemy.migrations.versions.1b8ed3425af9_remove_legacy_workflows.json_serializer(obj)[source]

JSON serializer for objects not serializable by default json code

aiida.backends.sqlalchemy.migrations.versions.1b8ed3425af9_remove_legacy_workflows.upgrade()[source]

Migrations for the upgrade.

Migration after the provenance redesign

Revision ID: 239cea6d2452 Revises: 140c971ae0a3 Create Date: 2018-12-04 21:14:15.250247

Scan the database for any links that are unexpected.

The checks will verify that there are no outgoing call or return links from calculation nodes and that if a workflow node has a create link, it has at least an accompanying return link to the same data node, or it has a call link to a calculation node that takes the created data node as input.

aiida.backends.sqlalchemy.migrations.versions.239cea6d2452_provenance_redesign.downgrade()[source]

The downgrade migration actions.

aiida.backends.sqlalchemy.migrations.versions.239cea6d2452_provenance_redesign.migrate_infer_calculation_entry_point(connection)[source]

Set the process type for calculation nodes by inferring it from their type string.

aiida.backends.sqlalchemy.migrations.versions.239cea6d2452_provenance_redesign.upgrade()[source]

The upgrade migration actions.

Data migration for legacy JobCalculations.

These old nodes have already been migrated to the correct CalcJobNode type in a previous migration, but they can still contain a state attribute with a deprecated JobCalcState value and they are missing a value for the process_state, process_status, process_label and exit_status. The process_label is impossible to infer consistently in SQL so it will be omitted. The other will be mapped from the state attribute as follows:

Old state            | Process state  | Exit status | Process status
---------------------|----------------|-------------|----------------------------------------------------------
`NEW`                | `Killed`       |    `None`   | Legacy `JobCalculation` with state `NEW`
`TOSUBMIT`           | `Killed`       |    `None`   | Legacy `JobCalculation` with state `TOSUBMIT`
`SUBMITTING`         | `Killed`       |    `None`   | Legacy `JobCalculation` with state `SUBMITTING`
`WITHSCHEDULER`      | `Killed`       |    `None`   | Legacy `JobCalculation` with state `WITHSCHEDULER`
`COMPUTED`           | `Killed`       |    `None`   | Legacy `JobCalculation` with state `COMPUTED`
`RETRIEVING`         | `Killed`       |    `None`   | Legacy `JobCalculation` with state `RETRIEVING`
`PARSING`            | `Killed`       |    `None`   | Legacy `JobCalculation` with state `PARSING`
`SUBMISSIONFAILED`   | `Excepted`     |    `None`   | Legacy `JobCalculation` with state `SUBMISSIONFAILED`
`RETRIEVALFAILED`    | `Excepted`     |    `None`   | Legacy `JobCalculation` with state `RETRIEVALFAILED`
`PARSINGFAILED`      | `Excepted`     |    `None`   | Legacy `JobCalculation` with state `PARSINGFAILED`
`FAILED`             | `Finished`     |      2      |  -
`FINISHED`           | `Finished`     |      0      |  -
`IMPORTED`           |       -        |      -      |  -

Note the IMPORTED state was never actually stored in the state attribute, so we do not have to consider it. The old state attribute has to be removed after the data is migrated, because its value is no longer valid or useful.

Note: in addition to the three attributes mentioned in the table, all matched nodes will get Legacy JobCalculation as their process_label which is one of the default columns of verdi process list.

Revision ID: 26d561acd560 Revises: 07fac78e6209 Create Date: 2019-06-22 09:55:25.284168

aiida.backends.sqlalchemy.migrations.versions.26d561acd560_data_migration_legacy_job_calculations.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.26d561acd560_data_migration_legacy_job_calculations.upgrade()[source]

Migrations for the upgrade.

Migrating ‘hidden’ properties from DbAttribute to DbExtra for code.Code. nodes

Revision ID: 35d4ee9a1b0e Revises: 89176227b25 Create Date: 2018-02-21 22:00:43.460534

aiida.backends.sqlalchemy.migrations.versions.35d4ee9a1b0e_code_hidden_attr_to_extra.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.35d4ee9a1b0e_code_hidden_attr_to_extra.upgrade()[source]

This migration adds uniqueness constraint to the UUID column.

This migration corresponds to the 0024_dblog_update Django migration.

Revision ID: 375c2db70663 Revises: ea2f50e7f615 Create Date: 2019-01-30 20:26:16.550071

aiida.backends.sqlalchemy.migrations.versions.375c2db70663_dblog_uuid_uniqueness_constraint.downgrade()[source]

Remove unique key constraint to the UUID column.

aiida.backends.sqlalchemy.migrations.versions.375c2db70663_dblog_uuid_uniqueness_constraint.upgrade()[source]

Add unique key constraint to the UUID column.

Make all uuid columns unique

Revision ID: 37f3d4882837 Revises: 6a5c2ea1439d Create Date: 2018-11-17 17:18:58.691209

aiida.backends.sqlalchemy.migrations.versions.37f3d4882837_make_all_uuid_columns_unique.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.37f3d4882837_make_all_uuid_columns_unique.upgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.37f3d4882837_make_all_uuid_columns_unique.verify_uuid_uniqueness(table)[source]

Check whether the database contains duplicate UUIDS.

Note that we have to redefine this method from aiida.manage.database.integrity.verify_uuid_uniqueness because that uses the default database connection, while here the one created by Alembic should be used instead.

Raises

IntegrityError if database contains nodes with duplicate UUIDS.

Remove DbComputer.enabled

Revision ID: 3d6190594e19 Revises: 5a49629f0d45 Create Date: 2019-04-03 14:38:50.585639

aiida.backends.sqlalchemy.migrations.versions.3d6190594e19_remove_dbcomputer_enabled.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.3d6190594e19_remove_dbcomputer_enabled.upgrade()[source]

Adding indexes and constraints to the dbnode-dbgroup relationship table

Revision ID: 59edaf8a8b79 Revises: a514d673c163 Create Date: 2018-06-22 14:50:18.447211

aiida.backends.sqlalchemy.migrations.versions.59edaf8a8b79_adding_indexes_and_constraints_to_the_.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.59edaf8a8b79_adding_indexes_and_constraints_to_the_.upgrade()[source]

Adding indices on the input_id, output_id and type column of the DbLink table

Revision ID: 5a49629f0d45 Revises: 5ddd24e52864 Create Date: 2019-03-04 16:38:42.249231

Migrations for the downgrade.

Migrations for the upgrade.

Invalidating node hash - User should rehash nodes for caching

Revision ID: 5d4d844852b6 Revises: 62fe0d36de90 Create Date: 2018-10-26 17:14:33.566670

aiida.backends.sqlalchemy.migrations.versions.5d4d844852b6_invalidating_node_hash.downgrade()[source]

drop the hashes also when downgrading

aiida.backends.sqlalchemy.migrations.versions.5d4d844852b6_invalidating_node_hash.upgrade()[source]

drop the hashes when upgrading

Renaming DbNode.type to DbNode.node_type

Revision ID: 5ddd24e52864 Revises: d254fdfed416 Create Date: 2019-02-22 17:09:57.715114

aiida.backends.sqlalchemy.migrations.versions.5ddd24e52864_dbnode_type_to_dbnode_node_type.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.5ddd24e52864_dbnode_type_to_dbnode_node_type.upgrade()[source]

Migrations for the upgrade.

Final data migration for Nodes after aiida.orm.nodes reorganization was finalized to remove the node. prefix

Revision ID: 61fc0913fae9 Revises: ce56d84bcc35 Create Date: 2019-02-16 15:32:42.745450

aiida.backends.sqlalchemy.migrations.versions.61fc0913fae9_remove_node_prefix.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.61fc0913fae9_remove_node_prefix.upgrade()[source]

Migrations for the upgrade.

Add a unique constraint on the UUID column of the Node model

Revision ID: 62fe0d36de90 Revises: 59edaf8a8b79 Create Date: 2018-07-02 17:50:42.929382

aiida.backends.sqlalchemy.migrations.versions.62fe0d36de90_add_node_uuid_unique_constraint.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.62fe0d36de90_add_node_uuid_unique_constraint.upgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.62fe0d36de90_add_node_uuid_unique_constraint.verify_node_uuid_uniqueness()[source]

Check whether the database contains nodes with duplicate UUIDS.

Note that we have to redefine this method from aiida.manage.database.integrity.verify_node_uuid_uniqueness because that uses the default database connection, while here the one created by Alembic should be used instead.

Raises

IntegrityError if database contains nodes with duplicate UUIDS.

Data migration for Data nodes after it was moved in the aiida.orm.node module changing the type string.

Revision ID: 6a5c2ea1439d Revises: 375c2db70663 Create Date: 2019-01-18 19:44:32.156083

aiida.backends.sqlalchemy.migrations.versions.6a5c2ea1439d_move_data_within_node_module.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.6a5c2ea1439d_move_data_within_node_module.upgrade()[source]

Migrations for the upgrade.

Add the process_type column to DbNode

Revision ID: 6c629c886f84 Revises: 0aebbeab274d Create Date: 2018-03-15 13:23:12.941148

aiida.backends.sqlalchemy.migrations.versions.6c629c886f84_process_type.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.6c629c886f84_process_type.upgrade()[source]

Deleting dbpath table and triggers

Revision ID: 70c7d732f1b2 Revises: Create Date: 2017-10-17 10:30:23.327195

aiida.backends.sqlalchemy.migrations.versions.70c7d732f1b2_delete_dbpath.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.70c7d732f1b2_delete_dbpath.upgrade()[source]

Unique constraints for the db_dbgroup_dbnodes table

Revision ID: 7a6587e16f4c Revises: 35d4ee9a1b0e Create Date: 2019-02-11 19:25:11.744902

aiida.backends.sqlalchemy.migrations.versions.7a6587e16f4c_unique_constraints_for_the_db_dbgroup_.downgrade()[source]

Remove unique constraints from the db_dbgroup_dbnodes table.

aiida.backends.sqlalchemy.migrations.versions.7a6587e16f4c_unique_constraints_for_the_db_dbgroup_.upgrade()[source]

Add unique constraints to the db_dbgroup_dbnodes table.

Migration of ProcessNode attributes for metadata options whose key changed.

Revision ID: 7ca08c391c49 Revises: e72ad251bcdb Create Date: 2019-01-15 15:03:43.876133

aiida.backends.sqlalchemy.migrations.versions.7ca08c391c49_calc_job_option_attribute_keys.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.7ca08c391c49_calc_job_option_attribute_keys.upgrade()[source]

Migration of ProcessNode attributes for metadata options whose key changed.

Renamed attribute keys:

  • custom_environment_variables -> environment_variables (CalcJobNode)

  • jobresource_params -> resources (CalcJobNode)

  • _process_label -> process_label (ProcessNode)

  • parser -> parser_name (CalcJobNode)

Deleted attributes:
  • linkname_retrieved (We do not actually delete it just in case some relies on it)

Add indexes to dbworkflowdata table

Revision ID: 89176227b25 Revises: Create Date: 2017-11-03 11:06:00.327195

aiida.backends.sqlalchemy.migrations.versions.89176227b25_add_indexes_to_dbworkflowdata_table.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.89176227b25_add_indexes_to_dbworkflowdata_table.upgrade()[source]

Drop the DbLock model

Revision ID: a514d673c163 Revises: f9a69de76a9a Create Date: 2018-05-10 19:08:51.780194

aiida.backends.sqlalchemy.migrations.versions.a514d673c163_drop_dblock.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.a514d673c163_drop_dblock.upgrade()[source]

Correct the type string for the code class

Revision ID: a603da2cc809 Revises: 5d4d844852b6 Create Date: 2018-11-13 18:15:07.300709

aiida.backends.sqlalchemy.migrations.versions.a603da2cc809_code_sub_class_of_data.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.a603da2cc809_code_sub_class_of_data.upgrade()[source]

Updating link types - This is a copy of the Django migration script

Revision ID: a6048f0ffca8 Revises: Create Date: 2017-10-17 10:51:23.327195

aiida.backends.sqlalchemy.migrations.versions.a6048f0ffca8_update_linktypes.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.a6048f0ffca8_update_linktypes.upgrade()[source]

DbGroup class: Rename name with label and type with type_string

Revision ID: b8b23ddefad4 Revises: 239cea6d2452 Create Date: 2018-12-06 15:25:32.865136

aiida.backends.sqlalchemy.migrations.versions.b8b23ddefad4_dbgroup_name_to_label_type_to_type_string.downgrade()[source]

The downgrade migration actions.

aiida.backends.sqlalchemy.migrations.versions.b8b23ddefad4_dbgroup_name_to_label_type_to_type_string.upgrade()[source]

The upgrade migration actions.

Delete trajectory symbols array from the repository and the reference in the attributes

Revision ID: ce56d84bcc35 Revises: 12536798d4d3 Create Date: 2019-01-21 15:35:07.280805

aiida.backends.sqlalchemy.migrations.versions.ce56d84bcc35_delete_trajectory_symbols_array.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.ce56d84bcc35_delete_trajectory_symbols_array.upgrade()[source]

Migrations for the upgrade.

Data migration for after ParameterData was renamed to Dict.

Revision ID: d254fdfed416 Revises: 61fc0913fae9 Create Date: 2019-02-25 19:29:11.753089

aiida.backends.sqlalchemy.migrations.versions.d254fdfed416_rename_parameter_data_to_dict.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.d254fdfed416_rename_parameter_data_to_dict.upgrade()[source]

Migrations for the upgrade.

Drop various columns from the DbUser model.

These columns were part of the default Django user model

Revision ID: de2eaf6978b4 Revises: 1830c8430131 Create Date: 2019-05-28 11:15:33.242602

aiida.backends.sqlalchemy.migrations.versions.de2eaf6978b4_simplify_user_model.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.de2eaf6978b4_simplify_user_model.upgrade()[source]

Migrations for the upgrade.

Initial schema

Revision ID: e15ef2630a1b Revises: Create Date: 2017-06-28 17:12:23.327195

aiida.backends.sqlalchemy.migrations.versions.e15ef2630a1b_initial_schema.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.e15ef2630a1b_initial_schema.upgrade()[source]

DbGroup class: change type_string values

Revision ID: e72ad251bcdb Revises: b8b23ddefad4 Create Date: 2018-12-06 19:34:47.732890

aiida.backends.sqlalchemy.migrations.versions.e72ad251bcdb_dbgroup_class_change_type_string_values.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.e72ad251bcdb_dbgroup_class_change_type_string_values.upgrade()[source]

Data migration for some legacy process attributes.

Attribute keys that are renamed:

  • _sealed -> sealed

Attribute keys that are removed entirely:

  • _finished

  • _failed

  • _aborted

  • _do_abort

Finally, after these first migrations, any remaining process nodes that still do not have a sealed attribute and have it set to True. Excluding the nodes that have a process_state attribute of one of the active states created, running` or waiting, because those are actual valid active processes that are not yet sealed.

Revision ID: e734dd5e50d7 Revises: e797afa09270 Create Date: 2019-07-04 18:23:56.127994

aiida.backends.sqlalchemy.migrations.versions.e734dd5e50d7_data_migration_legacy_process_attributes.downgrade()[source]

Migrations for the downgrade.

aiida.backends.sqlalchemy.migrations.versions.e734dd5e50d7_data_migration_legacy_process_attributes.upgrade()[source]

Migrations for the upgrade.

Invalidating node hash - User should rehash nodes for caching

Revision ID: e797afa09270 Revises: 26d561acd560 Create Date: 2019-07-01 19:39:33.605457

aiida.backends.sqlalchemy.migrations.versions.e797afa09270_reset_hash.downgrade()[source]

drop the hashes also when downgrading

aiida.backends.sqlalchemy.migrations.versions.e797afa09270_reset_hash.upgrade()[source]

drop the hashes when upgrading

This migration creates UUID column and populates it with distinct UUIDs

This migration corresponds to the 0024_dblog_update Django migration.

Revision ID: ea2f50e7f615 Revises: 041a79fc615f Create Date: 2019-01-30 19:22:50.984380

aiida.backends.sqlalchemy.migrations.versions.ea2f50e7f615_dblog_create_uuid_column.downgrade()[source]

Remove the UUID column

aiida.backends.sqlalchemy.migrations.versions.ea2f50e7f615_dblog_create_uuid_column.set_new_uuid(connection)[source]

Set new and distinct UUIDs to all the logs

aiida.backends.sqlalchemy.migrations.versions.ea2f50e7f615_dblog_create_uuid_column.upgrade()[source]

Add an UUID column an populate it with unique UUIDs

Delete the kombu tables that were used by the old Celery based daemon and the obsolete related timestamps

Revision ID: f9a69de76a9a Revises: 6c629c886f84 Create Date: 2018-05-10 15:07:59.235950

aiida.backends.sqlalchemy.migrations.versions.f9a69de76a9a_delete_kombu_tables.downgrade()[source]
aiida.backends.sqlalchemy.migrations.versions.f9a69de76a9a_delete_kombu_tables.upgrade()[source]