Migration archive files from old export versions to the newest, used by verdi export migrate command.
Utility functions for migration of export-files.
aiida.tools.importexport.archive.migrations.utils.
remove_fields
Remove fields under entities from data.json and metadata.json.
metadata – the content of an export archive metadata.json file
data – the content of an export archive data.json file
entities – list of ORM entities
fields – list of fields to be removed from the export archive files
update_metadata
Update the metadata with a new version number and a notification of the conversion that was executed.
version – string version number that the updated metadata should get
verify_metadata_version
Utility function to verify that the metadata has the correct version number.
If no version number is passed, it will just extract the version number and return it.
version – string version number that the metadata is expected to have
Migration from v0.1 to v0.2, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v01_to_v02.
get_new_string
Replace the old module prefix with the new.
migrate_v1_to_v2
Migration of archive files from v0.1 to v0.2, which means generalizing the field names with respect to the database backend
replace_requires
Replace the requires keys with new module path.
Migration from v0.2 to v0.3, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v02_to_v03.
migrate_v2_to_v3
Migration of archive files from v0.2 to v0.3, which means adding the link types to the link entries and making the entity key names backend agnostic by effectively removing the prefix ‘aiida.backends.djsite.db.models’
Migration from v0.3 to v0.4, used by verdi export migrate command.
The migration steps are named similarly to the database migrations for Django and SQLAlchemy. In the description of each migration, a revision number is given, which refers to the Django migrations. The individual Django database migrations may be found at:
aiida.backends.djsite.db.migrations.00XX_<migration-name>.py
Where XX are the numbers in the migrations’ documentation: REV. 1.0.XX And migration-name is the name of the particular migration. The individual SQLAlchemy database migrations may be found at:
aiida.backends.sqlalchemy.migrations.versions.<id>_<migration-name>.py
Where id is a SQLA id and migration-name is the name of the particular migration.
aiida.tools.importexport.archive.migrations.v03_to_v04.
add_extras
Update data.json with the new Extras Since Extras were not available previously and usually only include hashes, the Node ids will be added, but included as empty dicts
migrate_v3_to_v4
Migration of archive files from v0.3 to v0.4
Note concerning migration 0032 - REV. 1.0.32: Remove legacy workflow tables: DbWorkflow, DbWorkflowData, DbWorkflowStep These were (according to Antimo Marrazzo) never exported.
migration_add_node_uuid_unique_constraint
Apply migration: 0014 - REV. 1.0.14, 0018 - REV. 1.0.18 Check that no entries with the same uuid are present in the archive file if yes - stop the import process
migration_base_data_plugin_type_string
Apply migration: 0009 - REV. 1.0.9 DbNode.type content changes: ‘data.base.Bool.’ -> ‘data.bool.Bool.’ ‘data.base.Float.’ -> ‘data.float.Float.’ ‘data.base.Int.’ -> ‘data.int.Int.’ ‘data.base.Str.’ -> ‘data.str.Str.’ ‘data.base.List.’ -> ‘data.list.List.’
migration_calc_job_option_attribute_keys
Apply migrations: 0023 - REV. 1.0.23 custom_environment_variables -> environment_variables jobresource_params -> resources _process_label -> process_label parser -> parser_name
migration_code_sub_class_of_data
Apply migrations: 0016 - REV. 1.0.16 The Code class used to be just a sub class of Node, but was changed to act like a Data node. code.Code. -> data.code.Code.
migration_dbgroup_name_to_label_type_to_type_string
Apply migrations: 0021 - REV. 1.0.21 Rename dbgroup fields: name -> label type -> type_string
migration_dbgroup_type_string_change_content
Apply migrations: 0022 - REV. 1.0.22 Change type_string according to the following rule: ‘’ -> ‘user’ ‘data.upf.family’ -> ‘data.upf’ ‘aiida.import’ -> ‘auto.import’ ‘autogroup.run’ -> ‘auto.run’
migration_dbnode_type_to_dbnode_node_type
Apply migration: 0030 - REV. 1.0.30 Renaming DbNode.type to DbNode.node_type
migration_migrate_builtin_calculations
Apply migrations: 0019 - REV. 1.0.19 Remove ‘simpleplugin’ from ArithmeticAddCalculation and TemplatereplacerCalculation type
ATTENTION:
The ‘process_type’ column did not exist before migration 0010, consequently, it could not be present in any export archive of the currently existing stable releases (0.12.*). Here, however, the migration acts on the content of the ‘process_type’ column, which could only be introduced in alpha releases of AiiDA 1.0. Assuming that ‘add’ and ‘templateplacer’ calculations are expected to have both ‘type’ and ‘process_type’ columns, they will be added based solely on the ‘type’ column content (unlike the way it is done in the DB migration, where the ‘type_string’ content was also checked).
migration_move_data_within_node_module
Apply migrations: 0025 - REV. 1.0.25 The type string for Data nodes changed from data.* to node.data.*.
migration_process_type
Apply migrations: 0010 - REV. 1.0.10 Add DbNode.process_type column
migration_provenance_redesign
Apply migration: 0020 - REV. 1.0.20 Provenance redesign
migration_remove_dbcomputer_enabled
Apply migration: 0031 - REV. 1.0.31 Remove DbComputer.enabled
migration_remove_node_prefix
Apply migrations: 0028 - REV. 1.0.28 Change node type strings: ‘node.data.’ -> ‘data.’ ‘node.process.’ -> ‘process.’
migration_rename_parameter_data_to_dict
Apply migration: 0029 - REV. 1.0.29 Update ParameterData to Dict
migration_replace_text_field_with_json_field
Apply migration 0033 - REV. 1.0.33 Store dict-values as JSON serializable dicts instead of strings NB! Specific for Django backend
migration_trajectory_symbols_to_attribute
Apply migrations: 0026 - REV. 1.0.26 and 0027 - REV. 1.0.27 Create the symbols attribute from the repository array for all TrajectoryData nodes.
Migration from v0.4 to v0.5, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v04_to_v05.
migrate_v4_to_v5
Migration of archive files from v0.4 to v0.5
This is from migration 0034 (drop_node_columns_nodeversion_public) and onwards
migration_drop_computer_transport_params
Apply migration 0036 - REV. 1.0.36 Drop the column transport_params from the Computer model
migration_drop_node_columns_nodeversion_public
Apply migration 0034 - REV. 1.0.34 Drop the columns nodeversion and public from the Node model
Migration from v0.5 to v0.6, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v05_to_v06.
migrate_deserialized_datetime
Deserialize datetime strings from export archives, meaning to reattach the UTC timezone information.
migrate_v5_to_v6
Migration of archive files from v0.5 to v0.6
migration_migrate_legacy_job_calculation_data
Apply migration 0038 - REV. 1.0.38
Migrates legacy JobCalculation data to the new process system. Essentially old JobCalculation nodes, which have already been migrated to CalcJobNodes, are missing important attributes process_state, exit_status and process_status. These are inferred from the old state attribute, which is then discarded as its values have been deprecated.
migration_serialize_datetime_objects
Apply migration 0037 - REV. 1.0.37
Migrates the node attributes and extras from the EAV schema to JSONB columns. Since JSON does not support datetime objects, and the EAV did, existing datetime objects have to be serialized to strings. Just like the database migration they were serialized to the standard ISO format, except that they were first converted to UTC timezone and then the stored without a timezone reference. Since existing datetimes in the attributes and extras in the database were timezone aware and have been migrated to an ISO format string including the timezone information we should now add the same timezone information to datetime attributes and extras in existing export archives. All that one needs to do for this is to append the +00:00 suffix, which signifies the UTC timezone.
Since the datetime objects were the only types being serialized in the attributes and extras, after the reinstating of the timeonze information, there is no longer a need for the de/serialization dictionaries for each node, stored in node_attributes_conversion and node_extras_conversion, respectively. They are no longer added to new archives and so they can and should be removed from existing archives, reducing the size enormously.
Migration from v0.6 to v0.7, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v06_to_v07.
data_migration_legacy_process_attributes
Apply migration 0040 - REV. 1.0.40 Data migration for some legacy process attributes.
Attribute keys that are renamed:
_sealed -> sealed
Attribute keys that are removed entirely:
_finished
_failed
_aborted
_do_abort
Finally, after these first migrations, any remaining process nodes are screened for the existence of the process_state attribute. If they have it, it is checked whether the state is active or not, if not, the sealed attribute is created and set to True.
CorruptArchive – if a Node, found to have attributes, cannot be found in the list of exported entities.
CorruptArchive – if the ‘sealed’ attribute does not exist and the ProcessNode is in an active state, i.e. process_state is one of (‘created’, ‘running’, ‘waiting’). A log-file, listing all illegal ProcessNodes, will be produced in the current directory.
migrate_v6_to_v7
Migration of archive files from v0.6 to v0.7
remove_attribute_link_metadata
Remove Attribute and Link from metadata.json This is not a database migration, but purely to do with the import/export schema.
The “entities” ATTRIBUTE_ENTITY_NAME and LINK_ENTITY_NAME were introduced in v0.3 or v0.4 of the import/export schema. However, they were never used - or the usage has been reverted. They will be removed from metadata.json, slightly simplifying the import functions as well.
Migration from v0.7 to v0.8, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v07_to_v08.
migrate_v7_to_v8
Migration of archive files from v0.7 to v0.8.
migration_default_link_label
Apply migration 0043 - REV. 1.0.43
Rename all link labels _return to result.
Migration from v0.8 to v0.9, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v08_to_v09.
migrate_v8_to_v9
Migration of archive files from v0.8 to v0.9.
migration_dbgroup_type_string
Apply migration 0044 - REV. 1.0.44
Rename the type_string columns of all Group instances.
Migration from v0.9 to v0.10, used by verdi export migrate command.
aiida.tools.importexport.archive.migrations.v09_to_v10.
migrate_v9_to_v10
Migration of archive files from v0.9 to v0.10.