Release Notes v1.9

Version: 1.9

Released: February 2024

What’s new

Features

Support for vault data package versioning
Support for configuring metadata schemas per research group
Support for configuring retention period per research group
Support for basic controlled vocabularies in metadata schemas
Support for Creative Commons (BY-ND, BY-NC, BY-NC-ND) and GPL v3 licenses
Support for Data Access Passwords expiration notifications
Support for research group retention period notifications
Support for CSV group imports
Support for color mode user setting
Support for multiple replication resources
Support for configuring iRODS S3 resources
Support for database connection pooling with PgBouncer
Support for Automatic Resource Balancing
Support for viewing text files in portal
Support for multiple deposit groups
Support for groups connected to SRAM
Experimental support for vault archiving workflow
Improved overwrite actions in research space
Upgrade iRODS to v4.2.12
Upgrade python-irodsclient to v1.1.9
Upgrade davrods to v1.5.1
Upgrade to PostgreSQL 15

Known issues

Collections with single apex “’” in the name do not work irods/irods#5727
Deadlock in msiDataObjRepl & msiDataObjCopy when called from Python irods_rule_engine_plugin_python#54

Upgrading from previous release

The playbook requires Ansible 2.11.x or higher. Ansible 2.17.0 and higher is not yet supported.

Version constraints:

Requires Yoda external user service to be on version 1.9.x or higher.
Requires Yoda public server to be on version 1.9.x or higher.

Backup/copy custom configurations made to Yoda version 1.8. To view what files were changed from the defaults, run git diff.
After making sure the configurations are stored safely in another folder, reset the Yoda folder using git stash or when you want to delete all changes made: git reset --hard.
Checkout tag v1.9.3 of the Yoda Git repository.
```
git checkout v1.9.3
```
Set the Yoda version to v1.9.3 in the configuration.
```
yoda_version: v1.9.3
```
Change the default schema from default-2 to default-3 in the configuration. Person identifiers must be valid and new dependency between license and data access restriction. This requires an intervention by the responsible datamanager beforehand.
```
default_yoda_schema: default-3
```
Metadata schemas are configurable per research group. To configure the metadata schemas available the metadata_schemas configuration can be used. See also this documentation for more information on installing metadata schemas.
```
metadata_schemas:
  - name: default-2
 install: true
 active: false
  - name: default-3
 install: true
 active: true
```
Yoda version 1.9 restricts access to the anonymous account. If you run DavRODS on a separate server from the provider, you need to configure Yoda to permit access to the anonymous account from the DavRODS server using the new irods_anonymous_account_permit_addresses configuration parameter.

If you use the External User Service (EUS): some EUS parameters have changed from Yoda 1.8 to 1.9. Yoda 1.9 performs server certificate validation of requests from the provider to the EUS server by default. This can be disabled by setting eus_api_tls_verify to false. For some SMTP parameters, EUS uses joint parameters with the provider in Yoda 1.9. Please see the configuration guide for more information.

| Old parameter (1.8) | New parameter  | Notes                                |
|---------------------|----------------|--------------------------------------|
| eus_smtp_host       | smtp_server    | New format. e.g. smtp://localhost:25 |
| eus_smtp_port       | smtp_server    | New format. e.g. smtp://localhost:25 |
| eus_smtp_auth       | smtp_auth      |                                      |
| eus_smtp_security   | smtp_server    | New format. e.g. smtp://localhost:25 |

Unless your Yoda environment has already been upgraded to PostgreSQL 15, you should upgrade PostgreSQL during or immediately after the Yoda upgrade. Please consult the PostgreSQL upgrade information page for information about how to perform the upgrade. Example configuration:
```
postgresql_perform_db_upgrade: true
postgresql_remove_old_data_after_upgrade: false
```

Install all Ansible collections needed to deploy Yoda:

ansible-galaxy collection install -r requirements.yml

Run the Ansible playbook in check mode.

ansible-playbook -i <path-to-your-environment> playbook.yml --check
### EXAMPLE ###
ansible-playbook -i /environments/development/allinone playbook.yml --check

If the playbook has finished successfully in check mode, run the Ansible playbook normally.

ansible-playbook -i <path-to-your-environment> playbook.yml
### EXAMPLE ###
ansible-playbook -i /environments/development/allinone playbook.yml

Update statistics storage data to the latest format.

irule -r irods_rule_engine_plugin-irods_rule_language-instance -F /etc/irods/yoda-ruleset/tools/transform-storage-data.r

Report vault data package metadata containing invalid ORCID person identifiers.

irule -r irods_rule_engine_plugin-python-instance -F /etc/irods/yoda-ruleset/tools/metadata/vault-check-orcid-format.r

Correct vault data package metadata containing invalid ORCID person identifiers.

irule -r irods_rule_engine_plugin-irods_rule_language-instance -F /etc/irods/yoda-ruleset/tools/metadata/vault-correct-orcid-format.r

Update all publication metadata to support DOI versioning.

irule -r irods_rule_engine_plugin-python-instance -F /etc/irods/yoda-ruleset/tools/transform-existing-publications.r

Update all metadata JSON in the vault to latest metadata JSON version (default-2 to default-3).

irule -r irods_rule_engine_plugin-irods_rule_language-instance -F /etc/irods/yoda-ruleset/tools/check-metadata-for-schema-updates.r

In most cases, the previous steps will transform metadata of vault data packages to the new schema. In some specific cases, in particular when the metadata contains person identifiers in an unexpected format, you may need to edit vault metadata manually in order to make it compliant with the new schema.

In order to do this, first run a metadata schema report to see which data packages have metadata that is not yet compliant with the new schema:

/usr/bin/irule -r irods_rule_engine_plugin-python-instance -F /etc/irods/yoda-ruleset/tools/metadata/vault-metadata-schema-report.r

If the metadata schema report shows that any data packages have metadata that is not compliant with the schema, please consult the vault metadata troubleshooting documentation for instructions on how to resolve this manually.

Update publication endpoints if there are published packages (DataCite, landingpages and OAI-PMH):

irule -r irods_rule_engine_plugin-irods_rule_language-instance -F /etc/irods/yoda-ruleset/tools/update-publications.r

When upgrading an SRAM-enabled environment to Yoda 1.9.5 or later, please note that the sram_co_logo_url parameter (which took a URL) has been changed to sram_co_logo (which takes a BASE64-encoded image). If you have defined a custom logo, you will need to change this parameter in your configuration.