Kuha OAI-PMH Repo Handler Changelog

1.8.0 (2024-08-22)

Added

  • Support Python 3.11 & Python 3.12.

Changed

  • Update dependencies in requirements.txt to support Python 3.12:

    • Genshi 0.7.9

    • Kuha Common 2.5.0

    • Tornado 6.4.1

    • Py12fLogging 0.7.0

  • Remove setuptools from requirements.txt. Use importlib.metadata to facilitate plugin discovery.

Fixed

  • The submitted resumptionToken value is validated and BadResumptionToken OAI-PMH error is rendered if it is invalid. Invalid resumptionToken -message is also logged with WARNING severity.

1.7.0 (2024-05-24)

Changed

  • Don’t render empty IDNo & holdings elements inside relPubl in metadataprefixes ddi_c and oai_ddi25. CESSDA validator considers IDNo elements without agency-attribute as invalid according to CDC Profile 2.5.

Fixed

1.6.0 (2024-04-25)

Maintenance release due to move to https://gitlab.tuni.fi/fsd/kuha_oai_pmh_repo_handler.

Changed

  • Hosting platform changed from bitbucket.org to gitlab.tuni.fi. Update documentation & requirements.txt accordingly.

1.5.0 (2024-03-15)

Added

  • Support Study.principal_investigator attributes external_link, external_link_uri, external_link_role and external_link_title.

  • Render /CodeBook/stdyDscr/citation/rspStmt/AuthEnty/ExtLink to DDI-C (metadataprefixes ddi_c and oai_ddi25).

Changed

  • Require Kuha Common 2.3.0 in requirements.txt and setup.py.

1.4.1 (2023-12-08)

Fixed

  • The stylesheet static/oai2.xsl no longer misinterprets that an empty resumptionToken element means that a list response contains more results. (Fixes #39)

1.4.0 (2023-10-24)

Added

  • The script kuha_list_records now outputs deleted record identifiers separately.

Fixed

  • OAIDataciteMetadataFormat.get_funders() checks that grant_number.get_value() is not None, before calling startswith().

1.3.0 (2023-06-29)

This release contains API changes to facilitate changes in derivative projects. No functional changes in this application.

Added

  • Support using custom WebApplication class in http_api.get_app().

  • Add current OAI requests Protocol instance to handler instance attribute ‘oai_protocol’.

1.2.0 (2022-12-21)

Added

  • Add XML Stylesheet to make OAI responses more human-friendly. (Implements #38)

  • Configuration option to control XML Stylesheets (--oai-pmh-stylesheet-url):

    • Set to empty string to disable stylesheets completely.

    • Set to a full URL to serve the stylesheet from some external file server.

    • Start with a slash (‘/’) to serve via Kuha OAI-PMH Repo Handler server.

    • Defaults to ‘/v0/oai/static/oai2.xsl’, which works with other default configuration values and uses Kuha OAI-PMH Repo Handler server to actually serve the file.

1.1.1 (2022-11-25)

Fixed

  • Remove unallowed xml:lang attributes from oai_datacite serialization. (Fixes `37`_)

  • Wrap geoLocationPlace inside a geoLocation element in oai_datacite serialization. (Fixes `37`_)

1.1.0 (2022-10-27)

Added

  • Grant & funding information to oai_ddi25, ddi_c and oai_datacite metadata. (Implements #36)

  • Related publication identifiers and agencies to oai_ddi25, ddi_c and oai_datacite metadata. (Implements #36)

  • Include Python 3.10 in CI test environment.

Changed

  • Require Kuha Common 2.0.0.

  • Update requirements.txt:

    • Tornado 6.2.0

    • py12flogging 0.5.0

    • Genshi 0.7.7

1.0.2 (2022-08-25)

Changed

  • Add primary lookup to oai_datacite Publisher from Study.distributors. The current lookup from Study.publishers will remain as a secondary source. (Fixes #35)

Fixed

  • Change lookup order of preferred PublicationYear value for oai_datacite. (Fixes #33)

  • Format PublicationYear value for oai_datecite so that it is a year, instead of full datestamp. (Fixes #33)

  • Include mandatory Date property to oai_datacite. (Fixes #34)

1.0.1 (2022-06-09)

Fixed

  • Render subject in oai_dc metadata regardless if the study.keyword has a value. (Fixes #31)

1.0.0 (2022-03-16)

Added

  • Support for deleted records. (Implements #3)

    • The deletedRecords value (in response to Identify-verb) is configurable via --oai-pmh-deleted-records and defaults to value transient.

  • Configuration option to limit the list size of each metadataformat separately. (Implements #2)

  • New metadataformat OAI Datacite with prefix oai_datacite.

  • Plugin architecture for developing arbitrary metadataformats & OAI-sets.

  • Support insecure (unverified SSL certificate chain) https connections in list_records.py via configuration flag --insecure. Default is to reject insecure connections.

  • Generate setName values for language and data_kind sets which do not have one. (Implements #24)

  • Command line entrypoint kuha_list_records outputs deleted record count and supports HTTP Basic Authentication. (Implements #25)

Changed

  • Require Kuha Common 1.0.0.

  • Plenty of API changes to facilitate the developing of metadataformats as plugins:

    • Delete configure.py. Serve-endpoint configuration is now declared in serve.py. Modules declare configurations via their respective add_cli_args functions.

    • Add controller.py. The controller is responsible for gathering the OAI responses from metadataformats.

    • Add http_api.py. This module declares the HTTP interface and dispatches requests to the configured controller. The module contains minimum amount of non-server imports and calls to controller.

    • Add metadataformats-package, which replaces the oai/metadata_formats.py and oai/records.py modules.

    • serve.py loads and configures metadataformats using entrypoint-group kuha.oai.metadataformats. No metadataformats are directly imported.

    • oai/__init__.py now contains add_cli_args & configure methods to configure package classes. It defines a Protocol-class which wraps all OAI-PMH protocol objects.

  • Update dependencies in requirements.txt.

    • Tornado 6.1.0

    • ConfigArgParse 1.5.3

    • six 1.16.0

Removed

  • Drop support for Python versions below Python 3.8.

Fixed

  • Repeat EAD3 daterange element when a record has multiple date ranges. (Fixes #20)

  • Metadata rendered by prefix oai_ddi25 should place distrbtr-element before distDate-element in distStmt. (Fixes #21)

0.14.1 (2021-11-11)

  • Metadata rendered by prefix oai_ddi25 should place distrbtr-element before distDate-element in distStmt. (Fixes #21)

0.14.0 (2021-09-06)

  • Ensure future compatibility with CESSDA Data Catalogue

    • Require kuha_common 0.15.1

    • Include elements /codeBook/docDscr/citation/titlStmt/titl and /codeBook/stdyDscr/citation/holdings/@URI to DDI 2.5 serializations.

    • Use study.document_titles to populate /codeBook/docDscr/citation/titlStmt/titl and study.study_uris to populate /codeBook/stdyDscr/citation/holdings/@URI.

  • Drop testing with Python interpreters below py38.

    • Add py39 to test environments.

    • Remove py35, py36 and py37 from tox test environments.

    • Remove tasks running tests with py35, py36 and py37 interpreters from Jenkinsfile.

0.13.0 (2021-02-02)

  • Changes to EAD3 mapping:

    • /ead/archdesc/dsc/c01/c02/did/daoset/dao/@daotype value is now “unknown” instead of “derived”

    • study.principal_investigators are now divided into corpname & persname elements. If a principal investigator has an affiliated organization the value is placed into persname and the organization is placed into corpname. If a principal investigator has no affiliated organization, it’s value is expected to be an organization and the value is placed into corpname.

  • Lock pip version to 20.3.4 in install script, which is the latest pip that supports Python 3.5. The install script should be compatible with Ubuntu 16.04, which defaults to Python 3.5. The latest pip does not support Python 3.5 and therefore cannot be upgraded into.

  • Use ‘’python-latest’’ in Jenkinsfile. It is guaranteed to point to the latest python on the CI server. Note that this is a CI specific configuration and is not portable to some arbitrary Jenkins instance.

  • Add py39 to tox environments and use it in Jenkinsfile.

  • Upgrade to latest genshi 0.7.5 in requirements.txt. Python 3.9 introduced changes to ast module that were incompatible with previous genshi version 0.7.3 used.

  • Add six to requirements.txt, since genshi 0.7.5 requires it.

0.12.1 (2020-12-08)

  • Fix unhandled AttributeError, when requesting a list response with an unsupported set-parameter. Respond with OAI error code “noRecordsMatch”. (Fixes #19)

0.12.0 (2020-06-12)

  • Support Python 3.8: involves code changes to tests and requirement of kuha_common 0.14.0. No new functionality or bug fixes are introduced.

0.11.1 (2020-06-05)

  • kuha_oai_pmh_repo_handler.oai.records.get_sets_list_from_query_result() Check for query_result item’s set.record_field_setspec using dict’s get-method, since the item may not have such key. Remove check for the record_field_setname, since it is acceptable to have an OAI set without a setName. (Fixes #18)

0.11.0 (2020-05-07)

  • Use Study.related_publications to render relpubl elements in DDI 2.5. templates.

  • require kuha_common 0.13.0

0.10.0 (2020-04-29)

  • Project source code management changed to git. Does not contain code changes to OAI-PMH Repo Handler.

  • INSTALL.rst now instructs to use Git instead of Mercurial.

  • requirements.txt: kuha_common 0.12.0

0.9.0 (2020-03-19)

  • Add list_records.py to run through the entire ListRecords sequence on-demand.

  • setup.py: Create console script entry point to run list_records.py.

  • Add shell script to run the list_records entry point using runtime_env and Python virtualenv.

0.8.0 (2020-01-22)

  • Add rights element to OAI-DC serialization.

0.7.1 (2020-01-10)

  • Fixes for EAD3 serialization:

    • Correct schemaLocation declaration.

    • Add missing daotype attribute which is required for dao-element.

    • Don’t render empty langmaterial elements.

0.7.0 (2020-01-09)

  • Support for EAD3 metadata with metadataPrefix ead3

  • Use study.data_kinds as an OAI-PMH set.

  • Render following DDI 2.5. elements in ddi_c and oai_ddi25 metadata:

    • /codeBook/stdyDscr/citation/prodStmt/copyright

    • /codeBook/stdyDscr/dataAccs/useStmt/citReq

    • /codeBook/stdyDscr/dataAccs/useStmt/deposReq

    • /codeBook/stdyDscr/stdyInfo/sumDscr/geogCover

    • /codeBook/stdyDscr/stdyInfo/sumDscr/dataKind

  • Make sure ddi_c and oai_ddi25 templates won’t render duplicate ID-attributes per document.

  • Add metadataPrefix attribute to OAI-Header’s request element. (Fixes #15)

  • Add set, from and until attributes to OAI-Header’s request element.

  • Clear OAI-Header’s request elements attributes if response results in badVerb or badArgument.

  • Relax regex validating OAI-Identifier. (Fixes #16)

  • Add set info to resumptionToken and use it in subsequent queries to Document Store. (Fixes #17)

  • Fix unhandled exception when requesting a set with more than one colon.

  • Update python package requirements: * genshi 0.7.3 * kuha_common 0.10.0

0.6.0 (2019-03-14)

  • Support for kuha_common 0.9.0

  • Update copyright headers to 2019.

0.5.0 (2018-12-18)

  • Require kuha_common 0.8.0. for better support for testing tornado handlers.

  • Decouple setup of template_folder from importing handlers.

0.4.0 (2018-09-13)

  • DDI-C template: Handle the possibility of None values in variable.codelist_codes[n].code. (Fixes #14)

  • Support using base_url for OAI-PMH request element. (Implements #6)

    • Add configuration option --oai-pmh-respond-with-requested-url which defaults to False. Using this computes the base url for OAI-PMH request element from the HTTP request.

  • Require kuha_common>=0.6.0.

0.3.0 (2018-07-12)

  • Pin requirement to kuha_common version in requirements.txt. This way it is easier to use older releases.

  • Fix possible TypeError in kuha_oai_pmh_repo_handler.oai.records.is_valid_setspec(), by explicit check for None in kuha_oai_pmh_repo_handler.oai.records.get_set_specs_from_ds_record(). (Fixes #11)

  • Conceptual container elements should be presented even if no concept is found. (Implements #10)

  • Fix possible NameError in kuha_oai_pmh_repo_handler.oai.OAIRecord.add_question(). (Fixes #12)

  • Serve topcClas and keyword DDI 2.5 elements even if no @ID attribute can be found. (Implements #13)

  • Refactor end-to-end tests: Use kuha_common.testing package.

0.2.5 (2018-03-14)

  • Add DDI 2.5 metadata format for CESSDA Data Catalogue. (Resolves #8)

  • scripts/install_kuha_oai_pmh_repo_handler_virtualenv.sh: Use pip to install & upgrade.

0.2.4 (2018-03-09)

  • ListIdentifiers verb handler should not retrieve relative records. (Fixes #7)

0.2.3 (2018-03-07)

  • DDI-C metadata & template:

    • Add support for Study.document_uris

    • Add support for Study.publishers.attr_abbreviation

    • Add missing vocabURI-attribute to topClas element

    • Add missing vocabURI-attribute to keyword element

  • OAI-DC metadata & template:

    • Use Study.document_uris for identifier.

  • Don’t require metadataPrefix when request contains resumptionToken. (Fixes #5)

  • Add upgrade flag to pip install command.

0.2.2 (2018-01-31)

  • DDI-C metadata & template: add support for:

    • Study.identifiers

    • Study.distributors

    • Study.time_methods

    • Study.sampling_procedures

    • Study.collection_modes

    • Study.analysis_units

    • Study.collection_periods

    • Study.data_access_descriptions

  • Add support for Study.identifiers in OAI-DC metadata & template.

  • Fix incorrect handling of requested identifier when using OAI namespace-identifier. (Fixes #4)

  • Separate OAI-PMH error messages for no known item and invalid identifier structure

0.2.1 (2017-11-16)

  • Add ID-attribute to qstn-element in DDI-C template

0.2.0 (2017-11-10)

  • Support for variable level metadata in DDI-C

0.1.1 (2017-10-27)

  • Update documentation

0.1.0 (2017-10-25)

  • Initial release