Commonmeta is learning new tricks

Last week I reported that the commonmeta Go library can now directly register metadata with Crossref and InvenioRDM repositories. With version v0.6.10 released today, I further improved this functionality, making commonmeta an excellent tool for adding existing scholarly metadata to any InvenioRDM instance for testing or showcasing the software.

Installation

To install commonmeta on Linux, Mac, or Windows, run the following command: go install github.com/front-matter/commonmeta@latest. This installs a single binary that runs on X86 and ARM processors without external dependencies. To see the command-line options, type commonmeta --help.

Collect metadata from Crossref or DataCite

Using their sample API endpoints (Crossref, DataCite), fetch a sample of random DOI metadata records, further specified with one or more filters. For example:

Fifty preprints – what Crossref calls posted content – published by Crossref member Cold Spring Harbor Laboratory in 2022:

commonmeta list -n 50 --member 246 --type posted-content --year 2022 -f crossref

One hundred datasets published by DataCite repository Dryad in 203 that include author affiliations with Research Organization Registry (ROR) IDs:

commonmeta list -n 100 --client dryad.dryad --type dataset --year 2023 --has-ror-id -f datacite

Thirty software records published by DataCite repository Zenodo that include funding information:

commonmeta list -n 30 --client cern.zenodo --type software --has-award -f datacite

Collect metadata from other InvenioRDM instances or other sources

Another good source of metadata records are other InvenioRDM instances, e.g. Zenodo and of course Rogue Scholar. To fetch lists of metadata (rather than a single metadata record, which is also possible), specify the InvenioRDM instance via from-host and the community name:

commonmeta list --from-host zenodo.org --community grei -f inveniordm

Finally, commonmeta can import JSON files that represent lists of metadata records, e.g. from a Crossref or DataCite annual data dump, or from metadata exported in Citation-Style Language (CSL) JSON format from the reference manager Zotero – where file export_csl.json is in the same folder where you run the commonmeta command:

commonmeta list export_csl.json -f csl

Export to InvenioRDM

The examples above all generate JSON in commonmeta format. You can convert the metadata to other formats by adding the --format or -f flag to the commonmeta command, for example

commonmeta list --from-host zenodo.org --community grei -f inveniordm -t datacite

Supported export formats are listed here and include DataCite, Schema.org, Citation-Style Language, and also Crossref XML, and InvenioRDM. You can generate an XML or JSON file in the correct metadata format for the latter two formats and write directly to the respective API if you have the correct permissions. I am doing this to register/update Crossref DOI metadata for the Rogue Scholar science blog archive, and since last week you can also do this to register the metadata with an InvenioRDM instance where you are allowed to register metadata, for example a development instance running on your local computer at localhost. Replace list with push in the above commands, change the target format to inveniordm and add the hostname and token:

commonmeta push --from-host zenodo.org --community grei -f inveniordm -t inveniordm --host localhost --token xxx

This example will copy the 31 metadata records from the Zenodo GREI community to your local InvenioRDM instance – or rather convert, as the Zenodo REST API is heavily customized for legacy reasons. Commonmeta is fast, so for larger numbers of metadata records you may run into API rate-limiting errors, and some metadata might (not yet) map directly.

More work is needed to build workflows that regularly import metadata into an InvenioRDM instance, e.g. content published with Crossref or DataCite DOIs from a particular institution to an institutional repository. Commonmeta currently doesn't handle InvenioRDM file attachments, something that is important for this particular use case. Using a Go library instead of Python or Javascript is something new for InvenioRDM developers, so I need to figure out if that is fine and similar to using other Open Source infrastructure tools written in Go such as Docker and Kubernetes.

Commonmeta is a great new option to add metadata records to an InvenioRDM instance under development. Please reach out with questions or issues. To follow the development of commonmeta and Rogue Scholar, subscribe to this blog via RSS feed or newsletter.

References

Fenner, M. (2024, November 19). Going beyond GitHub Actions for Rogue Scholar. Front Matter. https://doi.org/10.53731/8gmaq-t7s42

Fenner, M. (2024). front-matter/commonmeta: V0.6.10 (Version v0.6.10) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.14217168