The Journal of Open Source Software (JOSS) is a developer friendly, open access journal for research software packages published as open source software. JOSS started publishing in 2016, and has published more than 1,500 articles so far. After five years and that many articles, it is time to have a closer look at how JOSS content is reused. Given the context of publishing open access articles about open source software, it is only natural to look at reuse using open (scholarly) metrics, taking advantage of the fact that JOSS is registering all its articles as Crossref DOIs, and Crossref keeps track of citation counts and makes them openly available in the Crossref REST API (via the is-referenced-by-count property). It is straightforward to collect this information from Crossref using a Jupyter notebook and visualize it:
The most obvious finding in this chart is the small number of JOSS papers with very high citation counts, including 2,094 citations for the most-cited paper published in November 2019 (Wickham et al. 2019, Welcome to the Tidyverse), and currently 19 papers with at least 19 citations – see all JOSS papers with DOI, title, publication date and citation count here.
While we typically are most interested in citations, there are other dimensions of reuse, summarized for example in this table:
Tracking views and downloads of scholarly articles is a significant effort as it requires a standardized approach, but some of the other dimensions are made easily available via the Crossref Event Data service and can be collected via a Jupyter notebook.
Here Crossref is the combined number of citations collected via the REST API, DataCite is the number of citations of JOSS articles by DataCite DOIs, typically for the software described in the JOSS article archived via GitHub in the Zenodo software archive, Twitter is the number of tweets referencing the JOSS article, Wikipedia is the number of Wikipedia edits referencing the JOSS article, Newsfeed is the number of blog posts or news articles mentioning a JOSS article, and Hypothesis is the number of publicly available Hypothesis.is annotations mentioning a JOSS article.
Using the Jupyter notebook there are several paths one could follow to go into more detail. Or one can expand on the work done for JOSS by Charlotte Soneson (JOSS Submission Analysis) addressing very similar questions and in much more detail (sans the social media usage), for example the software licenses used for the code repositories described in the JOSS papers:
Note: updated October 26, 2021 with a link to the JOSS analysis done by Charlotte Soneson.
Don't Reinvent the Wheel
In a post last week I talked about roads and stagecoaches, and how work on scholarly infrastructure can often be more important than building customer-facing apps. One important aspect of that infrastructure work is to not duplicate efforts.Image by ...
What is Scholarly Markdown?
One of the important discussions taking place at the Markdown for Science workshop last weekend was about the definition of Scholarly Markdown. We came up with this:Markdown that supports the requirements of scientific textsMarkdown as format that glues ...
Visualizing Scholarly Content
One topic I will cover this Sunday in a presentation on Open Scholarship Tools at Wikimania 2014 together with Ian Mulvany is visualization.Data visualization is all about telling stories with data, something that is of course not only important for scholarly content, ...