The Rogue Scholar API now automatically indexes blog posts

The dedicated API for the Rogue Scholar science blog archive launched two weeks ago. The initial release supported fetching metadata and content from Rogue Scholar. Today this API was updated with important new functionality: parsing of science blog posts and storing the metadata and content in the Rogue Scholar. This continues the transition from a Javascript serverless API that is part of the frontend application to a dedicated Python API that can better support long-running and/or complex processes such as constantly (currently every 10 min) fetching new blog posts from 60+ participating blogs and parsing the metadata and content.

The updated API is again available as Open Source software via PyPi, GitHub, and Zenodo. Installing the API on your computer requires data from the database (Postgres) and search index (Typesense), which both also use Open Source software but need authentication for access. The API is documented via the Swagger UI, and is well-suited to be used in Jupyter notebooks that take advantage of the metadata and content hosted on Rogue Scholar, as of today 9,512 science blog posts from 65 participating blogs.

One interesting idea is to write overlay blog posts, and Heinz Pampel did exactly that today when he published a blog post commemorating Open Access Week 2023 (which started Monday with the motto Community over Commercialization), referencing several blog posts archived in Rogue Scholar.

References

Fenner, M. (2023). Rogue Scholar has an API. Front Matter. https://doi.org/10.53731/ar11b-5ea39

Martin Fenner. (2023). front-matter/rogue-scholar-api: V0.7.1 (v0.7.1) [Computer software]. Zenodo. https://doi.org/10.5281/ZENODO.10037789

Fenner, M. (2023). Generating Overlay blog posts. Front Matter. https://doi.org/10.53731/gzrse-p5d35

Pampel, H. (2023). The Open Access Week in the Scholarly Blogosphere. Syldavia Gazette. https://doi.org/10.53731/xs2mj-epe20