Modernisation of documentation across the PySAL ecosystem

[martinfleis]

Project

pysal

Summary

PySAL federation has partially inconsistent, outdated, and incomplete documentation among individual packages. This project proposes its modernisation, reorganisation, and overall significant improvement.

submitter

Martin Fleischmann

project lead

@sjsrey

Community benefit

The PySAL project (Python Spatial Analysis Library) is organised as a federation of individual packages (19 at the time of writing, with 20th under development), which are pulled into the pysal meta-package but are typically used on their own or in conjunction with a selected subset of other packages. Each of these components has its own maintainers and a team of developers. Typically, there is a large overlap between packages, but due to the number of them, certain diversity of approaches emerges, despite the efforts to standardise the ecosystem. This has materialised in large inconsistencies in how the documentation of individual packages looks, what it contains, and what the expected standard is. For example, different packages occasionally use different Sphinx themes. While most of them are using a standardised one, it is an outdated bootstrap theme that does not allow easy integration of user guides, examples, and API references in a user-friendly manner. Due to the limited maintenance capacity focusing on documentation, certain parts are outdated and not reproducible in the latest environments, and some of the released functionality has never been properly exposed in the API reference.

The proposed project should focus on significant improvements of the state of the PySAL documentation across three main domains. First, it will modernise the infrastructure to adopt unified, well-maintained, contemporary Sphinx themes and extensions, allowing further expansion. Second, it will focus on completeness and organisation of API references across all packages, ensuring the visibility of features, consistency, and factual correctness of the information. Third, it will focus on updating the notebooks within user guides, ensuring they are fully documented, up to date, and automatically tested (and executed where possible).

The result will provide an enhanced user experience when interacting with the documentation of the PySAL ecosystem. There will be higher consistency, allowing easier orientation when switching between individual packages (a typical user will use more at the same time). Within individual documentations, there will be a higher discoverability of features offered, both from the standpoint of the API reference and user guides. From the long-term perspective, a higher degree of consistency and dependency on modern tooling lowers the maintenance cost for the whole development team and minimises barriers for new contributors interested in working with multiple packages.

Amount requested

9100

Execution plan

The project will be primarily implemented by Martin Fleischmann (@martinfleis), one of the PySAL core developers, in collaboration with the rest of the PySAL team and a wider community. The hourly rate used in the development is $70/hr.

The project implementation is divided into three consecutive steps, each applied to all maintained packages within the PySAL ecosystem:

1. Infrastructure update

The first pass across the ecosystem will ensure that all packages migrated from outdated Sphinx themes to a single modern theme supported by a consolidated set of Sphinx extensions and other infrastructure. This is to avoid situations where the same approach behaves differently in different packages. A typical example is a selection of an engine used to render Jupyter Notebooks (nbshinx vs myst). The ecosystem will migrate from RST to MyST Markdown and MyST rendering engines.

Budget allocation: 30 hours / $2,100
Timeline: Month 1-3

2. API reference completeness

Second step will focus on API reference completeness. It will verify whether the entire public API is properly exposed in the reference, including all class methods, some of which tend to be currently just listed without explicit docstrings. This requires sweeps through the codebase, expansion and reformatting of docstrings where needed (some may not follow numpydoc), and possible restructuring of the individual API reference pages to properly accommodate all methods.

Budget allocation: 40 hours / $2,800
Timeline: Month 4-7

3. User guide consolidation

The last step will focus on individual user guides, which are currently often treated as a barely documented collection of notebooks illustrating a subset of functionality. The notebooks are usually not executed during the documentation rendering and are not tested on CI. This leads to a situation where many are outdated and no longer reproducible. The project will ensure that all of the existing notebooks are properly exposed in the documentation, are properly described, are up-to-date, and are fully reproducible and either executed during rendering or tested on CI. The optimal target is the treatment of user guides as implemented in the pysal/momepy package.

Budget allocation: 60 hours / $4,200
Timeline: Month 8-12