Refactor, modernize and extend the Nipy neuroimaging library

[matthew-brett]

Project

Nibabel

Summary

We will merge the classic neuroimaging package Nipy with the new Xibabel package, into a new library, Nipy2, that can also benefit from advances in general tools such as ITK, scikit-image and DIPY; Nipy has valuable code and wide coverage of neuroimaging, whereas Xibabel is a modern brain imaging IO library taking advantage of Xarray and Dask for transparent use of multiprocessing and optimum memory layout. Bringing modern development to Nipy will allow us draw a much larger community of neuroscientists into open-source development in Python, and has the potential to transform a field that has been held back by a diverse and incompatible range of lab-based software, much of which has been hard for community developers to work on.

submitter

Matthew Brett

project lead

@matthew-brett

Community benefit

We want to bring a much larger community of scientists and developers to the open-source community, by using an existing Python library as the basis for new and fruitful development in neuroimaging analysis, based on modern Python projects and tooling.

Neuroimaging software is largely in stasis, with modest use of Python libraries, and little sign of change. Standard analysis uses aging, idiosyncratic packages in Matlab or C/C++. Each standard tool is a world in itself; each contains tools to perform basic processing; their own command line and GUI interface; their own way of describing and storing data, and automating analysis. Each lab tends to use one package, as switching cost is high. Current packages are difficult to work with for development; it is hard for experts in spatial processing, signal processing or statistics to contribute, slowing adoption of new methods. There is little culture of open-source contribution, so it is hard to attract good developer-users. The difficulty of using these tools restricts research to labs with resources to support the software packages, for example, on dedicated clusters. We have much experience advising researchers, and can attest to the resulting effects in terms of error, inefficiency and lack of reproducibility.

Neuroimaging now looks like astronomy before Astropy, with many fragmented lines of development and tools, or statistics before R, where departments were divided by their use of heavy and incompatible software packages. We need a Python package that can be the basis for development, research and teaching; it must allow the full range of analysis methods for functional brain imaging, with a high-level scripting interface, and with competitive performance on modern hardware. It must be well written and well documented so researchers can use and extend it to their needs.

We are in an excellent position to solve this problem, and bring these scientists and developers to Python, by combining work that already exists. A prior CZI grant enabled us to develop a prototype base library, Xibabel, that makes use of modern libraries for labeled and distributed arrays, as well as memory and IO-efficient storage formats. However, as with any new image library, we need users to stretch Xibabel and test it in real analysis.

Meanwhile, we are the lead maintainers of the Nipy Python package, that we started in 2006. Nipy has code for a substantial part of a full neuroimaging analysis pipeline, including several important innovations, such as: a reference library for spatial statistical correction written by Jonathan Taylor — a world expert in the field — a sophisticated symbolic language for statistical models in brain imaging (Taylor); integrated correction for head motion occurring during image acquisition; and diagnostic utilities for motion and acquisition artefacts. It is installable on current Python, and has packages for Fedora and Debian.

The grant work will bring users and developers in neuroimaging to Xibabel and Nipy, by refactoring Nipy on top of Xibabel. Nipy2 will provide a full range of standard processing with Xibabel as its standard image object. In turn, Nipy2 will form a foundation from which to build, that is based, via Xibabel, on modern large-data array interfaces such as Xarray. By integrating large well-maintained libraries for spatial processing, such as ITK, we give access to tools to assemble a wide-range of spatial processing pipelines, making it easier to do custom analyis or use new algorithms.

For the field, Nipy2 will form the heart of a Python imaging library to rival or surpass the current standard software. Neuroimaging has been held back by the ossified state of its tools; their inflexibility has largely frozen teaching and practice in the field — a state we believe we can change. Nipy2 can be a new home for users and developers in neuroimaging. By being accessible, we will change the culture of neuroimaging from users-as-consumers to users-as-contributors.

There are other Python packages that specialize in particular areas of neuroimaging, such as DIPY and Nilearn. DIPY covers diffusion imaging, in contrast to function brain imaging, that we consider here; it does have important and useful registration tools that we will wrap along with similar tools in ITK. Nilearn is a very well-written and well-maintained library for statistics and machine learning of neuroimaging data. This has some overlap with Nipy, but our main work here is on spatial processing, not covered by Nilearn. We aim to provide the full processing pipeline from raw images to statistical results; for example, users might use Nipy2 to generate images as input to Nilearn.

This work will bring Python neuroimaging scientists and developers back to the mainstream of imaging development; by providing a flexible and general package for a full neuroimaging analysis, we can attract many more users and developers, forming a virtuous cycle of use, use-cases, development and extension.

Amount requested

10000

Execution plan

Personnel

The work will be done by Matthew Brett (MB) and Chris Markiewicz (CM). Funding is for development hours divided between MB and CM, with approximate expected proportion 80:20, because MB has more time to devote to the work. MB will depend on CM for review and discussion for all submitted code.

MB has been researching, teaching and writing open-source code for brain imaging for more than 25 years; he is the co-founder of the Nipy project, main maintainer of the Nipy library, founder of the Nibabel project, and contributor to various scientific Python libraries, including Numpy and Scipy.

CM has been lead maintainer for Nibabel since 2018, is lead maintainer for the Nipype imaging processing package, and has been a core developer of the Nipy, fMRIPrep and FitLins packages.

Move and refactor existing code to Nipy2

We should preserve the reference library for spatial image correction, the unique routines for joint head-motion and image timing correction, and the other diagnostic utilities in Nipy. Move these to a new Nipy2 package, and refactor for maintainability (20 hours). To be finished by 1.5 months into project.

Refactor Nipy2 to use Xibabel image objects.

Integrate Xibabel image object into Nipy2: We have recently finished a CZI grant to the Nibabel project that included work on the next-generation brain image library, Xibabel, building on the standard imaging data formats, as well as Dask, Xarray and Zarr to allow flexible IO, parallel processing and labeled axes using modern data formats including HDF5, Zarr, URLs and cloud data buckets. We will build Nipy2 on this library to make more readable and performant APIs (30 hours). Expected finish at 6 months into project.

Integrate existing spatial processing libraries

Improvements in Python packaging mean we can depend on big libraries such as the Imaging Toolkit (ITK) and scikit-image for a wide range of image processing, and on more specific libraries such as DIPY, which has sophisticated algorithms for non-linear image registration. We will implement a general API that chooses one best-in-class default implementation of each of the available spatial processing methods in these libraries, but allows choice and extension by using other algorithms and libraries (50 hours). To be done by 12 months.