Department: Dept. Ophthalmology, Moran Eye Center
Project
While genomics aims to map the human genome, connectomics aims to map all connections in the brain and retina. Retinal connectomics help discover how retinas are wired via structural maps of connectivity, also revealing how retinal structure and function can be altered in specific diseases, such as age-related macular degeneration (AMD). Creating such connectome maps requires high-speed automated imaging and automated computational map-building and massive data storage. Dr. Jones' research group has built specialized infrastructure for data assembly, visualization, annotation and analysis.
Nornir is a set of python packages to facilitate assembly of large image data sets. These tools were originally designed to construct 3D connectomics volumes from terabytes of image data, including the use of slice-to-slice and slice-to-volume transforms. While current implementation leverages multi-threaded CPU processing, we aim to further speed up this process by directly leveraging GPUs, using the open-source CuPy library.
Progress
We implemented GPU processing for various transforms by leveraging the CuPy library. We implemented tests to ensure code quality, and enabled comparison between single CPU, multi-threaded and GPU processing. We achieved 3x to 10x speedups from single CPU to GPU processing, and 1.5x to 3x gains from multi-CPU to GPU processing. Additional improvements are expected with the implementation of novel features in future CuPy releases.
We also worked on CI/CD (continuous integration and continuous delivery), focusing on package configurations, library dependencies across multiple repositories, enabling automated cross-platform testing via Github actions, and assessing python packaging for deployment.
These stages will help facilitate data sharing of existing connectomes to the wider community, including research groups aiming to use AI or ML approaches for automated data annotations.
We implemented GPU processing for various transforms by leveraging the CuPy library. We implemented tests to ensure code quality, and enabled comparison between single CPU, multi-threaded and GPU processing. We achieved 3x to 10x speedups from single CPU to GPU processing, and 1.5x to 3x gains from multi-CPU to GPU processing. Additional improvements are expected with the implementation of novel features in future CuPy releases.
We also worked on CI/CD (continuous integration and continuous delivery), focusing on package configurations, library dependencies across multiple repositories, enabling automated cross-platform testing via Github actions, and assessing python packaging for deployment.
These stages will help facilitate data sharing of existing connectomes to the wider community, including research groups aiming to use AI or ML approaches for automated data annotations.
