A B C

(A) 2-D schematic illustrating the idea: two distinct specimens (left: source and middle: target) are compared in order to estimate pair-wise cell nuclei correspondences and an optimal transform that registers the source onto the target (right) (B, C) 2-D schematic illustrating the two use cases: (left, B) images of distinct, independent in-situ specimens, acquired through confocal microscopy are registered to each other, which enables formation of an average, virtual atlas. (right, C) images of in-situ specimens, acquired through confocal microscopy are registered to the appropriate frame (tp: time point) in a time-lapse movie acquired through SPIM imaging. Nuclei indicated in darker shades are the ones expressing the transcription factor being investigated. These transcription factor intensities are transferred from the source nucleus to the corresponding target nucleus in both use cases.

Early development of an animal from an egg involves a rapid increase in cell number and several cell fate specification events which are accompanied by dynamic morphogenetic changes. In order to correlate the morphological changes with the underlying genetic events, one typically needs to monitor the living system with several imaging modalities offering different spatial and temporal resolution. Live imaging allows monitoring the embryo at a high temporal resolution and observing the morphological changes during the early development. Confocal images of specimens fixed and stained for the expression of certain genes provide high spatially-resolved static snapshots and enable observing the transcription states of an embryo at specific time points during development. The two modalities cannot, by definition, be applied to the same specimen and thus, separately obtained images of different specimens need to be registered. Biologically, the most meaningful way to register the images is by identifying cellular correspondences between these two imaging modalities. In this way, one can bring the two sources of information into a single domain and combine dynamic information on morphogenesis with static gene expression data. The problem of establishing cellular correspondence is non-trivial due to the stochasticity of developmental processes and the non-linear deformation of the specimen during staining protocols. Here, we propose a new computational pipeline for identifying cell-to-cell correspondences between images from multiple modalities and for using these correspondences to register 3D images within and across imaging modalities. We demonstrate this pipeline by combining four dimensional time-lapse showing embryogenesis of Spiralian ragworm Platynereis dumerilii with three dimensional scans of fixed Platynereis dumerilii embryos stained for the expression of a variety of important developmental transcription factors. We compare our approach with methods for aligning point clouds and show that we match the accuracy of these state-of-the-art registration pipelines on synthetic data. We show that our approach outperforms these methods on real biological imaging datasets. In addition, our approach uniquely provides, in addition to the registration, also the non-redundant matching of corresponding, biologically meaningful entities within the registered specimen which is the prerequisite for generating biological insights from the combined datasets. The complete pipeline is available for public use through a Fiji plugin.

Intramodal Registration	Intermodal Registration
A	B
C	D
A: DAPI channels indicating cell nuclei for two distinct in-situ specimens (source: green and magenta: target image). B: DAPI channels indicating cell nuclei for an in-situ specimen (source: green) being registered to the corresponding frame containing equivalent number of cell nuclei, in the time-lapse movie (target: magenta). Landmarks for source image are indicated as yellow spheres and labeled from 1 . . . 12. Similarly, landmarks for the target image are labeled from 1’ . . . 12’. C, D: After performing registration with our proposed pipeline, corresponding landmarks from the source and target images appear to be much closer in Euclidean distance.

Lalit, Handberg-Thorsager, Hsieh, Jug, Tomancak. Registration of multi-modal volumetric images of embryos by establishing correspondences between cells. In BioImage Computing @ ECCV 2020.