Example input raw images (top-left), ground-truth instance masks (top-right) and predictions by the network (bottom-right). The Bottom-Left Image shows randomly-selected interior pixels (+), their learnt embeddings (.) and the learnt margin visualized as an axis-aligned ellipse centred on the ground-truth-center (x)).

Automatic detection and segmentation of objects in 2D and 3D microscopy data is important for countless biomedical applications. In the natural image domain, spatial embedding-based instance segmentation methods are known to yield high-quality results, but their utility for segmenting microscopy data is currently little researched. Here we introduce EmbedSeg, an embedding-based instance segmentation method which outperforms existing state-of-the-art baselines on 2D as well as 3D microscopy datasets. Additionally, we show that EmbedSeg has a GPU memory footprint small enough to train even on laptop GPUs, making it accessible to virtually everyone. Finally, we introduce four new 3D microscopy datasets, which we make publicly available alongside ground truth training labels.

Qualitative results of EmbedSeg and two baselines on one representative image of the BBBC010 and Usiigaci dataset. Columns show one input image, zoomed insets, ground truth labels (GT), and instance segmentation results by the 3-class U-Net baseline, the best performing competing baseline, and our results using EmbedSeg . Note that each segmented instance is shown in a random but unique color.

Quantitative Evaluation on three datasets. For each dataset, we compare results of multiple baselines (rows) to results obtained with our proposed pipeline (EmbedSeg) highlighted in gray. First results column shows the GPU-memory (training) footprint of the respective method, remaining columns the Mean Average Precision for selected IoU thresholds. Best and second best performing methods per column are indicated in bold and underlined, respectively. Note that the Cellpose results are either trained by us on exclusively the given dataset, or obtained by using the publicly available model. EmbedSeg is either the best performing method or is a very close runner-up, despite being by far the most GPU-memory efficient method.