When a microscope produces more data than you can read

A modern fluorescence microscope can capture tens of thousands of images in a single experiment. High-content screening campaigns generate millions. Going through such a volume of data manually is practically impossible — and that's exactly where artificial intelligence comes in.

As summarized in a recent review article published on Technology Networks, deep learning has in recent years replaced manual and threshold-based methods as the default approach for cell segmentation, object detection, time-lapse tracking, and image restoration. This doesn't mean biological judgment has lost its relevance — AI identifies structures and patterns, but what those patterns mean in the context of a specific experiment must still be decided by a human.

Cell segmentation: The cornerstone everything else builds on

Cell segmentation — precisely delineating the boundaries of individual cells, their nuclei, or membranes — was historically the most labor-intensive step of quantitative microscopy. Its quality determines the ceiling for all downstream analysis.

Cellpose, developed at Janelia Research Campus (USA), showed that a single deep learning model trained on over 70,000 segmented objects can accurately segment cells across a wide range of imaging modalities without the need for retraining. This generalization ability — not performance on a single benchmark — sets Cellpose apart from older specialized tools and accounts for its massive adoption. Published in Nature Methods (2021). The latest version, Cellpose-SAM, combines the original architecture with the capabilities of segmentation foundation models.

StarDist approaches the same problem differently — it represents cell nuclei as star-convex polygons instead of pixel masks. This approach excels especially with densely packed nuclei, where methods based on rectangular bounding boxes or pixel clustering often merge neighboring objects. The extension to 3D star-convex polyhedra (published at WACV 2020) enabled application to volumetric microscopy data. StarDist has 1,200 GitHub stars and an active community.

Both tools illustrate a broader trend: architectural decisions matter less than the diversity and scale of training data. The direction from specialist, single-modality tools toward broadly applicable deep learning models will continue to define segmentation research.

Tracking cells over time: When a static image isn't enough

Segmentation at a single time point is just the beginning. The real breakthrough comes when we can track individual cells, organelles, or particles across time-lapse sequences and reconstruct their trajectories, lineages, and dynamic behavior.

TrackMate, an open-source plugin for the Fiji platform developed at the Institut Pasteur in Paris, integrated Cellpose, StarDist, and ilastik algorithms directly into its tracking pipeline in version 7 (published in Nature Methods, 2022). This means researchers can now track even irregularly shaped or densely arranged objects that previous methods could not capture. The practical value lies especially in developmental biology and cell migration studies — including cancer metastasis research, where scientists need to quantify the migration dynamics of hundreds of cells simultaneously.

Seeing more with less illumination: AI image restoration

Fluorescence microscopy has always required a trade-off between acquisition speed, spatial resolution, and the light dose a sample can tolerate without damage (phototoxicity) or fading (photobleaching). Deep learning hasn't eliminated this trade-off, but it has pushed the boundary.

Content-aware image restoration techniques train neural networks on pairs of low- and high-quality images. The network learns to recover signal that would otherwise require more light or slower acquisition. The DeepCAD-RT method, published in Nature Biotechnology (2022), goes even further — it uses self-supervised denoising and enables imaging with 10× fewer photons while achieving 20× faster processing and 27× lower memory requirements. All in real time directly on a two-photon microscope.

For scientists working with live cells, this means the ability to observe sensitive organisms for longer periods without the microscope light killing them — a crucial advantage for embryonic development research or neuroscience.

High-content screening: Millions of cells, thousands of features

High-content screening generates some of the largest image datasets in biological research — individual campaigns produce millions of single-cell images. At this scale, AI isn't convenient, it's the only practically usable path to converting raw images into interpretable data.

The most prominent example of phenotypic profiling is the Cell Painting assay developed at the Broad Institute (MIT and Harvard). The current version of the protocol (published in Nature Protocols, 2023) uses six dyes across five channels to label eight cellular components simultaneously. From each cell, it then extracts thousands of quantitative features — from nuclear shape to cytoskeleton texture. This data-rich, unbiased approach enables comparing the effects of thousands of chemical or genetic perturbations without knowing in advance what exactly you're looking for. As noted by a review in Nature Reviews Drug Discovery (2021), machine learning is actively revitalizing this field — especially for predicting the mechanism of action of new drugs.

Five tools every bioimaging scientist should know

Four open-source tools form the backbone of most deep learning workflows in biological imaging. Each occupies a specific niche:

• Cellpose — generalist cell and nucleus segmentation across modalities. Developed at Janelia Research Campus (USA), Python/GPU, free.
• StarDist — specialized segmentation of densely packed nuclei using star-convex shapes. Supports both 2D and 3D. Free.
• ilastik — interactive pixel classification and object tracking without programming. Developed at EMBL Heidelberg (Germany). Free.
• CellProfiler — modular pipeline builder for high-throughput image analysis. Broad Institute (USA). Version 4 delivered 10× faster CPU time compared to the previous version. Free.
• TrackMate — object detection and tracking in time-lapse series. Institut Pasteur, Paris (France). Plugin for Fiji. Free.

These tools are typically used together, not as alternatives. A typical modern pipeline might use Cellpose for segmentation, feed the results into CellProfiler for feature extraction, and ilastik for tasks requiring interactive classification.

For Czech scientists, it is significant that two of these tools were developed at European institutions — ilastik at EMBL in Germany and TrackMate at the Institut Pasteur in France. The Czech bioimaging community is closely connected with them through the Czech-BioImaging infrastructure, which is part of the pan-European Euro-BioImaging ERIC network. Researchers from BIOCEV, CEITEC, or the Institute of Molecular Genetics of the Czech Academy of Sciences routinely use these tools in their imaging core facilities.

What to take away

The most important message for anyone working with biological imaging: the entry barrier for AI-assisted image analysis is lower than it has ever been. Generalist open-source tools work across modalities without retraining and often don't even require programming knowledge.

Those who benefit most from these tools are those who understand what the model was trained on, under what conditions it might fail, and how to validate the output against real data before relying on it in a publication. Automation has changed the mechanical part of image analysis — but not the interpretive one.