Keywords

1 Introduction

Neural progenitor cells (NPCs) play a key role in the generation and maintenance of the nervous system. NPCs are proliferative, undergoing mitosis to generate daughter cells that are genetic copies of the parent. As this process repeats, a family tree, or clone of related cells develops. NPCs are also migratory, with cells moving as needed to form and maintain the functionally, spatially and morphologically distinct components of the nervous system.

Individual NPCs exhibit a complex cellular morphology, with rapidly forming and changing cellular processes or protrusions [1]. These NPC processes may be used to attach to different regional structures such as the apical or basal surface of the neuroepithelium during development [2], and may also be used to explore the environment. During the non-mitotic, or interkinetic portion of the cell cycle, NPC nuclei undergo distinct motion along the processes. This interkinetic nuclear migration (INM) has been observed in a wide variety of vertebrate cell types [3], and has been associated with the cell fate, or type of daughter cells that will be produced post-mitosis [4]. The relationship between INM and mechanisms controlling cell cycle and mitosis is an important open question [5, 6].

Fluorescence microscopy is most commonly used to visualize INM. Because INM generally involves the spatial patterning or organization of a 3-D structure such as the cortex or retina, 3-D fluorescence microscopy, either confocal or multi-photon, is a way to observe not only the INM, but also the resulting divisions and tissue formation. Fluorescence microscopy allows for images to be captured with distinct nuclear and cytoplasmic markers, making the analysis of INM easier for both manual and computational approaches. However, fluorescence imaging of NPCs places a limit on the duration of the imaging experiment because of the increased photo-toxicity. Long-term observation of proliferating NPCs at a temporal frequency sufficient to establish accurate tracking of cells requires the use of transmitted light microscopy [7, 8].

Using phase-contrast microscopy to image live NPCs can capture multiple rounds of cell division over a period of 5–7 days or even longer without the photo-toxic effects of fluorescence. This time duration allows mouse cells to complete as many as 8 or even 10 rounds of cell divisions. In vitro phase contrast imaging is able to capture both the NPC nucleus, or cell body, as well as the processes. The quantification of INM from phase microscopy is more challenging compared to fluorescence imaging. The processes appear and disappear at a high temporal frequency, sometimes over a period of minutes making it difficult to integrate temporal information into the process segmentation. The process intensities vary only slightly from the background, and generally do not exhibit the phase contrast halo artifact that makes it possible to more reliably segment the cell nuclear region. Any clutter in the image sequence can be hard to differentiate from cell processes, with dead cells or other detritus appearing virtually identical to actual processes. Another challenge is the quantity of image data involved in a typical experiment. Stem cell populations often exhibit heterogeneous behaviors, so experiments examining behaviors for clones of cells can require the analysis of hundreds of clones, tens of thousands of cells and millions of individual cell segmentations. Adding process dynamics to such an experiment makes analysis by hand impossible - automated techniques are required.

Analyzing NPC process dynamics from time-lapse microscopy image sequences requires the cell body and processes to be segmented in each frame. The segmentation results are associated temporally by a tracking algorithm. Lineaging identifies mitotic events and establishes parent-daughter relationships. Together, the tracking and lineaging results establish the lineage tree showing the temporal and mitotic relationships among all cells in the clone. Here we present a new technique for segmenting and tracking NPC processes in time-lapse phase contrast microscopy. Our approach first uses an existing open-source software tool known as LEVER (for Lineage Editing and Validation) to segment, track and lineage the cells [7–11]. The LEVER analysis segments and tracks cells throughout the image sequence and is the basis for the subsequent segmentation and tracking of the NPC processes. The phase contrast LEVER segmentation [12] captures the cell soma, and the centroid of this is used as the nuclear location. LEVER can optionally integrate fluorescence markers for identifying the nucleus or process boundaries, but these are not required.

Once the LEVER analysis is complete, each image is thresholded to identify the foreground pixels that may belong to processes. Morphological processing on this threshold image is used to remove any background clutter in the images such as dead cells or detritus. Each of the process foreground pixels is then assigned to the LEVER segmentation result that is closest in the sense of minimizing the geodesic distance, or the distance in traversing only foreground process pixels to the LEVER nuclear segmentation results. Once the process pixels have been assigned to LEVER cell tracks, the process bounding box is computed, and the projection of the nuclear location onto the principal axis of the bounding box is taken. This provides the location of the cell in every image frame, and also the location of the nucleus relative to the processes, the key features for characterizing both cellular and nuclear motion.

Figure 1 illustrates the process segmentation and analysis steps. Starting with the LEVER segmentation and tracking results (panel A), the image is next adaptively thresholded to identify possible process pixels, shown in red (B). Process pixels are assigned to the nearest segmentation using a geodesic distance (C). Finally, the process bounding box is established, and the nuclear location is projected onto the process principal axis, shown as a white dot (D).

Fig. 1.
figure 1

Segmentation and tracking of adult neural progenitor cell (NPC) processes. Results are shown for a single image from an 1100 frame time-lapse sequence, with convex hulls showing cell segmentation, colored by tracking assignment (A). A thresholding step identifies pixels that may belong to processes (B). A geodesic distance transform on the thresholded image is used to assign each process pixel to the nearest connected cell segmentation, with unassigned pixels in gray (C). Nuclear location and migration are established by projecting the nuclear centroid onto the principal axis of the bounding box of each cells processes, shown as the white dot (D). Scale bar indicates \(50~\upmu \hbox {m}.\) (Color figure online)

Full size image

2 Related Literature

Examining Fig. 1, it can be observed that NPC process segmentation appears similar to the problem of neuron tracing, or neurite segmentation. Neuron tracing is an extremely well studied problem, including in phase contrast images [13]. A broad overview of the neuron tracing problem is given by Meijering [14]. Despite the superficial similarity between neurite segmentation and NPC process segmentation, the two applications are actually very different. Unlike neurons, NPCs are highly motile. The morphology of the cell body and of the processes changes dramatically over short periods of time. Also unlike neurons, NPCs are also proliferative, dividing to produce new cells. Finally, NPCs are more adherant compared to neurons, with cells frequently in close contact. For 2-D in vitro imaging, cells may also crawl on top of each other, causing significant overlap. Separating these touching cells is one of the principle challenges in working with NPC image sequence data.

The first automated computer algorithm for NPC segmentation, tracking and lineaging in phase contrast images was described by Al Kofahi, et al. [15]. More advanced tracking approaches for phase contrast images were descibed by Li et al. [16, 17]. For fluorescency microscopy images, Amat et al. describe an approach used with light-sheet microscopy images [18]. The CellProfiler program also has support for lineage analysis with track editing [19]. Winter et al. developed a tracking approach called Multitemporal Associate Tracking (MAT) that was designed specifically for objects that split and merge [20, 21]. The open-source LEVER program combines the MAT tracking algorithm with an online user-based validation [8]. LEVER works with 2-D+time phase contrast images, 5-D (3-D+multichannel+time) [11] and also with mixed phase and fluoresence microscopy images [9, 10]. LEVER uses an inference approach to incorporating human-provided corrections into the segmentation and tracking results. LEVER also uses contextual information about the cell populations obtained from the lineage tree to reduce the error rates of the automated analysis algorithms by more than 80 % [7]. The LEVER program is used here to generate the cell segmentation, tracking and lineaging results.

3 Results

Two different experimental questions involving adult and embryonic mouse NPCs are addressed in this paper. First, differences in process dynamics are compared between type C and type A adult NPCs, two distinct sub-populations of adult NPCs. The second dataset that was analyzed consisted of embryonic NPCs cultured at E12.5 (12.5 days post-fertilization) from either the anterior or the posterior regions of the cerebral cortex. This dataset was previously used to study differences between anterior and posterior NPCs [7], and has been further analyzed here with process segmentation and tracking. The results of the automated segmentation, tracking and lineaging algorithms have been fully validated by human observers using the LEVER program [8]. This validation corrects any errors in the tracking or lineaging results. Validation of the segmentation by the human observer establishes that every cell has a segmentation result in every image frame. Validation of individual pixels assigned to each cell segmentation is detailed in Sect. 4. All of the automated image analysis results, including lineaging, tracking, nuclear and process segmentation, can be viewed interactively together with the image data using the CloneView program [7] as described in Sect. 5.

3.1 Type A and Type C Adult NPCs

The adult NPC dataset contains movies from four different imaging experiments. The data consists of 142 NPC clones, with a total of 1,350 cells or tracks and 361,867 individual cell segmentations. A total of 48 movies were processed. At the end of each experiment, cells were stained using immunohistochemistry to identify the cell types including type C and type A NPCs.

Figure 2 shows NPC process segmentation for a clone of cells that produces both type A and type C NPCs. The LEVER segmentation results are shown with different colors to indicate track IDs (A). Process pixels (half-tone) are assigned to the closest cell segmentation (solid-tone pixels) using a geodesic distance (B). Process pixels colored in gray could not be assigned to any cell track. Bounding boxes for the process pixels are colored according to cell fate commitment, with type A committed NPCs in red and type C committed NPCs in blue (C). Cell fate is established for each cell at the end of the experiment by fixing and staining the cells. Fate commitment is then applied to the tree retrospectively. A cell is considered committed to a fate iff all cells in the subtree rooted at the given cell are of the same fate. The lineage tree is colored by the generation of fate commitment, with subsequent generations having a darker hue. The lineage tree also shows process area (green line) and soma area (yellow line), each normalized per-clone, plotted next to each cell. Showing features on the lineage tree in this manner is a new and effective way to visualize nuclear and process dynamics for an entire clone.

Figure 3 shows summary statistics for fate-committed type A and C NPCs. Type C cells have significantly larger soma area (\(p=10^{-46}\)) and larger process area (\(p=10^{-62}\)) compared to type A cells. Type C cells exhibit lower nuclear velocity measured along the normalized process principal axis (\(p=10^{-20}\)). Type C cells can also be seen to exhibit different patterns of motion compared to type A cells, with a higher entropy of nuclear migration (\(p=10^{-5}\)). This means that the type C cells are more uniformly distributed along the process principal axis as compared to the type A cells.

Fig. 2.
figure 2

Segmentation, tracking and lineage results with cell fate commitment and soma and process areas for type A and type C NPCs. Single image from 1100 frame sequence with cell segmentation and tracking results obtained from the LEVER program (A). Process segmentation and tracking result, with cell pixels shown in solid-tone and process pixels in half-tone (B). Bounding boxes are colored by cell fate commitment, with NPCs producing only type A NPCs in red and NPCs that produce only type C NPCs in blue (C). Lineage tree colored by fate commitment, with color hue darkening for each mitotic generation and soma and process areas ploted next to each cell cycle line. Process area for each cell is plotted in green normalized to the maximum process area per clone. Soma area for each cell is shown in yellow. Area values are normalized to the largest value on the clone. Scale bar indicates \(25~\upmu \hbox {m}.\) (Color figure online)

Full size image
Fig. 3.
figure 3

Summary statistics for type A and type C adult neural progenitor cells. Box plots show median value as red line, blue box extents indicate the 25th−75th percentile of the data, data beyond whiskers are outliers. Notches indicate 95 % confidence interval for the median. Type C cells have larger soma and processes, lower nuclear migration velocity and have increased entropy of nuclear location compared to type A cells. (Color figure online)

Full size image

3.2 Anterior and Posterior Cerebral Cortex Embryonic NPCs

The embryonic NPC dataset consists of 129 movies imaged in 3 different experiments. A total of 160 clones were analyzed, with 10,644 cells and 1,585,104 segmentations. The same analysis approach as used for adult NPCs was applied to the embyronic NPC data. Embryonic NPC clones are generally larger compared to adult clones. Figure 4 shows an example of the process segmentation and tracking results for an embryonic NPC clone. This clone contains 331 cells or tracks, and 30,229 individual cell segmentations. Starting with an unprocessed image frame (A), the LEVER segmentation and tracking results (B) are shown as the convex hulls of the cell pixels colored by track ID. The process pixels are shown segmented and tracked, colored by track assignment and with process pixels blended and cell pixels colored at full saturation (C). The process bounding box is shown in (D), with the projection of the nuclear centroid onto the process principal axis shown as a white dot. The feature used for analyzing nuclear location and velocity is the position of the nuclear centroid along this principal process axis, with the extents of the process axis normalized per frame to [−1,1].

Fig. 4.
figure 4

Cell and process segmentation and tracking with lineage tree for embryonic neural progenitor cell clone. One image from an 1100 frame sequence (A), with LEVER segmentation and tracking results shown as convex hulls colored by track ID (B). After thresholding, processes are assigned to a cell track as indicated by the color (C), with solid tone for the cell segmentation, half tone for the process pixels and unassigned process pixels shown in gray. The centroid of the cell nucleus is projected onto the principal axis of the bounding box enclosing each cell and its processes, shown as a white dot (D). A new rendering combines conventional lineage information with the nuclear location feature (bottom), with the location of the nucleus for each cell along the normalized principal axis of the process bounding box plotted alongside the cell cycle indicator on the tree. Scale bar indicates \(50~\upmu \hbox {m}.\) (Color figure online)

Full size image

The lineage tree from Fig. 4 also renders the time sequence of normalized nuclear location for each cell. For cell tracks that occur later in the movie, that lineage tree becomes very difficult to read as the spacing between cells on the lineage becomes quite tight. Figure 5 shows an alternative rendering for the lineage tree based on a radial tranform of cell relationships. The radius in this case corresponds to time, and angular orientation is used to position daughter cells below their parent. This alternative rendering for the lineage tree makes better use of the available space, making it easier to visualize the process features throughout the cell cycle.

Fig. 5.
figure 5

Radial rendering of the lineage tree with process dynamics. A new visualization for the lineage from Fig. 4 is shown, with time represented by the radius and cells distributed rotationally. Nuclear location along the process principal axis, normalized to [−1,1] is plotted alongside the radial line representing each cell lifespan. This visualization allows features associated with the lineage tree to be more clearly rendered at the distal cells in larger trees.

Full size image

A statistical summary of cell and process size, as well as nuclear migration features is shown in Fig. 6. Posterior cells have larger soma (\(p=10^{-43}\)) and process (\(p=10^{-10}\)) areas as compared to anterior cells. Posterior cells also exhibit signficantly different patterns of motion, with lower nuclear velocity (\(p=10^{-10}\)) and nuclear migration entropy (\(p=10^{-19}\)).

Fig. 6.
figure 6

Summary statistics for anterior and posterior cerebral cortex neural progenitor cells. Box plots show median value as red line, blue box extents indicate the 25th–75th percentile of the data, data beyond whiskers are outliers. Notches indicate 95 % confidence interval for the median. Anterior cells have smaller soma and process areas compared to posterior cells. Anterior cells also exhibit significantly different patterns in the motion of the nucleus along the processes, with increases to both nuclear velocity and to the entropy of the nuclear location.

Full size image

One of the advantages of an analysis that follows the cells across multiple rounds of cell division is the ability to analyze cells based on what mitotic generation they were produced in. Figure 7 shows the nuclear and process area, as well as the nuclear velocity and the entropy of nuclear location all separated by generation. Generation 0 refers to the initially plated NPC, generation 1 are the cells created in the first division, etc. Note that anterior and posterior cells exhibit similar patterns of soma and process size, and nuclear migration dynamics (entropy and velocity) across generations. Interestingly, anterior NPCs show a later peak in process and soma area compared to posterior cells. This increase is driven by a small number of anterior clones that are larger, both in the cell and process size, compared even to the generally larger posterior cells.

Fig. 7.
figure 7

Summary statistics for anterior and posterior cerebral cortex neural progenitor cells by generation. Cells are initially plated at generation zero, with each mitotic event producing a subsequent generation. Box plots show median value as red line, blue box extents indicate the 25th–75th percentile of the data, data beyond whiskers are outliers. Notches indicate 95 % confidence interval for the median. Anterior cells and clones are generally smaller compared to posterior cells and clones. A small subset of anterior clones contributes to a late and significant increase in cell and process size. (Color figure online)

Full size image

4 Methods

The cell culture and image acquisition uses the same approaches as described previously [7, 8, 22]. Following image acquistion, the images are segmented, tracked and lineaged using the LEVER program. LEVER uses separate segmentation algorithms for the adult and embryonic cells. LEVER requires two parameters for each – maximum cell size and maximum velocity. The same parameters were applied to all of the embryonic data and all the adult data.

Validation identifies and corrects errors in the automated algorithms [23]. The LEVER program is designed to make any errors in the segmentation, tracking and lineaging easy to identify and quick to correct [8]. Validation using the LEVER program verifies that tracking assignments and parent-daughter relationships are correct and that every cell has a segmentation in every image frame. Validation of the segmentation algorithms at the individual pixels was presented previously using the tracking algorithm and also using externally measured size differences between different cell populations based on flow cytometry [7].

Process segmentation and tracking consists of denoising, followed by thresholding and then assignment. Starting with a two dimensional image I, we invert the image and compute the morphological opening \(I_{\text {open}}=\max (\min (I))\) and then construct the denoised image from the white top-hat transform of I,

$$\begin{aligned} I_{\text {wth}}=I-I_{\text {open}}. \end{aligned}$$
(1)

A disk shaped kernel with a radius of seven pixels is used for the opening. This removes variations in the phase contrast background across I. This \(I_{\text {wth}}\) has negative values at the phase contrast halo pixels – the bright artifacts surrounding thick regions associated with the cell body, and postive values interior to the cells and processes. Negative values are removed,

$$\begin{aligned} I_{\text {denoise}}=\max (I_{\text {wth}},0), \end{aligned}$$
(2)

and the intensities are normalized to [0,1] forming the denoised image.

A dual-thresholding approach reduces the amount of background debris detected as process pixels (false positives). This uses the Otsu transform, \(T_{\text {Otsu}}\), weighted by \([\alpha _L,\alpha _H]=[0.3, 0.6]\),

$$\begin{aligned} BW_1=I_{\text {wth}}>\alpha _H*T_{\text {Otsu}}. \end{aligned}$$
(3)

Any connected components with area smaller than ten pixels are removed from this image, and then a morphological reconstruction uses this first thresholded image to identify high-intensity regions in \(I_{\text {denoise}}\),

$$\begin{aligned} I_{\text {final}}=\text {reconstruct}(BW_1,I_{\text {denoise}}). \end{aligned}$$
(4)

Finally, the second threshold is applied to form the final thresholded image,

$$\begin{aligned} BW_{\text {final}}=I_{\text {final}}>\alpha _L*T_{\text {Otsu}}. \end{aligned}$$
(5)

Following thresholding, each pixel is assigned the tracking ID as the nearest nuclear segmentation using a geodesic distance across \(BW_{\text {final}}\).

Once all foreground pixels from \(BW_{\text {final}}\) have been assigned a tracking ID from the nearest segmentation result, the process bounding box is computed from the process pixel locations for track ID i, \((x,y)_i\). The process pixels are centered, subtracting the mean value of the process pixel location,

$$\begin{aligned} (x,y)=(x,y)_i-\text {mean}(x,y)_i. \end{aligned}$$
(6)

The bounding box is computed using the singular value decomposition (SVD) of the spatial locations of the process pixels,

$$\begin{aligned} \text {SVD}(x,y)=USV, \end{aligned}$$
(7)

where the vectors V form an orthonormal basis that gives the principal axis directions for the process pixels [24]. The process pixels are projected into the V coordinate system, \((x',y')=(x,y)*V\). The bounding box in the orthogonal space is computed \([x'_{\min },x'_{\max },y'_{\min },y'_{\max }]\) and projected to the original space,

$$\begin{aligned} {[}x_{\min },y_{\min };x_{\max },y_{\max }]=[x'_{\min },y'_{\min };x'_{\max },y'_{\max }]*V^{-1}+\text {mean}(x,y)_i. \end{aligned}$$
(8)

The projection of the nuclear centroid on the first coordinate in the V space, or the principal axis, is used as the nuclear location feature for velocity and entropy calculations. For entropy calculations, the principal axis is divided into ten bins.

Finally, all statistical tests were done with the non-parametric Wilcoxon Rank-Sum test for difference of medians [25].

5 Open Source Software and Data Availability

The LEVER program is available under the GNU Public License (GPL v2 or later). Details on obtaining source code or compiled executables are available from http://n2t.net/ark:/87918/d9rp4t. All of the image analysis results described here, together with the image data, are available interactively on the CloneView website: http://n2t.net/ark:/87918/d9wc73.

6 Conclusions

This paper describes new a new segmentation and tracking approach for NPC processes. One key challenge is to reject debris including dead cells while still capturing fine processes in cluttered images. New visualization techniques for the lineage tree enhance the ability to show nuclear and process dynamics for entire clones. Analyzing hundreds of clones found previously unknown significant differences in nuclear and process dynamics among functionally different populations of NPCs.