1 Introduction

The highly folded cerebral cortex shows considerably variable folding patterns across subjects. To study cortical structure and function, cortical surface atlases have been built [10] for providing the common spaces for quantitative comparison of subjects and populations. However, most of existing cortical surface atlases are built from adult brains, e.g., the FreeSurfer atlas [1], PALS-B12 atlas [3], and the recent HCP atlas [2]. Few works are focused on the infant cortical surface atlas [6]. In fact, at early ages, the infant cortical surface undergoes a dynamic and critical development, not only in size but also in the folding degree [4]. Hence, to fully characterize, analyze, and understand dynamic cortical developmental trajectories during early brain development, instead of building a single atlas, it would be ideal to build a set of age-specific atlases with (a) vertex-wise correspondences across ages, (b) dense sampling at key time points during the cortex development, and (c) sharp folding patterns representing a population.

Motivated by above requirements, we build the 4D infant cortical surface atlas firstly covering 6 postnatal years, based on 339 longitudinal MRI scans from 50 healthy infants, with each scanned roughly at 1, 3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months of age. To establish cortical correspondences across different subjects and different time points, a two-stage groupwise surface registration is adopted to ensure both longitudinal consistency and unbiasedness. After registration, instead of averaging the co-registered surfaces, a spherical patch-based sparse representation is developed to better capture common cortical folding patterns and also overcome potential registration errors. To further equip our atlas with parcellations, we also warp the FreeSurfer parcellation [1] and the HCP MMP parcellation [2] onto this 4D atlas to facilitate early brain development studies.

2 Method

2.1 Materials and Image Processing

Totally 339 serial MRI scans from 50 healthy infants were acquired by a Siemens 3 T scanner. Each subject was scheduled to scan at 1, 3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months of age. The subject number and gender information (with M indicating male, and F indicating female) at each time point is given in Fig. 5.

All infant MR images were preprocessed by an established infant-specific pipeline [5, 12]. Briefly, it included skull stripping, cerebellum removal, intensity inhomogeneity correction, tissue segmentation, separation of left/right hemispheres, topology correction, cortical surface reconstruction, and computation of morphological features (e.g., sulcal depth, average convexity, and curvature). All cortical surfaces were then mapped onto a sphere for facilitating the registration.

2.2 Establishing Intra-subject and Inter-subject Correspondences

To establish cortical correspondences across subjects and time points, we adopt a two-stage (intra-subject and inter-subject) groupwise surface registration to ensure both longitudinal consistency and unbiasedness, using the spherical demons method [13]. The registration framework is illustrated in Fig. 1.

Fig. 1.
figure 1

Illustration of two-stage registration for building intra-subject and inter-subject cortical correspondences. Gray boxes indicate missing data at that time.

Full size image

The first stage is to build the unbiased intra-subject longitudinal correspondences for each subject. All longitudinal cortical surfaces of the same subject are groupwisely co-registered and then the intra-subject mean can be obtained. Note, because all primary cortical folds are present at term birth and preserved during postnatal development [3, 4], the intra-subject mean cortical folding pattern is sharp and contains representative subject-specific information.

The second stage is to build inter-subject correspondence across all subjects. Specifically, we groupwisely co-register those intra-subject mean surfaces of all subjects to a common space, i.e., the inter-subject mean space. For each subject at any age, the longitudinally consistent inter-subject cortical correspondences are established based on the correspondences defined by their intra-subject mean cortical folding pattern, thus each cortical surface can be warped into the inter-subject mean space and further resampled with a standard mesh tessellation. Finally, we can build the 4D cortical surface atlas in this common space using a sparse representation technique as detailed in the following section.

2.3 Atlas Built by Spherical Patch-Based Sparse Representation

After above two-stage registration, the correspondences across subjects and time points are obtained. All subjects are now sitting in the inter-subject mean space. Thus, all spherical cortical surfaces from different subjects now share the same mesh structure with the same topology. Although a direct average over subjects at each age could obtain age-specific average atlas, this may lead to the over-smoothed cortical folding patterns due to large inter-subject variance (even after registration). Many detailed folding patterns after averaging will be lost, thereby degrading the registration accuracy when using this atlas to align a new subject.

To address this issue, we consider atlas construction as a problem of robust and sparse representation of underlying cortical folding patterns, by using a dictionary of individuals’ folding patterns. This will significantly reduce influences of outliers and also increase clarity and representativeness of folding pattern in the atlas. Specifically, first, we adopt the spherical patch-based representation to capture local folding patterns. Then, corresponding patches across subjects are collected to build the representation dictionary. To account for potential registration errors, neighboring patches are also augmented into the dictionary. Finally, for each local patch on the surface atlas, the sparse representation is adopted to robustly construct the cortical surface atlas from the built dictionary.

Construction of Comparable Neighboring Spherical Patches. Each spherical cortical surface is a triangular mesh, composed of vertices and edges, thus the patch of a vertex can be regarded as the l-ring neighboring vertices set. Figure 2(a) demonstrates the 1-ring patch for vertices \(\varvec{v}_1\) and \(\varvec{v}_2\). However, due to the inconsistency of the mesh structure at different vertices, patches are not directly comparable (i.e., two patches centered at \(\varvec{v}_1\) and \(\varvec{v}_2\), respectively, are different in their local connections and sizes). Hence, we need to have comparable neighboring patches for building the dictionary. To address this issue, the patch at \(\varvec{v}_1\) is rotated onto its neighboring vertex \(\varvec{v}_2\) (with \(\varvec{v}_2\) as one of the 2-ring neighbors of \(\varvec{v}_1\)), and the rotated patch (indicated by the orange dotted lines in Fig. 2(b)) with resampled cortical attributes is used as the patch for \(\varvec{v}_2\). In this way, we can construct comparable neighboring patches for neighboring vertices.

Dictionary Construction. Once the comparable neighboring patches constructed, we can build the dictionary for each local patch on the atlas. For a patch centered at vertex \(\varvec{v}_i\), the corresponding patches from N co-registered subjects are extracted and included into the dictionary, denoted as \(\varvec{p}_{\varvec{v}_i}^{(n)}\), where \(n=1,\dots ,N\) denotes the subject index. To further overcome the potential registration errors, the neighboring patches close to the current local patch are also extracted and included into the dictionary, denoted as \(\varvec{p}_{\varvec{v}_i^k}^{(n)}\), where \(\varvec{v}_i^k\) is the k-th vertex neighboring to the vertex \(\varvec{v}_i\) (for example, \(\varvec{v}_i^k\), \(k=1,\dots ,K\), is the 2-ring neighbors of \(\varvec{v}_i\) as illustrated in Fig. 3). By combining the corresponding local patches and also their neighboring patches, the dictionary \(\mathbf {D}_{\varvec{v}_i}\) for the patch centered at \(\varvec{v}_i\) can be built, as also illustrated in Fig. 3.

Fig. 2.
figure 2

Illustration on construction of comparable neighboring patches. (a) Inconsistency of the mesh structures at vertices \(\varvec{v}_1\) and \(\varvec{v}_2\). (b) Rotation of the patch at \(\varvec{v}_1 \) to \(\varvec{v}_2\) to construct the two comparable neighboring patches.

Full size image
Fig. 3.
figure 3

Building dictionary for a local patch centered at vertex \(\varvec{v}_i\) of the cortical surface atlas.

Full size image

Sparse Representation. Once the dictionary is built, the atlas construction is to sparsely represent the underlying cortical folding pattern in the atlas by using the dictionary of individuals’ cortical folding patterns. For each vertex \(\varvec{v}_i\) on the atlas, N local patches from N co-registered subjects can be obtained. However, due to the potential registration errors and also inter-subject variability, some patches may have less agreement in representing the population folding pattern. An effective strategy to filter out the outlier patches is to select those highly-correlated patches from the population. To do this, first, the group center of these patches are computed as the average of all patches; second, the correlation coefficient between each patch and the group center patch is computed; finally, the top M (\(M\le N\)) patches corresponding to the top M correlation coefficients are selected, denoted as \(\hat{\varvec{p}}_{v_i}^{(m)}\), with \(m=1,\dots ,M\).

The sparse representation of those top M patches can be formulated as [14]:

$$\begin{aligned} \varvec{x}(\varvec{v}_i)=\arg \min _{\varvec{x}\ge 0}[\sum _{m=1}^{M}\Vert \mathbf {D}_{\varvec{v}_i}\varvec{x}-\hat{\varvec{p}}_{\varvec{v}_i}^{(m)}\Vert _2^2 +\lambda _1\Vert \varvec{x}\Vert _1+0.5\lambda _2\Vert \varvec{x}\Vert _2^2 ] \end{aligned}$$
(1)

where \(\hat{\varvec{p}}_{\varvec{v}_i}^{(m)}\) corresponds to m-th extracted patches from the top M patches, and \(\mathbf {D}_{\varvec{v}_i}\) is the dictionary for local patch centered at \(\varvec{v}_i\). The first term in Eq. 1 encourages the constructed patch \(\mathbf {D}_{\varvec{v}_i }\varvec{x}\) to be similar to the selected top M patches \(\hat{\varvec{p}}_{\varvec{v}_i}^{(m)}\). The second term is a \(L_1\) regularization used to encourage the representation vector \(\varvec{x}\) to be sparse, and the last term is a smoothness term used to group select similar patches. We add the smooth term because neighboring patches are overlapped and highly correlated, if only using L1 norm without L2 norm as did in LASSO, the optimization will select just one patch from many correlated patches. \(\lambda _1\) and \(\lambda _2\) are non-negative parameters. By solving the above optimization problem using [8], the atlas patch centered at vertex \(\varvec{v}_i\) is obtained based on the corresponding representation coefficients \(\varvec{x}(\varvec{v}_i)\). Since each vertex is covered by multiple patches, the final atlas will be created by averaging multiple estimations at each vertex. In this way, for each age, a spherical surface atlas that contains sharp, population-representative cortical folding pattern is constructed.

3 Experiments

To evaluate the constructed 4D atlas, we compare it with atlases generated by 3 other strategies, including (1) simple averaging, (2) averaging over the top M highly-correlated patches, and (3) sparse representation that ignores averaging of multiple estimations at each vertex, i.e., using only the sparsely estimated patch center for each vertex on the atlas. In the experiments, we used the following parameter setting. The top 80% highly-correlated patches are selected. Each patch is defined by the 2-ring neighbors, and neighboring vertices used to augment patches are set as the 3-ring neighbors. The parameter \(\lambda _1\) is set to be 0.05, and \(\lambda _2\) is set to be 0.002. These parameters are learned from cross validation.

Fig. 4.
figure 4

Comparison of 12-month cortical surface atlases built with 4 different strategies. (a)–(d) shows the average convexity, and (e)–(h) shows the curvature.

Full size image
Fig. 5.
figure 5

Demonstration of the built 4D infant cortical surface atlas. (a) and (b) are in the spherical space. (c) and (d) are the average cortical surface with the folding patterns from (a) and (b), respectively. (e) and (f) are the equipped FreeSurfer parcellation and HCP MMP parcellation, respectively. Numbers in the left denote the month(s) of age.

Full size image

Figure 4 demonstrates a comparison for the atlases built at 12 months by four methods (including our proposed method) at two scales such as average convexity and curvature. Figure 4(a)–(d) show the average convexity, denoting a coarse-scale measurement of the folding patterns, while Fig. 4(e)–(h) show the curvature, denoting a fine-scale measurement. It can be seen that our method can better encode the folding patterns, compared to the other three methods, especially for the fine-scale folding patterns.

To equip our 4D atlases with cortical parcellations, the FreeSurfer atlas is further aligned onto our atlas at the last time point. Then, the FreeSurfer parcellation [1] with 35 regions in each hemisphere is propagated to the 4D atlas at each of other time points. For fine-grained parcellation, the HCP multi-modal parcellation (MMP) with 180 regions in each hemisphere [2] is first mapped to the FreeSurfer space using HCP workbench [11] and then propagated to our 4D infant cortical surface atlases. Figure 5 shows the built 4D infant cortical surface atlas at all 11 time points based on the collected infant dataset.

Since there is no ground-truth to evaluate the quality of built atlas, we use following strategy to quantitatively assess our 4D atlas. We divide the subjects into three subsets randomly. Two subsets are used for building the 4D atlas, and the left subset is used for evaluation. For each surface in the evaluation subset, we register it onto (a) the FreeSurfer adult atlas [1]; (b) the atlas generated by the FreeSurfer strategy, i.e., directly align all subjects together in one step; and (c) age-matched atlases generated by different strategies mentioned above and also shows in Fig. 4. If the atlas can better encode the folding patterns, the registered surfaces would be aligned in a more agreement way. We measure the alignment degree in coarse and fine evaluation ways. In the coarse evaluation way, we follow the evaluation in [7]. That is, all registered cortical surfaces are partitioned into gyral and sulcal regions. Then, at each vertex, for all aligned subjects, we can get the frequency of subjects that belong to gyral or sulcal region. Based on this frequency, the entropy can be calculated [7]. Finally, for all the vertices, the average entropy can be obtained. Clearly, lower values of the average entropy indicates better alignment of gyral and sulcal regions. In the fine evaluation way, we use the curvature map correlation to evaluate the agreement of the aligned folding patterns, as in [6]. That is, for each pair of aligned surfaces, their curvature map correlation is calculated, and then we can average for all possible pair of aligned surface to get the average correlation coefficient. Obviously, higher correlation coefficient indicates better alignment. Table 1 reports quantitative evaluation of different atlases at different time points in a coarse way, while Table 2 reports the quantitative evaluation in a fine way.

Table 1. Atlas evaluation in a coarse way by entropy.
Full size table
Table 2. Atlas evaluation in a fine way by average curvature map correlation.
Full size table

As can be seen, FreeSurfer adult atlas has lower agreement among registered subjects, indicating inappropriateness for infant brain analysis. Also, in both coarse and fine evaluation ways, our atlas boosted registration accuracy, indirectly indicating the folding patterns of infant population are better preserved.

4 Conclusion

In this paper, we built the 4D infant cortical surface atlas at densely sampled time points, from neonate to 6 years old. By using sparse representation of spherical patches, the surface folding patterns of infant population can be better preserved in the built 4D atlas, thus also boosts the surface registration accuracy and subsequent analysis. It worths noting that there are more recent registration methods [9] which may further improve our results. In the future, we would test other registration methods and includes more extensive validations, and we will also release our 4D infant cortical surface atlas to the public.