Exploring Fiber Skeletons via Joint Representation of Functional Networks and Structural Connectivity

Abstract

Studying human brain connectome has been an important, yet challenging problem due to the intrinsic complexity of the brain function and structure. Many studies have been done to map the brain connectome, like Human Connectome Project (HCP). However, multi-modality (DTI and fMRI) brain connectome analysis is still under-studied. One challenge is the lack of a framework to efficiently link different modalities together. In this paper, we integrate two research efforts including sparse dictionary learning derived functional networks and structural connectivity into a joint representation of brain connectome. This joint representation then guided the identification of the main skeletons of whole-brain fiber connections, which contributes to a better understanding of brain architecture of structural connectome and its local pathways. We applied our framework on the HCP multimodal DTI/fMRI data and successfully constructed the main skeleton of whole-brain fiber connections. We identified 14 local fiber skeletons that are functionally and structurally consistent across individual brains.

1 Introduction

Understanding the brain connectome has been significantly important in cognitive and clinical neuroscience [1,2,3]. It is fundamentally critical for researchers to understand the organizational architecture of human brain from both structural and functional perspective. With advanced neuroimaging techniques such as MRI, we are able to measure and quantify brain structure/function in vivo. When mapping brain connectivity, functional MRI (fMRI) and Diffusion Tensor Imaging (DTI) are two modalities commonly used. Based on fMRI and DTI datasets, many studies have been done to investigate the brain connectome using either functional interactions, e.g., correlations [4], partial correlations [5] and regression [6], or the strength of white matter connections [7]. On the other hand, numerous reports have indicated that the structural connectivity patterns “connectional fingerprint” of brain areas can largely determine what functions they perform [8]. However, multi-modality (DTI and fMRI) brain connectome analysis is still under-studied, in our view. The challenge is the lack of an efficient framework that can integrate the knowledge from two different modalities together.

Here, our proposed computational framework integrates two lines of research efforts including sparse dictionary learning derived functional networks and DTI derived fiber based structural connectivity into a joint representation of brain connectome. In this way, functional connectivity and structural connectivity can be studied and analyzed simultaneously. As illustrated in Fig. 1, we applied our framework on the Human Connectome Project (HCP) multimodal DTI/fMRI datasets to derive the main skeletons of the whiter matter pathways that are most active when performing different brain functions. We identified 14 major local fiber patterns from the main skeletons which have both functional and structural consistency across multiple individuals. The derived white matter skeleton and its local fiber patterns provide a new way to study brain connectome via multimodalities of MRI and shed novel insights on integrating brain structural and functional information.

Fig. 1.

The proposed framework of joint representation of functional networks (based on fMRI data) and structural connectivity (based on DTI data). (A) Functional networks from fMRI. (B) The cortical surface within DTI space. (C) The result of mapping functional networks onto the DTI cortical surface. (D) The whole brain fibers. (E) The fiber bundles connecting to functional activation areas in C. (F) An example of main skeletons of brain connections. Step1: registering the functional networks to DTI space; Step 2: screening fibers which connect the activation areas on the cortical surface; Step 3: using the joint representation profiles from step 1 and 2 for statistical analysis and constructing the main skeletons of the brain connections.

2 Materials and Methods

2.1 Data Acquisition and Preprocessing

For this study we used the data from HCP Q1 release [1] that includes seven task-fMRI datasets of 68 participants. The tasks include working memory, gambling, motor, language, social cognition, relational processing and emotion processing. For task-fMRI, the acquisition parameters are as follows: 72 slices, TR = 0.72 s, TE = 33.1 ms and 2.0 mm isotropic voxels. The acquisition parameters were as follows: 2 × 2 × 2 mm spatial resolution, 0.72 s temporal resolution and 1,200 time points. For DTI data, spatial resolution = 1.25 mm × 1.25 mm × 1.25 mm. More details of data acquisition and preprocessing may be found in [9].

2.2 Representation of Functional Networks

A brain functional network can be defined as the brain regions that functionally “linked” [10]. It has been proven that the dictionary learning and sparse coding approaches are able to successfully identify task-related and resting state brain functional networks even when they have overlaps in both spatial and/or temporal domain [11, 12]. Based on dictionary learning, the whole-brain fMRI signals can be represented as a linear combination of a relatively small number of dictionary signals. The major steps are illustrated in Fig. 2. Firstly, the whole-brain normalized signals are arranged into a matrix X (Fig. 2A) with n columns (n voxels) and each column contains a single fMRI signal with length of t (t time points). Then X is decomposed into two parts: dictionary matrix D (Fig. 2B) and a sparse coefficient matrix α (Fig. 2C). The empirical cost function is summarized in (1), and its aiming of sparse representation using D, $\ell \left( {x_{i} ,D} \right)$ is defined in (2), where λ is a regularization parameter to trade off the regression residual and sparsity level.

Fig. 2.

Pipeline of using dictionary learning to derive 2,800 functional networks for each subject.

$$ f_{n} \left( D \right) \triangleq \frac{1}{n}\sum\nolimits_{i = 1}^{n} {\ell \left( {x_{i} ,D} \right)} $$

(1)

$$ \ell \left( {x_{i} ,D} \right) \triangleq \mathop {\hbox{min} }\nolimits_{{\alpha^{i} \in R^{m} }} \frac{1}{2}\left\| {x_{i} - D\alpha_{i} } \right\|_{2}^{2} + \lambda \left\| {\alpha_{i} } \right\|_{1} $$

(2)

Each element of α indicates the extent when the corresponding dictionary atom is involved in representing the actual fMRI signals. As a result, each row of α can be mapped back to the brain volume space as a functional brain network pattern (Fig. 2C). Because 400 was proven to be an appropriate number of dictionary size [11] for HCP Q1 dataset, in this work we also set 400 for all task fMRI data. Thus, for each HCP subject, 2,800 functional networks will be obtained from seven tasks.

2.3 Representation of Structural Connectivity

In this section, we explore DTI derived fibers connecting to the activated areas of each functional network (step 2 in Fig. 1). Note that the fibers are under the DTI space, so we need to register the individual fMRI data to its own DTI space. Here we adopted a widely used linear registration tool – FLIRT from FSL [13]. White matter surface can be obtained through the DTI tissue segmentation and DTI cortical surface reconstruction algorithms [14]. Then we mapped the voxel from the registered fMRI data to its nearest vertex on the cortical surface and thus each surface vertex can be linked to the corresponding functional intensity in the decomposed coefficient matrix in Sect. 2.2. At last, for each cortical surface labeled with functional intensity values, we will examine the whole brain fibers and extract every fiber if both of its ending locations connected to activated regions on cortical surface (Fig. 1E). A threshold T = 0.5 is used to judge if a vertex on the cortical surface is active or not. Similar to the threshold in task activation detection [11], T is set empirically in this work. In this way, we could extract the fiber bundles which include all the connections from the activation area of different functional networks. A vector $N_{i}$ can be used to represent the fiber connection of the corresponding functional network i:

$$ N_{i}^{j} = \left[ {f_{1} ,f_{2} ,f_{3} \ldots f_{n - 1} ,f_{n} } \right] $$

(3)

where i represents the i-th functional network, j represents the subject index, f represents a fiber which is from the whole brain fibers and n is the total number of the fibers of subject j. The value $f$ will be set to 1 if this fiber has a connection to the i-th functional network and 0 otherwise.

2.4 Joint Representation of Functional Networks and Structural Connectivity to Identify Main Skeletons of the Brain Connections

In this section, we introduce a novel joint representation approach to integrate the functional and structural connectivity together to explore the main skeleton of fiber connectomes. In the Sect. 2.3, we can obtain the registered functional networks and the related fiber connections. Here, each fiber connection pattern we achieved was from a single functional network. However, the human brain is widely considered to include a collection of specialized functional networks that flexibly interact when different brain functions are performed [15]. Thus, instead of studying a single connection pattern derived from single functional network, we need a way to discover the fiber connectome in a global vision. In this work, instead of working on the overlap of the functional networks, we focus on the overlaps of fibers. A matrix Y is generated for each subject:

$$ Y \in {\mathbb{R}}^{m*n} $$

(4)

n represents the total number of fibers, m is the total number of functional networks (2,800 in this work) for each subject and $N_{i}$ defined in Sect. 2.3 is one row from Y. Each row of Y represents the fiber connections for a single functional network and each column represents the functional networks connecting to the corresponding fiber. Then, we are able to conduct the statistics of the elements in each column of the matrix Y, thus a histogram vector H can be computed:

$$ H = \left[ {h_{1} ,h_{2} ,h_{3} \ldots h_{n - 1} ,h_{n} } \right], h_{i} = \sum\nolimits_{j = 1}^{2800} {y_{j,i} } $$

(5)

where $h_{i}$ is the total number of functional networks that fiber i participated, and the more networks i participated, the more activated intensity i is. After we have the fiber connectome matrix Y and its histogram vector H, then we can rank those fibers from most activated fibers to the least activated fibers. Thus, we could identify which fibers tend to be more activated in the functional networks and use them to generate main skeletons of the brain connectomes. An example is shown in the Fig. 1F. It contains 5000 most activated fibers across the whole brain. In order to examine the consistency of the skeleton we obtained, we applied our approach on HCP Q1 release data.

2.5 Local Connectome Analysis Based on the Main Skeletons of the Brain Connectomes

The skeletons of brain connectomes describe the main connections across the major brain regions. More importantly, they represent the most commonly used fibers and their connection pathways in multiple functional networks. In order to better understand the main skeletons we obtained, we perform further analysis to investigate the local brain areas and connections that the skeletons connected to. To analyze the main skeletons, here we only focus on the fibers from the main skeletons obtained from Sect. 2.4 and examine the relationship between those fibers and functional networks. The main skeleton fiber connection matrix is defined as $Y_{s}$:

$$ Y_{s} \in R^{{m*n^{\prime}}} $$

(6)

where $n^{\prime}$ is the number of fibers from main skeletons. We extracted each row of $Y_{s}$ and studied the corresponding functional networks and fiber connections as well. Through a simple k-means clustering algorithm, typical local fiber pattern will be identified from the equation:

$$ c^{i} \text{ := }arg\mathop {\hbox{min} }\nolimits_{j} \left\| {x^{i} - \mu_{j} } \right\|^{2} , \mu_{j} \text{ := }\frac{{\mathop \sum \nolimits_{i = 1}^{m} 1\left\{ {c^{i} = j} \right\}x^{i} }}{{\mathop \sum \nolimits_{i = 1}^{m} 1\left\{ {c^{i} = j} \right\}}} $$

(7)

where c is the cluster of i, $\mu_{j}$ is the center of cluster j, x is the sample data.

3 Experimental Results

3.1 The Main Skeletons of the Fiber Connections of Human Brain

According to Sects. 2.2, 2.3 and 2.4, we obtained the main skeleton of the fiber connections from one subject at three different connectome levels, which are shown in Fig. 3. Three different connectome levels have 500 fibers (Fig. 3B), 5,000 fibers (Fig. 3C) and 10,000 fibers (Fig. 3D), respectively. Although the number of the extracted fibers is largely different, we find that the connectome pathway is relative robust. For example, the fiber connections in the frontal lobe are obvious and consistent across those three levels. We named these connectome pathways as the skeletons of the fiber connections of human brain. We want to emphasize that, in the paper, we used the main skeletons with level of 5,000 fibers as the standard and further analyses are also based on this level. The reason we choose level of 5000 is that it has the clearness and robustness of the fiber connectome pathways. In details, level of 500 occupies only 0.25% from whole brain fibers, thus this number is too small to clearly and completely represent the connectome pathway. Level of 10,000 holds about 5% fibers from whole brain, but among those 10,000 fibers, some fibers are not very active. Thus the sparsity of the connection matirx $Y$ is only about 0.0035, which is too small from our experience. In contrast, level of 5,000 accounts for nearly 2.5% fibers and the sparsity of the connection matrix is about 0.008, thus, level of 5000 is chosen.

Fig. 3.

The main skeletons of the fiber connections of an individual case. (A) The cortical surface of the brain. (B) Main skeletons of the fiber connections on level of 500. (C) Main skeletons of the fiber connections on level of 5,000. (D) Main skeletons of the fiber connections on level of 10,000.

3.2 The Consistency of the Main Skeletons of the Fiber Connections Across Different Subjects

In order to check the robustness of the main skeletons of the fiber connections we obtained, we adopted our framework on HCP Q1 release dataset. The main skeletons are obtained and we show 10 of them as examples in Fig. 4 to illustrate their consistency across different subjects.

Fig. 4.

The main skeletons of the fiber connections for 10 subjects. Each main skeleton is shown separately. The sequence of sbj1 to sbj10 is from left to right, and top to bottom.

From the Fig. 4, we can see that the main skeleton of the fiber connections is clear and consistent across the subject. Compared with whole brain fibers (as shown in Fig. 1D), these 5,000 most activated fibers describe clear connectome pathways for the fiber connectomes. Those connections represent the most dominant connection patterns under task performances and they connected significant brain regions. This result is interesting because the functional networks and whole brain fibers are from each individual and the way to obtain main skeletons is totally possessed individually. Impressively, the pattern of the skeleton is quite similar across these subjects. Further analysis for the consistency will be provided in the Sects. 3.3 and 3.4.

3.3 Explore Major Local Pattern for the Fiber Bundles from the Main Skeleton of the Fiber Connections

Using the approaches from the Sect. 2.5, we can obtain fiber connections for each functional network at the level of 5,000 fibers from $Y_{s}$, and we aim to investigate how the main skeletons participated in the functional networks. Thus, for each subject, Eq. 7 will be applied on the corresponding main skeleton fiber connection matrix $Y_{s}$. After examining the consistency of each fiber bundle cluster across the subjects, 14 major local patterns are identified from the main skeletons and are shown in the Fig. 5. These local patterns are reasonably consistent across subjects, and they compose the main skeletons.

Fig. 5.

The main connection patterns of the skeleton. (A) The main skeletons of the fiber connections. 10 local regions are highlighted with different colored circles. (B–O) 14 major local patterns are shown to present the contribution of local regions to the main skeletons. The numbers of their local regions from A are also provided in the figure.

From those 14 major patterns, 8 of them are the connections within the single brain region. Another 6 patterns are the combination of those 8 unique connections, meaning that some functional networks have much stronger activation levels and they may include two or more functional networks. This is an evidence for the hierarchical theory of functional networks. Another interesting finding is about the fiber connectome between left and right hemispheres. As we can see from Fig. 5B, D, E and F, the main fiber connections between left and right hemispheres are from corpus callosum. Apart from corpus callosum, there are many fiber bundles connecting left and right hemispheres, however, they do not belong to the main skeleton.

3.4 Corresponding Functional Networks for Major Local Patterns

It is interesting to know whether the corresponding functional networks of those local pattern fiber connections are consistent. It is worth noting that local pattern fibers are from the main skeleton fiber connection matrix $Y_{s}$. However functional networks are corresponding to the whole brain fiber connection matrix Y, $Y \gg Y_{s}$. So it is not necessary that the corresponding functional networks of same local pattern fibers must be consistent. To examine functional consistency of local patterns, we retrieved the fiber connections from $Y_{s}$ and their corresponding functional networks. We used local pattern from Fig. 5E as an example and illustrated them in Fig. 6. From Fig. 6, the activation areas are consistent, and they are located in the occipital lobe. That is, for those local patterns, their corresponding function networks are also consistent. In addition to the local pattern in Fig. 5E, other major local patterns have similar characteristics. Thus, we have the conclusion that the main skeletons of the fiber connections we obtained have not only structural consistency but also consistent functional networks.

Fig. 6.

The functional networks for local pattern Fig. 5E across the subjects. (A) An overview of functional network and its fiber connections (green lines) from sbj1. (B) Another 9 examples. The sequence of sbj2 to sbj9 is from left to right, top to bottom. Color bar is shown on the bottom left.

4 Conclusion

In this paper, we proposed a novel framework for joint representation of structural connectivity and functional networks to explore the main fiber skeletons of the brain. The major advantage of our framework is that it enables learning connections from multimodality (both fMRI and DTI) to investigate the most activated fibers and then derive the main skeletons of fiber connections. The analysis of our framework on HCP multimodal DTI/fMRI data suggested that main skeletons of the fiber connections can be robustly identified. In addition, through studying the main skeletons of the fiber connections, typical local patterns can be discovered and studied. Those local patterns will help to not only present both functional and structural consistency across different subjects, but also provide a new insight to understand the mechanism of the fiber connectome of the brain.