1 Introduction

Dendritic spines, small protrusions of the dendritic shaft, are one of the most important structures of neurons. Ramón y Cajal first identified spines in the 19th century and suggested that neuronal activity variations change the spine morphology [1, 2]. This claim has been supported by several studies reporting changes in the morphology and density with changes in neuronal activity [3–6]. Spines are the post-synaptic partners of a synapse [7] and are main receivers for synaptic input [2]. Dendritic spines in the hippocampal neurons are related with learning and short-term memory [8, 9]. Studies also reported that spine density is decreased due to some neuro-degenerative diseases such as Alzheimer’s [9].

Fig. 1.
figure 1

A dendritic branch with several spines imaged using a two-photon laser scanning microscope (2PLSM).

Full size image

A dendritic branch with several spines is shown in Fig. 1. Each spine has two segments, head and neck. Spine head is connected to the parent dendrite through the neck [10]. Dendritic spines exhibit extraordinary diversity [11]; they have different sizes and densities across different cell types, brain areas, and animal species [2]. A great variety in spine head and neck dimensions is usually demonstrated even within the same cell [2]. These facts emphasize the challenging nature of the spine analysis task. Dendritic spines have different shape types; researchers suggest that different morphological variations could possibly be related to various developmental stages or functional roles [12]. In the literature, dendritic spines have mostly been grouped into four shape classes: filopodia, mushroom, thin, and stubby [2, 8, 13–15]. Filopodia spines have long necks and no heads, mushroom spines have long necks and large bulbous heads, thin spines have long necks and small heads, and stubby spines are known to have either no necks or short necks [2]. An example of each of these classes is given in Fig. 2. Distribution of different shape types varies in different areas of the brain; it is also dependent upon the age of the animal being imaged, for instance stubby spines are dominant during early postnatal development [2].

Fig. 2.
figure 2

Spine Classes: Mushroom, Stubby, Thin, Filopodia (Left to Right). Intensity and corresponding manually annotated images are shown for each shape class.

Full size image

This classification of spine shapes has been widely used in the literature, however, there is an open research question concerning whether distinct classes of spines shapes exist or whether spines should be modeled through a continuum of shape variations. Parnass et al. [12] proposed that morphological groups of spine shapes do not represent inherent shape types, instead they depict shape variations a spine can go through during its life time. Bourne and Harris [16] noticed spine enlargement as a result of synaptic enhancement, causing transition of thin spines to mushroom type. Peters and Kaiserman-Abramof [15] reported the existence of spines with intermediate shape types and they found it difficult to assign them to one of the standard shape types. Basu et al. [17] reported a human expert being unsure while assigning labels to some of the spines. Arellano et al. [18] who used morphological features for spine analysis, also found several spines with intermediate morphological characteristics in their dataset. Spacek and Hartman [19] could not classify some spines into standard shape types and introduced a new class between mushroom and stubby, and thin and mushroom spines. Ruszczycki et al. [11] hinted towards a different classification standard: classifying spines into large and small, they reported better sensitivity with this classification approach. Wallace and Bear [20] used spine length and head diameter to perform spine analysis and found a continuous distribution. Mancuso et al. [21] suggested using morphological features to perform clustering and count spines in different clusters. In summary, different groups work with single or multiple neuroscience experts and each group uses their defined rules for classification, which results in subjectivity.

Quantitative analysis of dendritic spines is important for neurobiological research as it can help neuroscientists understand the underlying structure-function relationship. Currently this analysis is performed manually due to unavailability of reliable automated spine shape analysis tools. Manual analysis is a laborious, time-intensive, and most importantly subjective task. Rodriguez et al. [13] reported inter-operator and intra-operator variations in the spine type labeling task. Availability of reliable automated analysis tools can expedite research in this domain and assist neuroscientists decode the underlying relationship between neuron function and structure.

One might question why perform clustering rather than treating this as a classification problem. First of all, classification methods use manually provided labels as ground truth and extracting those labels is a time-intensive task. It also introduces subjectivity, which could be reduced by employing several experts and using a majority vote approach but this would make the labeling effort even more time-intensive. Inter-operator and intra-operator variability reported by Rodriguez et al. [13] emphasizes that subjectivity is a major issue in performing classification. Another issue with supervised classification is that it inherently starts from a pre-defined set of classes and does not allow exploration of potential intermediate shapes or possible continuous variation of shapes. Although clustering does not explicitly enable the latter either, it can be viewed as a step in that direction. Furthermore, some existing techniques require manual annotation of spines either to directly use them for feature extraction or for training segmentation algorithms. The objective of clustering in this context is two-fold: confirm the hypothesis of some distinct shape classes and discover new natural groups. We discover natural groups in the data using different features and analyze whether they support the existing hypotheses or add new information to our understanding of spine shapes.

As suggested by Mancuso et al. [21], we present a clustering-based approach for spine shape analysis. We perform cluster analysis using several feature representations and gain insights by performing analysis of discovered natural groups. We use Disjunctive Normal Shape Models (DNSM) [22], Histogram of Oriented Gradients (HOG) [23], intensity profiles [24], and morphological features [25]. We use an extension of k-means, x-means [26], to perform cluster analysis that uses the Bayesian Information Criterion (BIC) to select the number of clusters automatically. This study is based on two-photon laser scanning microscopy (2PLSM) images. Analyzing 2PLSM images is more challenging in comparison to confocal laser scanning microscopy (CLSM) images due to low signal to noise characteristics. Additionally, following the Abbe’s law [27], resolution of 2PLSM images is half of the CLSM images. The reason behind using 2PLSM is that it allows imaging of living cells, which would capture shape transitions during synaptic process [10, 28].

The major contribution of this paper is application of HOG-based features for spine analysis and cluster analysis of dendritic spines with different representations. To the best of the authors’ knowledge, this is the first paper that performs such an analysis of dendritic spine shapes with a wide range of feature sets. The rest of this paper is structured as follows. A brief summary of some of the related work is presented in Sect. 2. Section 3 discusses the methodology of our approach in detail. Experimental analysis and results are presented and discussed in Sect. 4. Section 5 summarizes the findings and conclusions of this paper.

2 Related Work

There exist several studies on supervised spine classification but none of these studies have reported performing unsupervised cluster analysis of dendritic spine shapes. Rodriguez et al. [13] performed spine classification on 3D images using morphological features. They developed a decision tree based classifier and evaluated its performance using labels provided by human experts. Son et al. [8] also developed a classification approach using morphological features and evaluated their approach with labels assigned by a human expert. Shi et al. [7] developed a semi-supervised learning approach for spine classification based on morphological features, and used human experts for validation of their results. A recent study on spine analysis applied ISOMAP [29] to study the importance of different morphological parameters and found neck length and head diameter to be the most prominent features for mushroom and stubby spines [30]. Ghani et al. [31] exploited the parametric nature of the DNSM approach and used its parameters for spine classification; they also used labels assigned by a human expert for performance evaluation. Erdil et al. [24] developed a joint classification and segmentation approach, within which they used intensity profiles for classification of spines. Labels assigned by a human expert were used to evaluate the performance of their algorithm.

As it can be noticed from a small subset of studies on classification summarized here, most of the groups use one or more human experts to assign class labels which are later used to evaluate the performance of their supervised classification approaches. Even though using the manually extracted labels as ground truth is a viable approach for this problem, it introduces subjectivity. We attempt to address this issue by presenting a clustering approach aiming to discover natural groups of spine shapes in an unsupervised fashion using various feature representations.

3 Methodology

We provide the details of our methodology in this section. Post natal 7 to 10 days old mice are imaged using 2PLSM.Footnote 1 We have acquired 15 stacks of 3D images using 2PLSM. After applying median filtering, we project 3D images to 2D using maximum intensity projection (MIP) [32]. We used 2D projections for this analysis, because resolution along the z-axis in our data is \(0.3\,\upmu \)m which is much worse than lateral resolution, which is \(0.019\,\upmu \)m or \(0.024\,\upmu \)m for different stacks. The slices along z-axis provide limited information [33]. The spines consist of a small head (\(\sim 1\,\upmu \)m diameter) and a thin neck (\(\sim 0.2\, \upmu \)m diameter), and are generally \(0.5\,\upmu \)m to several \(\upmu \)m long [2]. Due to low 2PLSM resolution, complete spine covers only a few slices along the z-axis. The low axial resolution makes the spine analysis in 3D very challenging even for human experts [34]. While there are other projection methods available, MIP is a standard projection procedure used in most of the neuroscience studies [10, 17, 33–35]. In total, 242 dendritic spines have been selected from 15 dendritic branches for this study.

3.1 HOG Features

HOG [23] computes histogram of gradient orientations and applies contrast normalization to improve performance. It is observed that spine heads have uniform intensities whereas intensity in the neck region is not uniform. A decreasing intensity pattern can be noticed in the neck part of the spines. Using this appearance information would help us discover clusters with different appearance patterns. In order to compute HOG features, we select a region of interest (ROI) in intensity images such that the spine is completely inside the ROI. This does not require the ROI for all spines to have the same dimensions. Further, we rotate the ROI such that spine necks are vertically aligned. Examples resulting from this process are shown in Fig. 3.

Fig. 3.
figure 3

Sample images from the dataset prepared for HOG.

Full size image

In order to capture fair amount of small-scale details, we selected the cell size as a function of width and height: \(CellSize=\big [height/5,width/5\big ]\); cells are small spatial regions. A large block size value allows to suppress local intensity changes; blocks are relatively large spatial regions. To keep moderate level of information about local illumination variations, we selected a block size value equal to twice the CellSize. Contrast normalization is controlled through block overlap, and we selected a block overlap of 1 cell. We used 9 signed histogram orientation bins, because using signed orientation allows to track light to dark and dark to light intensity changes. We computed 576-dimensional HOG feature vectors with these settings and later used these features for cluster analysis.

3.2 DNSM Features

DNSM is a parametric shape model, proposed recently by Ramesh et al. [36]. DNSM represents a shape as a union of convex polytopes, which are constructed by intersections of half spaces. DNSM attempts to approximate the characteristic function of a shape. For further details of the DNSM, readers are referred to [22, 36]. DNSM-based features provide a shape representation; it would be an interesting experiment to perform clustering using DNSM-based shape features. We apply DNSM to segment dendritic spine images following the approach in [31] and use 384-dimensional DNSM parameters as feature vectors to perform cluster analysis. A few images from the dataset used for DNSM features are presented in Fig. 4.

Fig. 4.
figure 4

A few images from dataset prepared for DNSM features: before segmentation (above) and segmented images (below).

Full size image

3.3 Morphological Features

Morphology of dendritic spines has been extensively studied in the literature. Most of the studies on spine analysis compute morphological parameters to perform classification of spines. In this paper, we use 12 morphological features suggested in a recent study on spine classification [25]. The morphological features we use are listed in Table 1. In order to compute these morphological features, we perform segmentation using DNSM and apply methods suggested in [25].

Table 1. List of Morphological features used.
Full size table

3.4 Intensity Profiles Based Features

Erdil et al. [24] suggests that intensity information in the regions in which a potential neck is likely to be contained can be used to differentiate spine classes. Regions where the neck might appear is found using the assumption that the spine neck lies below the spine head. Once the spine head is found by minimizing an intensity-based energy function using active contours [37], the approach in [24] creates two rectangular regions below the spine head as shown in Fig. 5. The first region shown in Fig. 5(a) is constructed such that the bottom point of the spine head (shown by a red cross) lies at the center of the rectangle. The second rectangular region shown in Fig. 5(b) is a narrower one and is drawn such that it is located just below the spine head. Erdil et al. [24] extract three sets of feature vectors by exploiting intensities in these rectangular regions which are combined to form 378-dimensional feature vectors. The first set of feature vectors is obtained by summing up the intensities in the first rectangle horizontally. Similarly, the second set of feature vectors are obtained by vertical summation of the intensities in the corresponding rectangle. The final set of feature vectors are the histograms of intensities in the second rectangular region.

Fig. 5.
figure 5

Regions in which a potential neck is likely to be contained. (Color figure online)

Full size image

3.5 Feature Selection

Considering the high-dimensionality of feature representations being used (except morphological features), we apply a feature similarity based unsupervised feature selection algorithm [38]. Mitra et al. [38] introduced the maximum information compression index, which attempts to minimize the information loss while selecting a certain number of features. Here, the aim of feature selection is to aid the clustering algorithm, we select 100 features for each feature representation (except morphological features) and use these selected features to perform clustering.

3.6 Clustering

Jain [39] suggests there are two objectives for clustering: (i) exploratory: when there is no existing hypothesis or model, the aim is to discover patterns, and (ii) confirmatory: when a pre-specified model or hypothesis exists, the objective of cluster analysis is to confirm the model on the dataset being used. For dendritic spine analysis, the literature provides a pre-specified model as described in the introduction section. The nature of our analysis is: (i) an attempt to analyze how well a pre-specified model fits our data, (ii) if such a model does not fit our data, discover and explore natural groups within the data.

Jain [39] argues that there is no best clustering algorithm, because every clustering technique implicitly or explicitly imposes a structure on the data, and it gives good results if there is a good match. Jain further emphasizes that it is rather crucial to select the appropriate representation that implicitly or explicitly makes the pattern discovery an easy process. Considering the clustering analysis problem as a selection of appropriate representation rather than selection of a clustering method, we have compared different feature representations in terms of clustering results. We applied x-means [26], an extended version of k-means, which does not require the number of clusters to be provided. It uses BIC to automatically select the number of clusters in the available data from a given range of number of clusters, which we set as 2 to 10. It begins with lower bound of given range for number of clusters and continues computing clusters until upper bound for number of clusters have been reached; during this process it also computes BIC score for each cluster assignment. Finally, it selects the number of clusters based on best BIC score.

4 Results and Discussion

Our dataset consists of 242 dendritic spines selected from 15 dendritic branches for this analysis. These are spines that have been labeled as mushroom or stubby by a human expert. Analysis of clusters formed using different feature representations is presented in this section.

4.1 HOG Features Based Analysis

Using HOG based appearance feature representation for x-means clustering resulted in 4 clusters. The average image for each cluster is computed by averaging manually segmented binary images in that cluster. The resulting images are shown in Fig. 6. There are 49 spines in cluster 1, 93 spines in cluster 2, 72 spines in cluster 3, and 28 spines in cluster 4. As it is evident from the average images, cluster 2 and cluster 3 represent mushroom spines (long neck and big head). However, clusters 1 and 4 appear to consist either of spines from both classes or of spines that may possibly lie in between these two classes in the shape space. When we examine individual samples from these clusters, illustrated in Fig. 7, we observe that they exhibit similar characteristics, i.e., have small heads and no necks. However, closer analysis of intensity images shows existence of short necks, i.e., low intensity regions just below the head part. These observations support the produced clusters in the sense that although there are some spines which are easy to be classified (grouped in clusters 2 and 3), even a human expert would have difficult time providing labels for most of the spines in cluster 1 and cluster 4. This analysis also points to what one might call two subclasses (cluster 2 and cluster 3) within the mushroom class.

Fig. 6.
figure 6

Average image for each cluster generated using the HOG features.

Full size image
Fig. 7.
figure 7

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines grouped in cluster 1 and cluster 4 using the HOG features.

Full size image

4.2 DNSM Features Based Analysis

We computed shape features using DNSM and performed clustering on this representation. The algorithm produced 4 clusters consisting of 32, 48, 50, and 112 spines. Average images of these clusters are given in Fig. 8. Most of the spines in cluster 1 have short or no necks; their head diameter to neck diameter ratio is approximately 1. A few spines from cluster 1 are presented in Fig. 9. This cluster appears to contain spines that clearly exhibit the characteristics of stubby spines as well as spines with distinct heads and thick necks. Cluster 2, cluster 3, and cluster 4 are mostly mushroom clusters.

Fig. 8.
figure 8

Average image for each cluster generated using the DNSM features.

Full size image
Fig. 9.
figure 9

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines grouped in cluster 1 using the DNSM representation.

Full size image

4.3 Morphological Features Based Analysis

Clustering analysis with morphological features resulted in 4 clusters with sizes: 102, 64, 64, and 12 spines. Average image for each of the produced clusters is given in Fig. 10. It is clear from Fig. 10 that cluster 1, and 2 are mushroom majority clusters. However, cluster 3 and cluster 4 show a mixed pattern, most of the spines have short thick neck, small head, and most importantly their neck diameters and head diameters are similar. A few spines from cluster 3 and cluster 4 along with their manually annotated images are presented in Fig. 11. These cluster appear to contain many stubby spines as well as spines with distinct heads and thick necks. It would be interesting to analyze which features are dominant in the clustering process, which might provide important information to neuroscientists. In this context, we perform an initial analysis using information gain [40] and conclude that neck length is the most dominant feature for data used in this study, which confirms analysis performed in some of our previous studies [25, 30].

Fig. 10.
figure 10

Average image for each cluster generated using morphological features.

Full size image
Fig. 11.
figure 11

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines from cluster 3 and cluster 4 using the morphology based features.

Full size image

4.4 Intensity Profile Features Based Analysis

Using the intensity profile based features resulted in 4 clusters consisting of 45, 81, 48, and 68 spines. The average image for each of these clusters is presented in Fig. 12. It is clear that cluster 1, cluster 2, and cluster 3 are similar and appear to consist mostly of mushroom-like spines, i.e., they have big heads and long necks. Spines in cluster 3 have relatively shorter necks as compared to cluster 2, spines in cluster 4 have big heads and very short or no necks. Some of the spines clustered in cluster 4 are shown in Fig. 13. This cluster appears to contain many stubby spines as well as spines with distinct heads and thick necks.

Fig. 12.
figure 12

Average image for each cluster generated using the intensity profile based features.

Full size image
Fig. 13.
figure 13

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines from cluster 4 generated using the intensity profile based features.

Full size image

4.5 Combined Features Based Analysis

Since, shape and appearance are complementary features, it is intuitive to combine both types of features and perform cluster analysis. We have already selected 100 features from each group using a feature similarity based approach. We combine these selected features to perform clustering in this section. Using a combination of HOG and DNSM based features results in 4 clusters consisting of 30, 78, 22, and 112 spines. The average image for each of these clusters is presented in Fig. 14. It is clear that cluster 2 and cluster 4 are similar and consist most of the mushroom-like spines, i.e., they have big heads and long necks. Spines in cluster 1 and cluster 3 are similar to one another in the sense that they have big heads and very short or no necks, as illustrated in Fig. 15.

Fig. 14.
figure 14

Average image for each cluster generated using HOG+DNSM features.

Full size image
Fig. 15.
figure 15

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines from cluster 1 and cluster 3 using HOG+DNSM based features.

Full size image

Using a combination of DNSM and intensity profile based features results in 4 clusters consisting of 32, 62, 36, and 112 spines. Average image for each cluster is presented in Fig. 16. Cluster 2, cluster 3, and cluster 4 consist of mostly mushroom-like spines, having big heads and long necks. However, cluster 1 consists of spines with intermediate properties: short, thick necks and big heads, as illustrated in Fig. 17. These spines have some morphological properties similar to mushroom spines and some similar to stubby spines, therefore, we may call cluster 1 a mixed or intermediate cluster.

Fig. 16.
figure 16

Average image for each cluster generated using DNSM+IntensityProfile features.

Full size image
Fig. 17.
figure 17

Intensity (top) and corresponding manually annotated images (bottom) for some of the spines from cluster 1 generated using DNSM+IntensityProfile features.

Full size image

4.6 Clustering vs. Human Expert

In this section, we compare the clustering results achieved using different representations to the labels assigned by a neuroscience expert. The idea is that similar data samples (belonging to same class) should be clustered in the same group. There are two challenges in spine shape analysis: (i) separating mushroom spines from stubby spines, and (ii) separating thin spines from filopodia type spines. Because of the developmental age of the animals we use, we see few filopodia in our data, this is why we focused on mushroom vs. stubby problem for this study. Stubby vs. mushroom analysis is a challenging task due to 2PLSM resolution limits. In fact, in stimulated emission depletion (STED) microscopy images, many reported stubby spines look like mushroom spines [41].

Table 2. Comparison of clustering results and labels from human expert
Full size table

A human expert manually labeled 242 spine images, 182 spines as mushroom and 60 as stubby. Table 2 shows the class membership of the spines in each of the clusters formed using each feature type. We observe that some clusters are dominated by shapes from one class whereas other are mixed. We have already analyzed the similarity within each of these clusters in the previous subsections, and observed the exploratory nature of our approach pointing to possibly intermediate shapes. Given the availability of manual labels, let us now carry out an analysis on the confirmatory aspects of our approach. In particular, to evaluate how strongly each clustering approach based on a different feature set confirms the manual shape labels, let us evaluate our clustering results using the manual labels as ground truth. To this end, let us pretend our clustering methods assign each cluster to the shape class with the majority of samples in that cluster. Then we can count the number of “correct and incorrect classifications". Using this approach, we evaluate these feature representations and find out that HOG features perform best on the available data taking the human expert’s labels as the ground truth, viewing this it as a classification problem we can achieve \(88.02\,\%\) classification accuracy.

According to expert’s labels, clusters 2, 3, and 4 formed with the DNSM representation correspond to the mushroom class, whereas cluster 1 is the stubby majority cluster. Sample images shown in Fig. 9 suggests that spines in cluster 1 have similar characteristics, however, the expert has labeled some of these spines as mushroom and others as stubby. This itself depicts the challenging nature of spine analysis and subjective nature of the manual labeling task. We have similar observations on clusters formed through the use of the other features. In particular, we observe both the confirmatory role of the clustering methods through the formation of clusters dominated by one of the classes as labeled by the human expert (e.g., HOG clusters 2 and 3), as well as the exploratory nature of clustering through the generation of clusters with mixed membership (e.g., HOG clusters 1 and 4). Our experimental analysis suggests that the possibility of intermediate shape types in addition to the conventional shape classes should be considered in spine shape analysis. One further step along this direction could involve efforts to characterize the distribution of spines in a continuous shape space.

5 Conclusion

In this paper, we have proposed a clustering approach to perform spine shape analysis. The advantages of adopting a clustering approach for spine shape analysis are: such an approach would not suffer from subjectivity, and analysis time would be reduced by avoiding manual labeling tasks. To the best of our knowledge an extensive clustering analysis of spine shapes has not been published. We use appearance, shape, and morphological feature based representations to perform clustering and shed some light on this problem. We perform clustering using x-means that uses BIC to select the number of clusters automatically; interestingly it produces 4 clusters for all of the features considered here, this implies there are 4 sub-groups in our data. Additionally, we have observed that, for the data used in our analysis, although there are many spines which easily fit into the definition of standard shape types (confirming the hypothesis), there are also a significant number of others which do not comply with standard shape types and demonstrate intermediate properties. Existence of intermediate shape types has been observed using all representations. It would be interesting to perform a neuroscientific analysis of produced clusters and understand biological meaning of each cluster produced. This is an initial analysis that provides clustering perspective on spine analysis and compare it with expert labels, it would also be interesting to use proposed approach to perform an analysis tying clusters to different experimental conditions.

The emergence of this phenomenon can be explained in several ways. It is a known fact that dendritic spines exhibit shape type transitions over time, this phenomenon happens over the period of hours. If the spines are captured at these transition periods, for instance a mushroom spine changing to a stubby spine, it might happen to have a short and thick neck and a head diameter to neck diameter ratio close to 1. As some spines in our data demonstrate such properties, it would be difficult to label them as mushroom or stubby. An alternative solution could be to define an intermediate class/group/cluster. A temporal analysis of several spine shapes would provide more insight into this phenomenon. It should also be noted that based on the expert labels, our data consists of two shape classes: mushroom and stubby. Including other shapes of spines such as thin and filopodia in the type of analysis we have proposed here might facilitate an even better understanding of the nature of shape classes and distribution. It might also be interesting to pose this as unsupervised regression problem, which would allow to study continuum of shape variations in a principled manner. It is important to mention that the distribution of spine shapes are dependent on various aspects of the data used, including which anatomical region of the brain the imaged neurons belong to as well as the age of the imaged neurons. This might also contribute towards different conclusions from different studies on spine shapes. Another potential issue might be performing the analysis on 2D projections versus 3D data. Therefore, it would be interesting to perform similar analysis with different 2D projection methods as well as 3D data. To conclude, the clustering perspective we propose in this paper can both be used to perform automated spine shape analysis to identify known shape classes as well as to help neuroscientists discover and explore unknown patterns in the shape space which have been previously ignored.