Radiological image retrieval technique using multi-resolution texture and shape features

Abstract

Medical image analysis plays a very indispensable role in providing the best possible medical support to a patient. With the rapid advancements in modern medical systems, these digital images are growing exponentially and reside in discrete places. These images help a medical practitioner in understanding the problem and then the best suitable treatment. Radiological images are very often found to be the critical constituent of medical images. So, in health care, manual retrieval of visually similar images becomes a very tedious task. To address this issue, we have suggested a content-based medical image retrieval (CBMIR) system that effectively analyzes a Radiological image’s primitive visual features. Since radiological images are in gray-scale form, these images contain rich texture and shape features only. So, we have suggested a novel multi-resolution radiological image retrieval system that uses texture and shape features for content analysis. Here, we have employed a multi-resolution modified block difference of inverse probability (BDIP) and block-level variance of local variance (BVLC) for shape and texture features, respectively. Our proposed scheme uses a multi-resolution and variable window size feature extraction strategy to maintain the block-level co-relation and extract more salient visual features. Further, we have used the MURA x-ray image dataset, which has 40561 images captured from 12173 different patients to demonstrate the proposed scheme’s retrieval performance. We have also performed and compared image retrieval experiments on Brodatz and STex texture, Corel-1K, and GHIM-10K natural image datasets to demonstrate the robustness and improvement over other contemporaries.

Introduction

Medical image production has been drastically increased from the last couple of years. This generated data is stored in distributed places to be available to the authorized person worldwide for studying past cases for the best probable treatment. Radiological images are the elementary constituent of medical images to help a respective medical practitioner. These radiological images are not self-explanatory or sometimes misleading. So, doctors use visually similar images based on past cases for their treatment purposes. Hence, they are looking for a sophisticated retrieval mechanism to retrieve visually similar images.

So, a conventional mechanism like tag-based image retrieval [7] is not suitable as data is stored at distributed places. This data is handled by various persons with different expertise and perception. Like, one can tag the stored data as colon cancer data, while others may keep them as rectal cancer stage-1 data and so on. Therefore, it is better to search for images according to their visual content. These results may not be accurate, and finally, medical experts have to make the final decision based on their own experience. In this scenario, content-based image retrieval (CBIR) [18] is the most promising tool to retrieve visually similar images. In CBIR, an image has been represented in terms of its primitive visual features like color, texture, and/or shape. There are various works available in the literature based on natural images. Hence, their process might be different and may have better results as they include rich color information in their works. But, in the gray level image retrieval process like radiological image retrieval, we need an effective mechanism that can capture the deep collection of texture and shape since color information is missing. So, here, we have addressed the issue of extracting texture and shape based feature vector formation. In our work, we have found that this proposed scheme is working effectively with the radiological image dataset. Further, the system has been validated with texture image dataset, and also for the comparison purpose, we have corroborated our work with benchmark natural image datasets that are frequently used in the CBIR processes.

Related work

According to CBIR literature, it revolves around its three fundamental visual features i.e., color, texture, and shape. Initially, CBIR systems are designed which indulge only one of its primitive features [15]. But, later to enhance retrieval performance, researchers start exploring its various combinations. In CBIR, natural image-based retrieval is one of the key aspects, and natural images contain a wide range of color information. In work [23],authors have proposed an improved chroma-based CBIR that combines luminance-based and chroma-based texture information with low dimensional feature vector, which can achieve retrieval accuracy of less than 60% only. Similarly, in work [13], authors have presented a scheme that incorporated texture and shape image features. To extract image texture feature, foreground and background regions are captured from the Y-component of YCbCr counterpart of the input RGB image. Afterward, to incorporate shape features, edge histogram descriptor has been employed to both the chrominance parts where accuracy is around 67%. These schemes result in lower retrieval efficiency as the color feature is not included. Hence, color information must be addressed to improve retrieval efficiency. In the scheme [40], authors have proposed a multi-resolution approach for extracting image features. In this work, discrete cosine transformation (DCT) has been employed in different resolutions formed through the Gaussian image pyramid. Then, the DC coefficient and AC coefficients are extracted from each resolution. Now, to form the final feature vector, some statistical parameters have been evaluated. Here, retrieval efficiency has been upgraded with a small margin as color information is not directly addressed. In work [29], all the primitive features are extracted with clustering images based on their semantics. So, if the extracted features are fussed properly, then this will improve the retrieval performance. In work [30], a hierarchical framework is used where at each level, only one primitive feature has been extracted. Similarly, in work [44], color, texture, and shape features are extracted and later fussed together for the retrieval process. In the work [9], for making LBP effective for the colored images, multi-channel LBP is proposed, combining color information and texture. In the paper [45], which clubs together color, texture, and shape image features. For the color information, clustering-based color quantization is used, and then dominant colors have been favored. Afterward, for the texture image features, steerable filter decomposition came into existence. Lastly, pseudo-Zernike moments is deployed for extracting the shape image features. In the paper [12], color, texture, and shape features are extracted from their individual histogram. In work presented in [13], initially, pertinent image is selected using color moments. Then, local binary pattern followed by canny edge detector is used for texture and edge information, respectively. Here, Manhattan distance is used for similarity measurement to select the most appropriate images. In work [32], the authors have presented a work which includes color, shape, and texture features. This paper proposed the color edge map where color and shape features are extracted. In the later stage, they have extracted texture features from the non-over block using principle texture direction. In the end, all the features are combined to form the final feature vector, and the same is used in the image retrieval process. In work [43], authors have comprised color and texture information. This work has proposed a maximal multi-channel local binary pattern to overcome the issue of the lengthy feature vector for the local binary pattern. In work [16], authors have incorporated color and shape features. Here, the color feature is extracted using color auto-correlogram. For the shape feature, background and foreground are detected, then GLCM is employed on the foreground and statistical parameters on the background. Then all features are combined together to form the feature vector, and the same is used in retrieval. The work [27] modify enhances the capabilities of MSD by first find the relationship between shape and texture and then between color and texture. The scheme [31] combines color, texture, and shape features. Here, color information is extracted from a probability-based semantic centered annular histogram. Later, texture and shape features are extracted from SIFT and dual-tree complex wavelet transform. The work [39] illustrated a layered approach. In the first layer, color, texture, and shape features are extracted. Then, on the second level, there are two layers. The first layer found similar images based on color and shape features. At last, the second layer uses the result of the first layer and again found the desired numbered of similar images based on color and texture information. In work [2], color and texture features are extracted from color moments and Gabor and discreet wavelet transformation. At last, color and edge directivity descriptor is inducted for better performance.

Motivation

From the above discussion, it is quite clear that color is one of the vital information for natural images. But, for images like radiological images that are gray in nature, one emphasizes texture and shape features. So, a system is required, which can capture profound information and, at the same time, be able to handle the uneven distribution on an image. Secondly, with the widely distributed picture archiving and communication systems (PACS), the stack of medical images increases rapidly. Since these images are generated at an associated cost, a medical practitioner, rather than confirming with multiple test images, search the visually similar past cases corresponding to the input image for the probable best treatment. So, at the very first stage, designing a texture and shape based CBIR system to retrieve visually similar radiological images becomes the need of the hour. So, in this paper, authors have proposed a content-based radiological image retrieval (CBReIR) scheme, which incorporates texture and shape visual features to retrieve visually similar images. In a retrieval system, features are extracted either globally i.e., where a feature is extracted from the entire input image or locally i.e., the feature is extracted from the non-overlapping blocks. But, it has been observed that global image features failed to provide the desired feature representation as in radiological images, features are quite localized; hence local feature extraction proves to have more correlation with the actual content of the input image. But the situation with the radiological images is that a small portion even within a block has an uneven distribution. So, block-level feature extraction fails to capture the situation and raise a serious concern. So, to overcome this issue, in this paper, authors have proposed a novel multi-resolution approach based on modified BDIP and BVLC. Modified BDIP and BVLC have been applied to capture the multi-resolution images by applying them with different window sizes. Then, over these images, various rounds of DWT are employed. Since the sizes of these DWT images are not negligible, GLCM parameters are deployed to form both portions of the feature vector. Finally, both feature vector parts are clubbed together with a weight factor to form the final feature vector. This weight factor allows us to incorporate greater information from a more significant part. The same feature vector is used for the retrieval process on the accounts of a similarity measurement.

Contribution

In this paper, authors have proposed a novel CBReIR scheme based on the texture and shape features. In this work, not only medical images but typical natural images are considered for the retrieval and comparison purpose. The major contributions of this scheme are as follows:

  • This paper exhibits a novel methodology to extract texture and shape information from a typical x-ray as well as from the natural images in multi-resolution mode.

  • To capture the uneven distribution in multi-resolution mode with modified BDIP and BVLC have been deployed.

  • In order to form the feature vector, GLCM with various parameters depending upon the significance of the information has been applied.

  • To validate the scheme, this work has been tested on MURA x-ray image database. This image dataset contains 7 different underlying objects as well as results are retrieved and compared on the basis of one texture image dataset i.e. Brodatz and STex, and two natural image datasets i.e. Corel-1K, and GHIM-10K.

The rest of the paper is organized as follows: Section 2 is the part of the preliminaries where different existing used techniques are discussed in brief. Section 3 illustrate the proposed content based radiological image retrieval (CBReIR) scheme. Further, Section 4 describes the various results and comparison of results with other similar works. Finally, Section 5 concludes the paper followed by the corresponding references.

Preliminaries

This section describes the basic concepts briefly those are used through out the paper.

Block difference of inverse probability (BDIP)

For any digital image, an edge signifies a region which contain sudden change in intensity around their surroundings and valley signifies a region with minimum intensity in a particular region. These two operators have a vital role in human identification of objects shapes. The operator which is used to draw the sketch features of digital images with valleys and edges is known as difference of inverse probabilities (DIP) [35]. In DIP, probability in defined as the ratio of the pixel intensity to the sum of the intensity for a image window of size n × n. So, according to definition, DIP can be described as the difference of the inverse probability of the center pixel to the maximum intensity for particular window. BDIP [8] is the blocked version of DIP. It is defined as the difference between the number of pixels in a window to the ratio of sum of the all the intensity of a window to the maximum intensity of that window and it can be explained mathematically as in (1)

$$ \beta^{k}(b)=\frac{\frac{1}{|\beta^{k}(b)|}{\sum}_{\left( i,j\right)\in\beta^{k}(b)}\left( max_{i,j\in \beta^{k}(b)}I\left( i,j\right)-I\left( i,j\right)\right)}{max_{i,j\in \beta^{k}(b)}I\left( i,j\right)} $$
(1)

where \(I\left (i,j\right )\) denotes the intensity of the pixel \(\left (i,j\right )\) in the block βk(b) of size (k + 1) × (k + 1), l is addressing the position of the block in the image, k is the maximum distance of pairs of pixel in the block, |βk(b)| is the size of block or window. In the Fig. 1, we have pictorially represent the BDIP with window size 2 × 2 based results for various medical images.

Fig. 1
figure1

Some BDIP exemplifiers for radiological image

Block level variation of the local variance

One of the tool to measure the textural smoothness of an image is block variation of local correlation coefficient (VLCC) [37]. It can be illustrated as the difference between the maxima and minima of the local correlation coefficients. Block Level Variation of the Local Variance (BVLC) [35] is the block based edition of the VLCC. BVLC can be described as the difference between the local minimum and maximum correlation coefficients in four different directions where local correlation can be explained as local covariance normalized by local variance as illustrated in (2). Figure 2 is showing the different BVLC results. In Fig.2a, is the input Lena intensity image, Fig. 2b is exemplifying BVLC output for window size of 2 × 2 and Fig. 2c is exemplifying the BVLC output for window size of 4 × 4. So, from the figures as well it is very clear that if the size of the input image is N × M then if the window size is k × k then the size of the BVLC output will be \( \frac {N}{k} \times \frac {M}{k}\).

$$ \rho^{k}\left( {b,\triangle(k)}\right)=\frac{\frac{1}{|\rho^{k}(b)|}{\sum}_{(i,j)\in\rho^{k}(b)}I(i,j)I\left( i+\triangle_{i}(k),j+\triangle_{j}(k)\right)-\mu_{b}\times\mu_{b+\triangle_{(k)}}}{\sigma_{b} \times\sigma_{b+\triangle(k)}} $$
(2)

where μb and σb are the local mean and local standard deviation of the block \({\rho ^{k}_{b}}\) respectively. \(\triangle _{(k)}=\left (\triangle _{i}(k),\triangle _{j}(k)\right )\) is the represent the shift in either of four directions as shown in Fig. 3 i.e 90o,0o,− 90o,− 180o i.e. ρ(0,1),ρ(1,0),ρ(0,− 1),ρ(− 1,0)

Fig. 2
figure2

a Input Lena image b2 × 2 BVLC image c4 × 4 BVLC image

Fig. 3
figure3

Four direction of BVLC

Similarly, \(\mu _{b+\triangle _{(k)}}\) and σb+△(k) is the mean and standard deviation of the resultant shifted block or window of size \(\frac {1}{|\beta ^{k}(b)|}\).

$$ BVLC= max\left[\rho^{k}({b, \triangle(k)}\right]_{\triangle_{(k)}\in O_{4}}- min\left[\rho^{k}({b, \triangle(k)}\right]_{\triangle_{(k)}\in O_{4}} $$
(3)

where O4 = {(0,1),(1,0),(0,− 1),(− 1,0)}. Since, BVLC measure the roughness of the surface so higher the value of BVLC higher rough surface as explained in (3).

Proposed scheme

CBIR is an active research area where the main concern is to retrieve visually similar images based on their primitive visual content like color, texture, and shape. As discussed earlier, most of the CBIR techniques have used all three primitive features in a proper order to improve retrieval performance. However, in few CBIR works have addressed the retrieval of radiological images. It is a challenging task to retrieve radiological images based on extracted texture and shape features, so in this paper, authors have proposed a novel scheme that extracts these features very profoundly. Thus, we can match retrieval performance with its contemporaries.

It is a well-known fact that local image features represent an image more proficiently than the global feature image features [47]. But, even local feature extraction is not feasible in case of radiological images as within the same block, there exists diversification. So by following this lead, In this paper, authors have extracted image features from the different resolution of an image so that retrieved image features correspond to the actual image as close as possible. If a feature remains untouched in a more significant-resolution, it can be addressed in a smaller one and vise versa. Since color information is missing in radiological images hence in this work, texture and shape features are extracted from the different multi-resolution images. Here, BVLC with variable window sizes is deployed to create multi-resolution texture images from the input radiological image. For the same input image, for multi-resolution shape images is achieved by BDIP. Now, to extract local features from these received images, 2-D DWT has been employed accordingly on each received image, which means that higher numbers of rounds are employed on the image with a smaller resolution. Now, This leads to having an approximate image of the same size at the end.

As the approximate images size is not negligible, to summarize the extracted features GLCM with the different number of statistical parameters is employed on each sub-band of the DWT. This means band with higher information like LL sub-band of the 2-D DWT contains more vital information than the other three sub-bands, so the LL sub-band must get more privilege than others. From LL-band, energy, and standard deviation and from others, only energy has been calculated. So, here weight factor in terms of GLCM parameters is given to LL sub-band.

Algorithm 1 is partitioned into two sections, i.e., shape based feature and second is texture based feature extraction. Since BDIP is the sketch operator so it is used with shape features while BVLC is the roughness estimator hence used for texture feature. In the Fig. 4, we have pictorially represent the feature extraction process for a radiological image. Since, in this work, we have also considered natural images as well. In that case, first of all, the input natural image is divided into its R, G, and B color plane and then the same process depicted in Algorithm 1 is carried out. The various steps involved in the feature extraction process are illustrated as follows:

figuref
Fig. 4
figure4

Pictorial representation of proposed scheme for radiological image

In the Algorithm 2, we have illustrated the image retrieval process on the basis of extracted features for a respective database.

figureg

Experimental result and discussion

This process has been carried out in MATLAB environment using a system with Intel(R) Core(TM) i7-4770 CPU @ 3.40 GHz, 4 GB RAM. In this work, we have examined our scheme through MURA image x-ray dataset which consist of seven categories namely Elbow, Finger, Forearm, Humerus, Shoulder, and Wrist.

Now, to quantify the performance of the proposed scheme, three widely parameters are used. They are as follows:

First one is the precision which can be describe as in (4)

$$ Precision= \frac{REL_{R}}{TOTAL_{R}} $$
(4)

Second parameter is recall that can be formulated as:

$$ Recall=\frac{REL_{R}}{DB_{N}} $$
(5)

Final parameter is F-Score which is the weighted harmonic mean of the precision and recall and measure the overall performance of the scheme, can be mathematically modeled as:

$$ F-Score=\frac{2 \times Precision \times Recall}{Precision + Recall} $$
(6)

For the equations (4-6) RELR is the relevant images corresponds to the category of the query image, TOTALR is the total images retrieved which is the collection of relevant and non-relevant images and DBN is the total relevant images presented in the concerned dataset.

In the proposed work, a feature vector is constructed with the concatenation of features extracted from the multi-resolution BDIP and BVLC images. To start the process, firstly user selects a random image from the medical image dataset. Then, BDIP and BVLC based processes are executed simultaneously at the same time independently over the user-selected query image QI. In BDIP based multi-resolution feature extraction process starts with applying 2 × 2, 4 × 4 and 8 × 8 window sized BDIP process over QI and the same process is carried out for the BVLC part. So, if the size of medical image is R × C then the after the BDIP process the resultant images will be of size \(\frac {R}{2} \times \frac {C}{2}\), \( \frac {R}{2^{2}} \times \frac {C}{2^{2}}\), and \(\frac {R}{2^{3}} \times \frac {C}{2^{3}}\) respectively. Afterward, to extract local features, 3-level, 2-level, and 1 level 2-D DWT are employed on each of them respectively. Finally, the approximated images will be of the size of \( \frac {R}{2^{4}} \times \frac {C}{2^{4}}\). Since the dimensionality of these retrieved images is not negligible so to extract final information GLCM with the different number of variables is employed.

Computational time overhead is one of the primary parameters to judge any scheme. Time taken by a scheme to execute solely depended on the feature vector formation and by combining all the features of an image, we get the final feature vector. So, the length of the feature vector plays an important role in scheme analysis. In this proposed scheme, two statistical parameters of GLCM have been evaluated from every LL sub-band and only one statistical parameter of GLCM has been evaluated from every other sub-band. Now, 2 × 2 window size, there are 3 LH, 3 HL, 3 HH, and 1 LL bands i.e. total 10 sub-bands. For 4 × 4 window size, there are 2 LH, 2 HL, 2 HH, and 1 LL bands i.e. total 7 sub-bands. At last for 8 × 8 window size, there exist 1 LH, 1 HL, 1 HH, and 1 LL sub-band i.e. combining 4 sub-bands. So feature vector size for a input medical image is ((9 + 6+ 3)×1(LH, HL and HH subbands)+ 3(LL subband)×2)×2(BDIP and BV LC) = 48.

Results and discussion on medical image dataset

In this section, we will discuss the results for the associated medical image dataset based on the three defined parameters. The concerned MURA [34] x-ray image dataset has a total of 40561 images from the total 12173 patients. This image dataset contains total 7 underlying categories namely Elbow, Finger, Forearm, Hand, Humerus, Shoulder, and Wrist. Here, the size of an image is either 512 × a or b × 512. In Fig. 5, we have showcased the confusion matrix for the medical image dataset. The confusion matrix or error matrix is another way to represent our mode statistically. It allows us to visualize, how perfect is our proposed scheme working for the medical image dataset.

Fig. 5
figure5

Confusion matrix for radiological image dataset

Here, in Fig. 6, we have represented various values of precision, recall, and F-score for each category concerning a different number of outcome images starting from Top-5 up to Top-25. In this figure, the X-axis demonstrates the number of categories and the Y-axis represents the percentage value of the concerned parameter. In the Fig. 7, we have exemplified some of the visual results for all the categories of the medical image dataset. The mean average precision (mAP) of proposed system is 89.2%.

Fig. 6
figure6

a Precision, b Recall, c F-Score for the MURA x-ray image database based on different numbers of retrieved images

Fig. 7
figure7

Retrieved results for all the categories of MURA x-ray image dataset

Since we can not compare our work based on the respective medical image dataset. So, for the comparative study, we have drawn results from several benchmark datasets as well. Their results and comparative analysis are described in the upcoming section.

Results and discussion on benchmark datasets

In this section, we have demonstrated our results on the basis of two texture image dataset i.e. Brodatz available at [http://sipi.usc.edu/database/database.php?{volume}=rotate,], STex [14] and two natural image datasets Corel-1K [19] and GHIM-10K [22]. Since the medical images are gray in nature, so we have incorporated texture database which allows us to compare the results on the basis of similar structured images. The various details of these image datasets are given in Table 1.

Table 1 Various datasets

In The Table 2, we have displayed the various results based on precision, recall, and F-score for Top-10, Top-20, Top-30 retrieved images. As, in this work, texture and shape features are deeply extracted so the results for the texture images are on the higher side. Afterward, for the comparative study, we have compared our results with its contemporaries in Table 3 on the basis of average precision. In CBIR literature, it is very common to draw results from some benchmark image datasets. Corel-1K and GHIM-10K are one of those well established and accepted datasets. The results we got from these natural image datasets are not as good as other datasets because natural images have rich color content and those color information must be addressed for better results. As, in this paper, only texture and shape features are incorporated, so we have received some downgraded information but still our work produces comparative results. In Fig. 8, confusion matrix has been displayed.

Table 2 Categories wise results for Brodatz texture dataset for various number of output images
Table 3 Comparison of Brodatz dataset on the basis of mAP
Fig. 8
figure8

Confusion matrix for Brodatz dataset

Now, for the Salzburg texture image database (STex) has 476 images of size 512 × 512. Then each image is divided into 128 × 128 hence from one image there will be total 16 images is formed which produces the cardinality of database to 7616. Here, the categories like Buildings, Grass, Hair, Leaf, Leather etc. have images varies from 32 to 1232. We have drawn results for Top-16, Top-32, and to compare our results with its contemporaries Top-25 images for each category. Mean average precision, recall, and f-score for Top-16 images are 77.72%, 10.25%, and 18.11% and for Top-32 the corresponding values are 56.15%, 13.66%, and 21.97% respectively. Afterwards, we have compare our results with similar works like LBP (local binary pattern) [28], CND (combined neighborhood difference) [36], DLEP (directional local extrema pattern) [25], LTrP (local tetra pattern) [26], LBP+LNDP (local binary pattern + local neighborhood difference pattern) [42] on the basis of mean average precision for Top-25 retrieved images in Table 4.

Table 4 Comparison of STex database on the basis of mAP

In Fig. 9, we have demonstrated our result through a confusion table for the Corel-1K dataset. In the Fig. 10, various parameters for the Corel-1K database corresponding to different numbers of output images ranging from Top-5 to Top-25 retrieved images have been illustrated. In this figure, X-axis denotes the number of categories of the Corel-1K dataset and the Y-axis represents the percentage precision, recall, and f-score values successively for the figure (a), (b), and (c). From these figures, it is clear that we got good results where the shape of the primary object is distinct. In Fig. 11, average values for precision, recall, and f-score for the different number of output images have been represented. Now, to prove that our work produces comparable results, we have compared our work in Table 5 on the basis of precision, recall, and F-score respective to each category of Corel-1K dataset.

Fig. 9
figure9

Confusion matrix for Corel-1K dataset

Fig. 10
figure10

a Precision, b Recall, c F-Score for the Corel-1K database based on different numbers of retrieved images

Fig. 11
figure11

Average precision, recall, and F-score for different numbers of outcome images of Corel-1K database

Table 5 Comparison table for Corel-1K database

In the Fig. 12, we have shown the comparison of the proposed scheme to other state-of-art works with respect to average precision, recall, and F-score as a whole. Form the figure, it is clear that the as an average our proposed scheme has better performance than the other CBIR systems.

Fig. 12
figure12

Comparison based on mean average precision, recall and F-score for Corel-1K database

As the Fig. 12 indicates that the mAP of the proposed system is 82.5% but there exist other schemes like [1, 2, 39] whose respective mAP are 87.5%, 83.5%, and 92%. Their system has higher performance because the color feature is extracted vividly in each case that enhances their performance. Since this proposed system only shapes and texture features are extracted, its result is on the lower side but still comparable.

Here, in the Fig. 13, we have presented the percentage values of various evaluated parameters of GHIM-10K database for the various numbers of output images in the figure (a), (b), (c) respectively. In this figure, X-axis start with 1 and ends at 20 which represents the total number of categories in the GHIM-10K database while the Y-axis represents various parameters for the concern sub-figures. In the Fig. 14, we have exhibited the confusion matrix for GHIM-10K dataset. Now, to proof the overall performance of the proposed system average of each of the evaluated parameters are also computed which has been pictorially represented in the Fig. 15 for different numbers of output images. At last, we have compared our scheme with other similar works on the basis of average precision in the Table 6.

Fig. 13
figure13

a Precision, b Recall, c F-Score for the GHIM-10K database based on different numbers of retrieved images

Fig. 14
figure14

Confusion matrix for GHIM-10K dataset

Fig. 15
figure15

Average precision, recall, and F-score for different numbers of outcome images of GHIM-10K database

Table 6 Comparison of GHIM-10K dataset on the basis of mAP

Conclusion

In a CBIR system, rich content of color information must be addressed for better retrieval performance but when it comes to images like radiological images, where color information is missing then it becomes a challenging task. So, in this paper, authors have proposed a novel texture and shape based radiological image retrieval where modified BDIP and BVLC information has been captured in a multi-resolution mode with different window size. This multi-resolution approach has increased the retrieval performance of radiological images. Here, we have tested our scheme on MURA x-ray image dataset. Now, for the comparative analysis, we have compared our scheme based on a texture image dataset as it has a similar structure and we have produced excellent results. On the other hand, we have also drawn results from two benchmark natural image datasets and compare our results with its contemporaries. Though results are not as good as other database results yet they are still comparable with state-of -art works.

In a radiological image a portion of the image is on the darker side which is not dealt in this paper. So, for the future, we will emphasis on the region of interest in each input image. Secondly, with the inclusion of the high level semantics not only the efficiency will increase but also a musculoskeletal case based system can be generated.

References

  1. 1.

    Ahmed KT, Ummesafi S, Iqbal A (2019) Content based image retrieval using image features information fusion. Inf Fusion 51:76–99

    Article  Google Scholar 

  2. 2.

    Ashraf R, Ahmed M, Ahmad U, Habib MA, Jabbar S, Naseer K (2020) MDCBIR-MF: multimedia data for content-based image retrieval by using multiple features. Multimed Tools Appl 79(13):8553–8579

    Article  Google Scholar 

  3. 3.

    Ashraf R, Bashir K, Irtaza A, Mahmood MT (2015) Content based image retrieval using embedded neural networks with bandletized regions. Entropy 17(6):3552–3580

    Article  Google Scholar 

  4. 4.

    Backes AR (2013) A new approach to estimate lacunarity of texture images. Pattern Recogn Lett 34(13):1455–1461

    Article  Google Scholar 

  5. 5.

    Backes AR, Gonçalves W. N., Martinez AS, Bruno OM (2010) Texture analysis and classification using deterministic tourist walk. Pattern Recogn 43(3):685–694

    Article  Google Scholar 

  6. 6.

    Backes AR, Junior JJDMS (2017) LBP maps for improving fractal based texture classification. Neurocomputing 266:1–7

    Article  Google Scholar 

  7. 7.

    Chen L, Xu D, Tsang IW, Luo J (2010) Tag-based web photo retrieval improved by batch mode re-tagging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3440–3446

  8. 8.

    Chun YD, Seo SY, Kim NC (2003) Image retrieval using BDIP and BVLC moments. IEEE Trans Circ Syst Video Technol 13(9):951–957

    Article  Google Scholar 

  9. 9.

    Dubey SR, Singh S, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25 (9):4018–4032

    MathSciNet  Article  Google Scholar 

  10. 10.

    Guo Z, Zhang L, Zhang D (2010) Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recogn 43 (3):706–719

    Article  Google Scholar 

  11. 11.

    Kaya Y, Ertuğrul ÖF, Tekin R (2015) Two novel local binary pattern descriptors for texture analysis. Appl Soft Comput 34:728–735

    Article  Google Scholar 

  12. 12.

    Khokher A, Talwar R (2017) A fast and effective image retrieval scheme using color-, texture-, and shape-based histograms. Multimed Tools Appl 76 (20):21787–21809

    Article  Google Scholar 

  13. 13.

    Kumar S, Pal A (2016) A CBIR scheme using active contour and edge histogram descriptor in YCbCr color space. IJCTA 9(41):889–898

    Google Scholar 

  14. 14.

    Kwitt R, Meerwald P (2012) Salzburg texture image database. Avaiable online: http://www.wavelab.at/sources/STex

  15. 15.

    Kumar S, Pradhan J, Pal A (2017) A CBIR scheme using GLCM features in DCT domain. In: 2017 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC). IEEE, pp 1–7

  16. 16.

    Kumar S, Pradhan J, Pal A (2018) A CBIR technique based on the combination of shape and color features. In: Advanced Computational and Communication Paradigms. Springer, Singapore, pp 737–744

  17. 17.

    Kundu MK, Chowdhury M, Bulò S. R. (2015) A graph-based relevance feedback mechanism in content-based image retrieval. Knowl-Based Syst 73:254–264

    Article  Google Scholar 

  18. 18.

    Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: State of the art and challenges. ACM Trans Multimed Comput Commun Appl (TOMM) 2(1):1–19

    Article  Google Scholar 

  19. 19.

    Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002

    Article  Google Scholar 

  20. 20.

    Liu GH, Li Z, Zhang L, Xu Y (2011) Image retrieval based on micro-structure descriptor. Pattern Recogn 44(9):2123–2133

    Article  Google Scholar 

  21. 21.

    Liu GH, Yang JY (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198

    Article  Google Scholar 

  22. 22.

    Liu GH, Yang JY, Li Z (2015) Content-based image retrieval using computational visual attention model. Pattern Recogn 48(8):2554–2566

    Article  Google Scholar 

  23. 23.

    Liu M, Yang L, Liang Y (2015) A chroma texture-based method in color image retrieval. Optik- Int J Light Electron Opt 126(20):2629–2633

    Article  Google Scholar 

  24. 24.

    Liu GH, Zhang L, Hou YK, Li Z, Yang JY (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389

    Article  Google Scholar 

  25. 25.

    Murala S, Maheshwari RP, Balasubramanian R (2012) Directional local extrema patterns: a new descriptor for content based image retrieval. Int J Multimed Inf Retr 1(3):191–203

    Article  Google Scholar 

  26. 26.

    Murala S, Maheshwari RP, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886

    MathSciNet  Article  Google Scholar 

  27. 27.

    Niu D, Zhao X, Lin X, Zhang C (2020) A novel image retrieval method based on multi-features fusion. Sig Process Image Commun, pp 115911

  28. 28.

    Ojala T, Pietikäinen M., Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recognit 29(1):51–59

    Article  Google Scholar 

  29. 29.

    Pandey S, Khanna P, Yokota H (2016) A semantics and image retrieval system for hierarchical image databases. Inf Process Manag 52(4):571–591

    Article  Google Scholar 

  30. 30.

    Pradhan J, Kumar S, Pal A, Banka H (2018) A hierarchical CBIR framework using adaptive tetrolet transform and novel histograms from color and shape features. Digit Sig Process 82:258–281

    Article  Google Scholar 

  31. 31.

    Pradhan J, Kumar S, Pal A, Banka H (2019) Texture and colour region separation based image retrieval using probability annular histogram and weighted similarity matching scheme. IET Image Process 14(7):1303–1315

    Article  Google Scholar 

  32. 32.

    Pradhan J, Pal A, Banka H (2019) Principal texture direction based block level image reordering and use of color edge features for application of object based image retrieval. Multimed Tools Appl 78(2):1685–1717

    Article  Google Scholar 

  33. 33.

    Raghuwanshi G, Tyagi V (2016) Texture image retrieval using adaptive tetrolet transforms. Digit Sig Process 48:50–57

    MathSciNet  Article  Google Scholar 

  34. 34.

    Rajpurkar P, Irvin J, Bagul A, Ding D, Duan T, Mehta H, Langlotz C (2017) Mura: Large dataset for abnormality detection in musculoskeletal radiographs. arXiv preprint arXiv:1712.06957

  35. 35.

    Ryoo YJ, Kim NC (1988) Valley operator for extracting sketch features: DIP. Electron Lett 24(8):461–463

    Article  Google Scholar 

  36. 36.

    Saipullah KM, Ismail NA, Kim DH (2011) The pruning of combined neighborhood differences texture descriptor for multispectral image segmentation. Int J Comput Technol Electron Eng 1(3):1–6

    Google Scholar 

  37. 37.

    Seo SY, Lim CW, Chun YD, Kim NC (2000) Extraction of texture regions using region-based local correlation. In: Visual Communications and Image Processing 2001. International Society for Optics and Photonics, vol 87, 4310, pp 694–702

  38. 38.

    Shrivastava N, Tyagi V (2015) An efficient technique for retrieval of color images in large databases. Comput Electr Eng 46:314–327

    Article  Google Scholar 

  39. 39.

    Singh S, Batra S (2020) An efficient bi-layer content based image retrieval system. Multimed Tools Appl, vol.79 pp 1–29

  40. 40.

    Varish N, Kumar S, Pal A (2017) A novel similarity measure for content based image retrieval in discrete cosine transform domain. Fundamenta Informaticae 156(2):209–235

    MathSciNet  Article  Google Scholar 

  41. 41.

    Varish N, Pradhan J, Pal A (2017) Image retrieval based on non-uniform bins of color histogram and dual tree complex wavelet transform. Multimed Tools Appl 76(14):15885–15921

    Article  Google Scholar 

  42. 42.

    Verma M, Raman B (2018) Local neighborhood difference pattern: A new feature descriptor for natural and texture image retrieval. Multimed Tools Appl 77(10):11843–11866

    Article  Google Scholar 

  43. 43.

    Vimina ER, Divya MO (2020) Maximal multi-channel local binary pattern with colour information for CBIR. Multimed Tools Appl, vol. 79(35) pp 1–21

  44. 44.

    Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348

    Article  Google Scholar 

  45. 45.

    Wang XY, Yu YJ, Yang HY (2011) An effective image retrieval scheme using color, texture and shape features. Comput Stand Interfaces 33(1):59–68

    Article  Google Scholar 

  46. 46.

    Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by Gaussian mixture models. Neurocomputing 171:673–684

    Article  Google Scholar 

  47. 47.

    Zhang J, Marszałek M., Lazebnik S, Schmid C (2007) Local features and kernels for classification of texture and object categories: A comprehensive study. Int J Comp Vision 73(2):213–238

    Article  Google Scholar 

Download references

Acknowledgements

The author Mr. Sumit Kumar (Admission No: 2015DR0056) is supported by the institute Ph.D. scholarship, IIT[ISM] Dhanbad, Jharkhand, India. The author Prof. Muhammad Khurram Khan acknowledges that his work is supported by Researchers Supporting Project number (RSP-2020/12), King Saud University, Riyadh, Saudi Arabia.

Author information

Affiliations

Authors

Corresponding author

Correspondence to SK Hafizul Islam.

Ethics declarations

Conflict of interests

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kumar, S., Pradhan, J., Pal, A.K. et al. Radiological image retrieval technique using multi-resolution texture and shape features. Multimed Tools Appl (2021). https://doi.org/10.1007/s11042-021-10525-8

Download citation

Keywords

  • Modified block difference of inverse probability (BDIP)
  • Block level variance of local variance (BVLC)
  • Content based image retrieval (CBIR)
  • Radiological image retrieval
  • Multi-resolution texture and shape features