1 Introduction

Image repositories contain well annotated Computer Aided Design (CAD) shape images [1, 2], scanned object images [3], scene images [4] among others. Existing image repositories are mostly generic as not much attention was paid to relevant images peculiar to specific fields and domain. These repositories do not contain sufficient data about each domain due to their generic properties. We however intend to focus on developing a domain-based repository taking the education as our domain. We develop an open source shape image repository LabNet; consisting of laboratory apparatus shapes which can be easily accessible to students and teachers for easy identification, electronic learning and retrieval purposes. It can also be adopted as database platform for electronic, virtual and simulated science laboratories. This will meet the needs of those who cannot participate physically in the laboratory exercises.

2 Related Work

There has recently been a shift from text data to image data. This is definitely because an image easily attracts captivates its viewer’s attention and mind. This in turn makes it easier for information to be passed across and assimilated by its viewers much faster than text documents [5]. There is however the need to store related and similar images together for easy accessibility and future retrieval. Image repositories contain large related image datasets. Much work has been done using image repositories and databases. [6] designed MORPH; a longitudinal face database containing human faces useful for face recognition and modelling. [7]’s work shows a color face image database for benchmarking automatic face detection algorithms. ImageNet [8] also developed a large scale ontology of images useful for visual recognition applications. [9]’s work highlighted an image database containing 3D scenes useful for training of object detection systems. [1] used Convolutional Deep Belief Network to developed ModelNet which contains a large scale 3D shapes of Computer Aided Design dataset. [10] provide a novel face database containing face images which were gotten in real-world conditions. They are useful for improvement of face recognition approaches. [11] designed ‘Places’ which is a database of scene photographs having scene semantic categories and attributes using Convolutional Neural Networks which could be used for scene recognition and analysis.

Specifically focusing on shape object images; [12] developed a shape database of polygonal models which provides multiple semantic labels for each 3D model. [1] also developed a shape repository that contains 3D shapes having different semantic annotations for each 3D model. Although the image data content is very largely specifically more than 3,000,000 shapes [1], they don’t provide some domain specific shapes or objects for relevant field of interest. This is why domain specific image repositories have been designed for proper image processing such as face recognition repositories [7, 10], medical images repositories [13, 14] and floricultural object image purposes [15,16,17].

It can be noted that despite the fact that existing image repositories have large image datasets, their method of dataset collection varies. Some repositories such as [18,19,20] retrieved 3D models from existing Scenes; generate 3D models from existing 2D Objects [21]. Others obtain camera photographs from real life environments, use segmentation and extraction algorithms on the photographs to populate their databases [3, 10, 17]. The use of sensor tools to automatically sense objects in any given scene and image is also in literature [22]. Once the identification is done, the shapes and objects can be extracted through some object extraction techniques. Another procedure involves crawling of related images, conducting edge detection and segmentation to extract objects from images [23]. A brief critical review of image repositories in literature is shown in Table 1.

Table 1. Comparative literature review of existing model/shape/image repositories
Full size table

There are several edge detection algorithm such as Prewitt, Robert, Sorbel and Canny algorithms among others [24]. We however choose Canny algorithm because it performs better detection and localization than other edge detection techniques [25,26,27,28].

Image segmentation techniques also exist in literature such as region-based, feature-based, segmentation based on weakly-supervised learning in CNN [29, 30] and morphological-based techniques [31, 32]. Morphological-based segmentation has been widely used [33]. [34] used morphological image processing for weed recognition in weed and plant images. [35] applied fuzzy-canny and morphological techniques for edge detection and segmentation to some selected images for better output. We therefore present LabNet repository using canny algorithm for the detection of edges in images and morphological image processing technique for image segmentation.

3 LabNet Repository

LabNet consist of apparatus shape images categorized under each subject (physics, chemistry, biology). We categorize the images under specific exercises for each subject which enables quick search query and retrieval. Figure 1 shows a pictorial representation of how LabNet works.

Fig. 1.
figure 1

LabNet design methodology

Full size image
Fig. 2.
figure 2

A taxonomy showing the classes, sub-classes and categories under LabNet

Full size image
Fig. 3.
figure 3

A visualization of the classes, sub-classes, categories and entities (apparatus names)

Full size image
Fig. 4.
figure 4

User search query “cylinder” and search query result images

Full size image

3.1 Problem Formulation

We define the apparatus shape retrieval problem as follows: given a user query as apparatus name in text as input, the output is n apparatus shapes where n = 4. Each apparatus shape is a pictorial representation of the apparatus text input by the user, we make the following statements

  • Let a represent the apparatus shape.

  • Let \( \{ {\text{H}}^{\text{b}}_{\text{n}} \} \) be a set of practical exercises under a subject category and

    $$ H = \left\{ {{\text{a}}_{1} ,{\text{ a}}_{2} ,{\text{ a}}_{3} ,{\text{ a}}_{4} } \right\} $$
    (1)

    Where a1, a2, a3 and a4 represent the practical exercises which have 4 apparatus shapes, n is the finite number of practical exercises under each subject.

  • Let the sets B, C and P represent biology, chemistry and physics subjects respectively. b, c, p are used to indicate an exercise belongs to biology, chemistry or physics respectively. Then,

    $$ {\text{B}} = \left[ {\left\{ {{\text{H}}^{\text{b}}_{1} } \right\}, \, \left\{ {{\text{H}}^{\text{b}}_{2} } \right\}, \, \left\{ {{\text{H}}^{\text{b}}_{3} } \right\} \ldots , \, \{ {\text{H}}^{\text{b}}_{\text{n}} \} } \right] $$
    (2)
    $$ {\text{C}} = \left[ {\left\{ {{\text{H}}^{\text{c}}_{1} } \right\}, \, \left\{ {{\text{H}}^{\text{c}}_{2} } \right\}, \, \left\{ {{\text{H}}^{\text{c}}_{3} } \right\} \ldots , \, \{ {\text{H}}^{\text{c}}_{\text{n}} \} } \right] $$
    (3)
    $$ {\text{P}} = \left[ {\left\{ {{\text{H}}^{\text{p}}_{1} } \right\}, \, \left\{ {{\text{H}}^{\text{p}}_{2} } \right\}, \, \left\{ {{\text{H}}^{\text{p}}_{3} } \right\} \ldots , \, \{ {\text{H}}^{\text{p}}_{\text{n}} \} } \right] $$
    (4)

3.2 Dataset Gathering and Update

The raw images of the repository was gotten from several web-crawling from existing images on online repositories including science-based databases. A crawl is performed and the best 10 related images are shown to the user based on a web scraping technique: Beautiful Soup which is a python library for scraping urls and websites from the internet. Manual selection of related images is then done by specified users. Upon selection of an image, object detection and segmentation techniques are performed on the image. The extracted object is saved in the image base (Fig. 5a, b, c).

Fig. 5.
figure 5figure 5

(a) LabNet ‘search online’, (b) User selection of best image (s), (c) Search query of the same keyword and segmented object as output

Full size image

3.3 Edge Detection and Segmentation

Once an image is selected, edge detection is performed on the image to determine the edges of the detected object. Canny algorithm is used for edge detection; while morphological operations will be used for object segmentation.

3.3.1 Canny Edge Detection Algorithm

This is an optimized auto thresholding algorithm which uses localized values to determine a threshold and create a single pixel thick edge around a given image. Given an image f(x, y) as coordinates; the steps given below were used on each image

  1. 1.

    Gaussian kernel filter was generated using Gaussian function G(x, y)

    $$ G_{{\left( {x,y} \right)}} = \frac{1}{{2\pi \sigma^{2} }}\exp \left[ { - \frac{{x^{2} + y^{2} }}{{2\sigma^{2} }}} \right] $$
    (5)

    Where, ‘σ’ is the standard deviation of Gaussian function

  2. 2.

    The generated Gaussian kernel was used to blur the image and to filter noise. Smooth images \( S_{{\left( {x,y} \right)}} \) were obtained using the convolution of the original image \( f_{{\left( {x,y} \right)}} \)

    $$ S_{{\left( {x,y} \right)}} = G_{{\left( {x,y} \right)}} * f_{{\left( {x,y} \right)}} $$
    (6)
  3. 3.

    Next, the image gradients \( M_{{\left( {x,y} \right)}} \) and direction \( D_{{\left( {x,y} \right)}} \) were determined by:

    $$ M_{{\left( {x,y} \right)}} = \sqrt {b_{x}^{2} \left( {x,y} \right) + b_{y }^{2} \left( {x,y} \right)} $$
    (7)
    $$ D_{{\left( {x,y} \right)}} = \tan^{ - 1} \left[ { b_{x} \left( {x,y} \right)*b_{y} \left( {x,y} \right)} \right] $$
    (8)

    Where bx and by are the results of the filter effects \( f_{\left( x \right)} \) and \( f_{\left( y \right)} \) on the image \( S_{{\left( {x,y} \right)}} \) along the row and column respectively.

  4. 4.

    Hysteresis thresholding to eliminate breaking up of edge contours [36]

  5. 5.

    Final edges are determined by suppressing all edges that are not connected to a very certain (strong) edge.

3.3.2 Morphological Image Segmentation

To perform any morphological operation, a given image is converted to gray scale and then to binary image. A morphological filter/mask which is a small binary array is also created. Although, there are four major operations carried out in morphological image segmentation which are dilation, erosion, opening and closing [37]. Segmentation will be done using dilation and erosion algorithms as these two operations are sufficient to effectively extract background from foreground. Dilation operation is done by systematically moving the filter over the input image to make the image bigger in size. Mathematically, Given A to be input image and B is the filter;

$$ A \oplus B = \left\{ {\left. x \right| \left( {\hat{B}} \right)_{x} \;{\bigcap }\;A \ne \phi } \right\} $$
(9)

Erosion basically erodes away the boundaries of the foreground. Mathematically, dilation is given by:

$$ A\,{ \ominus }\,B = \left\{ {\left. x \right| \left( B \right)_{x} \; {\bigcap }\;A^{c} \ne \phi } \right\} $$
(10)

3.4 Shape Annotation and Classification

Annotation of shape was done using initial search query from the user. Selected images are annotated using the search query of users. The repository is classified under strict categories. Although there are several science subjects, we restrict our database to only physics, chemistry and biology subjects. This is because these subjects are the major core and fundamental subjects in high school basic science. These subjects have laboratory practical schedule as part of their course curriculum. LabNet repository has 3 major classes, 20 subclasses and 82 categories under the subclasses. Figure 2 shows the classes, subclasses and subcategories. Figure 3 shows the visualized hierarchy of the classes, subclasses, subcategories and entities of each subcategory.

4 Results

The search technique is keyword-based. The user is expected to type a search query apparatus and select a subject category. Extracted keywords are matched based on annotated objects in the database. The output is shown in Fig. 4. The output images contain four related but different images of the same search query. This is to enable user selection based on choice.

5 Conclusion

LabNet can be useful for further image processing tasks such as text to scene or picture conversion processes especially for science-oriented domains. It can also be utilized as a teaching aid for passing laboratory apparatus based information in elementary science to primary school students especially. LabNet repository is a very important tool that can as well be incorporated for science-oriented institutional repositories as a means for electronic learning. Our method of addition of new apparatus shapes provides flexibility and the ease of access and use by users. We hope that LabNet will increase and grow into a very large dataset for wider scientific-oriented research.