Keywords

1 Introduction

When processing crushed ore mass at ore mining and processing enterprises, one of the main indicators of the quality of work of both equipment and personnel is the assessment of the size of the crushed material at each stage of the technological process. This is due to the need to reduce material and energy costs for the production of a product unit manufactured by the plant: concentrate, sinter or pellets.

The traditional approach to the problem of evaluating the size of crushed material is manual sampling with subsequent sieving with sieves of various sizes. The determination of the grain-size distribution of the crushed material in this way entails a number of negative factors:

  • the complexity of the measurement process;

  • the inability to conduct objective measurements with sufficient frequency;

  • the human error factor at the stages of both data collection and processing.

These shortcomings do not allow you to quickly adjust the performance of crushing equipment. The need for obtaining data on the coarseness of crushed material in real time necessitated the creation of devices for in situ assessment of parameters such as the grain-size distribution of ore particles, weight-average ore particle and the percentage of the targeted class. The machine vision systems are able to provide such functionality. They have high reliability, performance and accuracy in determining the geometric dimensions of ore particles. At the moment, several vision systems have been developed and implemented for the operational control of the particle size distribution of crushed or granular material. In [9], a brief description and comparative analysis of such systems as: SPLIT, WIPFRAG, FRAGSCAN, CIAS, IPACS, TUCIPS is given.

Common to the algorithmic part of these systems is the stage of dividing the entire image of the crushed ore mass into fragments corresponding to individual particles with the subsequent determination of their geometric sizes. Such a segmentation procedure can be solved by different approaches, one of which is to highlight the boundaries between fragments of images of ore particles. Classical methods for borders highlighting based on the assessment of changes in brightness of neighboring pixels, which implies the use of mathematical algorithms based on differentiation [4, 8]. Figure 1 shows typical images of crushed ore moving on a conveyor belt.

Fig. 1.
figure 1

Examples of crushed ore images

Full size image

Algorithms from the OpenCV library, the Sobel and Canny filters in particular, used to detect borders on the presented images, have identified many false boundaries and cannot be used in practice.

This paper presents the results of recognizing the boundaries of images of stones based on a neural network. This approach has been less studied and described in the literature, however, it has recently acquired great significance in connection with its versatility and continues to actively develop with the increasing of a hardware performance [3, 5].

2 Main Part

To build a neural network and apply machine learning methods, a sample of images of crushed ore stones in gray scale was formed. The recognition of the boundaries of the ore particles must be performed for stones of arbitrary size and configuration on a video frame with ratio 768 \(\times \) 576 pixels.

To solve this problem with the help of neural networks, it is necessary to determine what type of neural network to use, what will be the input information and what result we want to get as the output of the neural network processing. Analysis of literary sources showed that convolutional neural networks are the most promising when processing images [3, 5,6,7].

Convolutional neural network is a special architecture of artificial neural networks aimed at efficient pattern recognition. This architecture manages to recognize objects in images much more accurately, since, unlike the multilayer perceptron, two-dimensional image topology is considered. At the same time, convolutional networks are resistant to small displacements, zooming, and rotation of objects in the input images. It is this type of neural network that will be used in constructing a model for recognizing boundary points of fragments of stone images.

Algorithms for extracting the boundaries of regions as source data use image regions having sizes of 3 \(\times \) 3 or 5 \(\times \) 5. If the algorithm provides for integration operations, then the window size increases. An analysis of the subject area for which this neural network is designed (a cascade of secondary and fine ore crushing) showed: for images of 768 \(\times \) 576 pixels and visible images of ore pieces, it is preferable to analyze fragments with dimensions of 50 \(\times \) 50 pixels.

Thus, the input data for constructing the boundaries of stones areas will be an array of images consisting of (768 − 50)*(576 − 50) = 377668 halftone fragments measuring 50 \(\times \) 50 pixels. In each of these fragments, the central point either belongs to the boundary of the regions or not. Based on this assumption, all images can be divided into two classes.

To mark the images into classes on the source images, the borders of the stones were drawn using a red line with a width of 5 pixels. This procedure was performed manually with the Microsoft Paint program. An example of the original and marked image is shown in Fig. 2.

Fig. 2.
figure 2

Image processing (Color figure online)

Full size image

Then Python script was a projected, which processed the original image to highlight 50 \(\times \) 50 pixels fragments and based on the markup image sorted fragments into classes preserving them in different directories To write the scripts, we used the Python 3 programming language and the Jupyter Notebook IDE. Thus, two data samples were obtained: training dataset and test dataset for the assessment of the network accuracy.

As noted above, the architecture of the neural network was built on a convolutional principle. The structure of the basic network architecture is shown in Fig. 3 [7].

Fig. 3.
figure 3

Basic convolution network architecture

Full size image

The network includes an input layer in the format of the tensor 50 \(\times \) 50 \(\times \) 1. The following are several convolutional and pooling layers. After that, the network unfolds in one fully connected layer, the outputs of which converge into one neuron, to which the activation function, the sigmoid, will be applied. At the output, we obtain the probability that the center point of the input fragment belongs to the “boundary point” class.

The Keras open source library was used to develop and train a convolutional neural network [1, 2, 6, 10].

The basic convolutional neural network was trained with the following parameters:

  • 10 epoch;

  • error - binary cross-entropy;

  • quality metric - accuracy (percentage of correct answers);

  • optimization algorithm - RMSprop.

The accuracy on the reference data set provided by the base model is 90.8%. In order to improve the accuracy of predictions, a script was written that trains models on several configurations, and also checks the quality of the model on a test dataset.

To improve the accuracy of the predictions of the convolutional neural network, the following parameters were varied with respect to the base model:

  • increasing the number of layers: +1 convolutional +1 pooling;

  • increasing of the number of filters: +32 in each layer;

  • increasing the size of the filter up to 5*5;

  • increasing the number of epochs up to 30;

  • decreasing in the number of layers.

These modifications of the base convolutional neural network did not lead to an improvement in its performance - all models had the worst quality on the test sample (in the region of 88–90% accuracy).

The model of the convolutional neural network, which showed the best quality, was the base model. Its quality in the training sample is estimated at 90.8%, and in the test sample - at 83%. None of the other models were able to surpass this figure. Data on accuracy and epoch error are shown in Fig. 4 and 5.

If you continue to study for more than 10 epochs, then the effect of retraining occurs: the error drops, and accuracy increases only on training samples, but not on test ones.

Fig. 4.
figure 4

The dependence of the accuracy on the training and test datasets from the training epochs

Full size image
Fig. 5.
figure 5

The dependence of the error on the training and test datasets from the training epochs

Full size image
Fig. 6.
figure 6

Images of crushed ore particles with boundaries detected by a neural network

Full size image

Figure 6 shows examples of images with neural network boundaries. As you can see from the images, not all the borders are closed. The boundary discontinuities are too large to be closed using morphological operations on binary masks; however, the use of the “watershed” algorithm [8] will reduce the identification error of the boundary points.

3 Conclusion

In this work, a convolutional neural network was developed and tested to recognize boundaries on images of crushed ore stones. For the task of constructing a convolutional neural network model, two data samples were generated: training and test dataset. When building the model, the basic version of the convolutional neural network structure was implemented. In order to improve the quality of model recognition, a configuration of various models was devised with deviations from the basic architecture. An algorithm for training and searching for the best model by enumerating configurations was implemented.

In the course of the research, it was found that the basic model has the best quality for recognizing boundary points. It shows the accuracy of the predictions for the targeted class at 83%.

Based on the drawn borders on the test images, it can be concluded that the convolutional neural network is able to correctly identify the boundary points with a high probability. It rarely makes mistakes for cases when there is no boundary (false positive), but often makes mistakes when recognizing real boundary points (false negative). The boundary breaks are too large to be closed using morphological operations on binary masks, however, the use of the “watershed” algorithm will reduce the identification error for boundary points.