Predictive Maintenance in a Digital Factory Shop-Floor: Data Mining on Historical and Operational Data Coming from Manufacturers’ Information Systems

Abstract

Predictive maintenance is regarded by many as a key factor in Industrial Internet of Things (IIoT) and the development of “smart” factories. With the growing use of sensors and embedded computing systems, the term predictive maintenance is most often understood as a strategy that relies on collecting streaming sensor data and performing condition monitoring. Thus, the majority of academic papers base their research work solely on sensorial sources coming from the shop floor machinery, neglecting the knowledge already existing in legacy systems and maintenance historical logs. The UPTIME project aims to develop a unified predictive maintenance framework that incorporates information from heterogeneous data sources, both from sensor devices and legacy/operational systems. In this contribution, we share our first insights on legacy data analytics in the predictive maintenance context, and outline the tools and approaches we developed in the course of the project. Experimental work has been conducted using real world datasets deriving from an actual manufacturing facility in the White Goods/Home Appliances sector. The results provide significant knowledge about the manufacturing processes and show the potential of the proposed methodology.

1 Introduction

Predictive maintenance, in general, describes strategies and actions to prevent breakdowns of technical equipment by predicting just in-time wear-out or failure situations. Such strategies usually imply the continuous processing of sensor data in order to derive information on possible abnormalities and prospective failures, which in turn, can lead to predictions on future states, trends and remaining useful life (RUL) of the observed equipment. In simple terms, the first objective of a predictive maintenance strategy is to identify abnormal behaviour of technical equipment by continuously observing sensor data streams. The second objective is to analyse the data streams using prediction algorithms, already trained with historical sensor data, in order to provide a solid guess of a future undesired condition or failure. The latter is accompanied by an obvious challenge: such an analysis would require a large amount of sensor data that already have experienced failures of -ideally- all possible causes. And that, in the real world, could take many years to acquire and would require an enormous storage capacity. Additionally, in most cases, the sensor data involve only a few of the components of a production line, or a small number of aspects of each component (e.g. only temperature, or pressure), which means that a prediction system based on these sensors can rarely capture the whole picture of the factory shop floor and the possible correlations among all machines.

Legacy systems, on the other hand, refer to operational information already collected in databases that cover many aspects of the shop floor and manufacturing activities. Legacy datasets may contain historical values related to the daily production line operation (e.g. products created per day, wastes created per day, pause times of production line, etc.), or to maintenance events, failures and causalities, in the form of a log file (e.g. On date d, machine x stopped for n mins in order to be sufficiently lubricated). They can even include shift schedules, environmental conditions or logistics. All these factors indicate the need for intelligent and automated data analysis methodologies, which aim to discover useful knowledge from data. Data mining has emerged as a prominent tool for knowledge acquisition from the manufacturing databases with many examples in the academic research area [1, 2]. The knowledge extracted can come in various forms (predictions, optimized values, rules, etc.) and may be propagated to or/and combined with condition monitoring systems.

The combination of legacy systems’ extracted knowledge with real-time data streams has been investigated in the past in few research studies [3,4,5]. The main outcome of all these studies is more or less identical; the combined power of data analytics can lead to more accurate predictive maintenance results and can cover a broader spectrum of manufacturing processes and operations. If we take into consideration that these approaches are over a decade old, and that machine learning has evolved significantly since then, we are confident that a data mining approach based on today’s tools (software and hardware) can certainly advance predictive maintenance a step forward.

The next section outlines the methodology and architecture proposed in this paper, while Sect. 3 presents a real-world case study from the manufacturing domain of white goods. Section 4 is devoted to experimental results and Sect. 5 concludes the paper with useful insights and future challenges.

2 Methodology and Architecture

The UPTIME project introduces methodological and technological innovations to address the challenges of a predictive maintenance unified system. One of those innovations is a Data Analytics engine driven by the manufacturers’ need to leverage legacy and operational data related to the broader overview of the shop floor performance and maintenance, as well as to extract and correlate relevant knowledge. Variables in manufacturing databases that would be useful for analysis, could be classified into the following groups [6]:

• 1. Manufacturing process variables: machining, casting, forgings, extrusions, stampings, assembling, cleaning, etc.

• 2. Machining variables: cutting speeds, temperature, pressure, lubricants, coolants, voltage, current, etc.

• 3. Resource variables: product materials and numbers, machines, fixtures, etc.

• 4. Environment variables: humidity, temperature, etc.

• 5. Working condition variables: duration, shift, injuries and accidents, etc.

• 6. Target variables: quality, yield, productivity, performance index, etc.

• 7. Maintenance events variables: duration of downtime, duration of maintenance, reasons, causes, solutions, etc.

In the case study that follows, we use variables of the last 2 groups, but any combinations are possible depending on the availability and type of the data collected.

The proposed Data Analytics system allows a manufacturer to upload different datasets that have been extracted from the legacy and operational systems. In order to ensure that the datasets provided fall within the scope of predictive maintenance, they are mapped to a pre-defined predictive maintenance Data Model which is based on the MIMOSA international standard (OSA-CBM v3.3.1 and OSA-EAI v3.2.3a)¹. In case a dataset has no correlation with the data model, it cannot be further processed; otherwise, its semantic and syntactic mapping is performed and the data are stored. From this phase on, two parallel processes start:

• I. The data analyst has at his disposal the datasets and may experiment with the different machine learning algorithms already provided in the Data Analytics engine in the dedicated data analyst interface. Such an interface is built on a popular open-source notebook and allows for further customization of the features of the algorithms and comparison of their performance (accuracy, variance, etc.).

• II. Through the business interface, the business user can obtain a quick understanding of the data uplifted, including: (a) the distribution of the values across the features/fields of the dataset to detect missing or unexpected values, min and max values, as well as stats like mean, median, and standard deviation, (b) the relationships detected within the data provided through interactive exploration of different data facets across multiple dimensions, and (c) the outcomes of the more “statistical” analysis that is automatically computed, e.g. timelines of the interruptions vis-à-vis the failures, the actual versus the planned downtime per day, and the different types of interruption per machine.

As soon as the analysis performed on any dataset is completed, it is appropriately exported from the data analyst interface and imported into the business user interface. In this way, the business users have the outcomes of the machine learning analysis at their disposal and may also browse and examine the different views and reports in an intuitive manner using their own interface.

The high-level architecture and the workflow of this component can be viewed in Fig. 1. The data analytics engine practically consists of five layers:

Fig. 1.

High level architecture

• 1. The Data Uplifting Layer which handles the dataset upload process in the component, by receiving batch data extracts from legacy software platforms of earlier technology, as well as from operational systems on the shop floor with events collected by the workers Due to the nature of the data analysis to be performed, the uplifting may take place once a week or once per month or even once per year, so that we have sufficient data to work on.

• 2. The Data Curation, Matching and Transformation Layer which is responsible for mapping the data model of the dataset to the pre-defined predictive maintenance data model.

• 3. The Data Storage Layer that handles the storage of the data contained in one or more given datasets.

• 4. The Data Mining and Analytics Layer practically delivers the intelligence of the component by defining, training, executing and experimenting with different machine learning algorithms.

• 5. The Analytics Results Visualization, Patterns & Rules Extraction acts as the user interface to the business user, visualizing and interpreting the results of the Data Mining & Analytics Layer and extracting rules and patterns that are exposed (together with specific data extracts and results) through APIs to the integrated UPTIME platform.

The functionality of the proposed data analytics engine is evaluated over a real business case from the manufacturing domain of white goods and home appliances and is presented, along with the experimental results, in the following sections.

3 White Goods Case Study

The presented research study is supported by an industrial partner, that produces white goods and home appliances worldwide. It is a preliminary study that focuses on a single production line that produces drums for dryers. The product is basically a carbon steel cylinder used to keep and rotate clothes during drying stage. Currently only preventive and reactive maintenance are implemented in the shop floor and thus there is a requirement to expand the list of maintenance activities performed by effectively modifying their strategy to include predictive maintenance processes. In the UPTIME context, a set of sensors will be installed on important assets of the shop floor and data analysis will be performed covering two different viewpoints: data coming from sensors and data from legacy/operational systems.

This paper presents the second viewpoint’s preliminary analysis. Along with useful insights, the results of this process can be used as an initialisation procedure for the first viewpoint, until sufficient amount of sensorial data has been collected and analysed. For this purpose, the industrial partner provided us with two different datasets with data collected through a whole year. They are both pretty common in manufacturing processes and can be extracted from typical software programs found in factory premises. The first dataset (OEE dataset) is related to the production line performance indicators and the way the OEE (Overall Equipment Effectiveness) is computed on a daily basis. Therefore, the most useful information in this dataset includes the number of items produced, the true operational time, the time of interruptions and pauses, the number and duration of breakdown events, etc. as depicted in Table 1.

Table 1. Description of main attributes from OEE dataset

The second dataset (M-log dataset) is a log file of maintenance events that occurred during a full year, and consists of brief textual descriptions of observed faults and interruptions per day and per asset, and the actions taken to correct these faults. Each instance in this dataset therefore, represents a single fault on a single machine at a particular date. It is worth noting that a single machine may have multiple interruptions during the same day, and a single fault, usually a breakdown, may persist for more than one day (Table 2).

Table 2. Description of main attributes from M-log dataset

These datasets can be analysed separately, but also as a joint set using the Day feature as a common key. The analysis presented in this paper is performed on each dataset separately. In the lines to follow, we first explore the dataset using descriptive analytics before applying a set of classification algorithms to make predictions about future failures.

3.1 Descriptive Analytics

Descriptive analytics is an initial stage of data processing in an attempt to visualize the available information, extract some initial useful insights and prepare the data for further analysis. To this end, several statistical approaches can be employed. Nevertheless, this paper will focus on data mining techniques for descriptive results with Self-Organising Maps being our first choice.

A self-organising map (SOM) is a tool for the analysis and visualization of high-dimensional data. It is based on the principles of vector quantisation and belongs to a set of unsupervised classifiers trained by competitive learning [6]. SOMs’ ability of grouping patterns based on similarity make them ideal first technique for the descriptive analysis of complex datasets and for this reason it has been used extensively in industrial processes [8].

The SOM algorithm was applied to OEE dataset and the resulted visualization of the dataspace is depicted in Fig. 2. In simple terms, the blue squares show nodes of data with great similarity, while the reddish squares have nodes of data with much less similarity. Adjacent squares of approximately the same colour, can form larger groups which we call clusters. Therefore, one can observe clusters of different behaviours among the whole data space by examining the features of similarity that form each cluster. For example, in Fig. 2, the cluster on the lower right corner is characterised by the following feature states:

Fig. 2.

The resulted U-matrix after training a SOM with OEE data

After closer observation and consulting with the manufacturer, it resulted that this cluster describes days that the factory experiences breakdown events.

In the same way, we use a SOM on the M-log dataset, this time using only 2 variables (Machine Id and Cause of Interruption). In order to examine the relationships that form between interruptions and manufacturing equipment through SOM the categorical values were transformed into binary using one hot encoder. The result is depicted in Fig. 3 and an example of the most interesting cluster is the one with the following characteristics:

Fig. 3.

The resulted U-matrix after training a SOM with M-log data

The above output tells us that the SOM algorithm detected a group of machines and causes which produce interruptions frequently occurring together. This is an interesting insight since in a complex shop floor environment with so many machines and different interruption types, is not easy for a human being to observe such patterns and correlations.

3.2 Classification Algorithms

The next step of the analysis involved the testing of the prediction power of the algorithms. The request was to predict if the factory is going to have a breakdown on the next day, given today’s performance indicators. For this reason, we employed the OEE dataset, which gives us a daily overview of the production line. As a first action, we setup the dataset as a binary classification task. For this reason, we had to create labels according to whether there is a breakdown event on the next day or not, based on the “Number of breakdowns” feature input. Thus, the first class denotes a normal day without unexpected incidents, and the second class represents a day with at least one type of failure.

For the classification task we employed and tested 6 different classifiers, namely the linear Support Vector Machine (SVM), the multinomial Naïve Bayes classifier, the k-NN, Decision Trees, Random Forest, and the Multilayer Perceptron (MLP). These algorithms were selected due to their popularity and efficiency in similar predictive tasks [9, 10]. Even though different parameters were tested during the first runs, the accuracy of all classifiers did not manage to exceed the “normal day” class ratio using a k-fold cross validation evaluation. This means that the classifiers failed to capture the task at hand and find sufficient patterns within the given data space, resulting in an overfitting output that favored the major (in quantity) class. Overfitting is a common challenge in classification problems and there are a few techniques that allow an analyst to overcome it. Nevertheless, in this particular case, none of the known techniques could efficiently improve the results, so we had to come up with something different. This is described in the next section.

3.3 Improving Model Accuracy

By reviewing the data at hand, it was evident that some time-related information regarding the breakdown events was missing. This is why we introduced a metadata feature that we called “Days from previous breakdown event”, which could be easily calculated by the given OEE dataset. As the experimental results prove, this feature alone gave a boost to results by increasing the accuracy from 77.7% to an impressive 95%. In fact, further feature evaluation analysis using permutation importance showed the same thing. That this feature alone carries the most important information regarding breakdowns.

Permutation importance works by randomly shuffling a single column of the dataset, leaving the target and all the other columns in place, and calculates how much the accuracy is affected by this [11]. Model accuracy is of course reduced by this method, but the aim is to see how much the loss function suffered from shuffling. This performance deterioration measures the importance of the variable that was just shuffled. In our case the results on the input features are listed in Table 3.

Table 3. Permutation importance (top-6) on input features.

It is clear that the new metadata input plays the most important role in the classification outcome. The reasons why are explained in the outcome of the experimental analysis that follows.

4 Experimental Results

As already described, the experimental analysis involved 6 different classifiers, namely the linear Support Vector Machine (SVM), the multinomial Naïve Bayes classifier, k-NN, Decision Tree, Random Forest, and the Multilayer Perceptron (MLP). The input space included all features from the OEE dataset and the algorithms were evaluated using a stratified 5-fold cross validation on a shuffled dataset.

Using the newly introduced “Days from previous breakdown event” feature, the Naïve Bayes, SVM, and k-NN failed to generalise and reached a performance of 77.66%, which is precisely the rate of the first class. The MLP raised its performance slightly reaching an 80%. In contrast, the decision tree reached an accuracy of 95%, followed by random forest with 93.4%. The following table lists the results for both data scenarios examined; with and without the new metadata feature (Table 4).

Table 4. Classification results after 5-fold cross validation.

It is worth mentioning that the interpretation of the decision tree outcome offers additional information in the form of rule-based knowledge, as shown below. Note that the outcome is actually a single rule of an IF-ELSE IF format:

Rule

In short, the above rule claims that a day with failure is usually followed by a next day of failure, which implies either that a given malfunction may create another malfunction on the same production line or that a failure usually persists for more than 1 days. Both of these assumptions were empirically confirmed as true by the manufacturer. We should also note that, given more years of data, the predictions could be enhanced to support a greater time horizon, but still on a scale of days.

5 Conclusions and Future Work

As discussed in a very recent study [12], 97% of the manufacturing companies in Germany and Switzerland are planning to extend their activities in data analytics. Nevertheless, the manufacturing domain is still at the early stages of exploiting all available data it creates and stores, especially for predictive maintenance purposes. The analysis of real-time sensor data has been the main focus lately in the majority of the research papers, while there is a huge treasure of legacy systems data remaining untouched. Additionally, the sensor data analysis is usually applied only on a small part of a shop floor, monitoring the condition of a few machines. Legacy data on the other hand, contain information regarding the whole factory cycle and store events from all machines, even if they have sensors installed or not. And this is a gap that the UPTIME project and the proposed methodology aims to bridge, with the power of data mining technics and machine learning.

A data analytics engine is presented that allows a manufacturer to upload, examine and analyze different datasets extracted from legacy and operational systems. The datasets are required to follow a specific data model especially designed for predictive maintenance purposes. An industrial case study highlights the importance of the derived knowledge, which, in many cases, can assist and calibrate the information derived from sensorial real-time data streams. Examples of descriptive and predictive analytics are discussed, emphasizing on the experimental results on two datasets coming from a real operating factory.

One of the main practical challenges identified during this study was the fact that on an operational shop floor that already performs preventive maintenance on a fixed schedule, is very hard to obtain frequent failures or breakdowns in the data. Even more so, when a factory has newly installed equipment. Moreover, a dataset with a year’s details cannot be regarded as sufficient and representative of the overall factory and machinery lifecycle and it can, therefore, reveal the potential effectiveness of the method.

The next steps of this on-going research, apart from tackling the aforementioned challenges, include the application of data analytics on a larger set of historical data from factory operations, joined by historical data generated by the installed sensors, so that the interoperability and knowledge exchange of the two data processing systems will be tested. At the same time, different factories from the manufacturing industry will be included as case studies in order for us to obtain a more generic approach that can reduce maintenance costs and improve quality, productivity and profitability for any manufacturing organisation.