Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

A time series is a set of observations xt, each one being recorded at a specific time t [3]. Current time series forecasting methods generally fall into two groups: methods based on statistical concepts and computational intelligence techniques. Computational intelligence techniques for time series forecasting generally fall into two major categories: first, the methods based on neural networks (NNs); and second, the methods based on evolutionary computation. The second category is divided into methods based on genetic algorithm (GA), evolutionary programming (EP), and genetic programming (GP). All these methods are inspired by the study of biological processes [22].

Hybrid intelligent systems are computational systems that integrate different intelligent techniques. These systems are now being used to support complex problem solving and decision making in a wide variety of tasks. This kind of systems allows the representation and manipulation of different types and forms of data and knowledge which may come from various sources [28]. There are various works where this kind of systems have been proposed and they combine different techniques and good results have been obtained, such as in [4, 12, 15, 20], where techniques as modular neural networks (MNNs), fuzzy logic (FL) or GA has been integrated among them. In this work, two intelligence techniques are combined; modular neural networks and type-1 fuzzy logic. These techniques are used because in other works have demonstrated to be a good combination of techniques, such as in [13, 14, 19].

This paper is organized as follows: Sect. 2 contains the basic concepts used in this research work, Sect. 3 contains the proposed method, Sect. 4 presents experimental results and in Sect. 5, the conclusions of this work are presented.

2 Basic Concepts

In this section, a brief overview of the basic concepts used in this research work is presented.

2.1 Times Series

A time series is a sequential set of data points, measured typically over successive times. A time series in general is supposed to be affected by four main components, which can be separated from the observed data. These components are: Trend, Cyclical, Seasonal and Irregular components. Time series observations are frequently encountered in many domains such as business, economics, industry, engineering and science, etc. [21, 26, 27]. Depending on the nature of analysis and practical need, there can be various different kinds of time series [1].

2.2 Modular Neural Networks

A Modular Neural Network (MNN) is a Neural Network (NN) that consists of several modules, each module carrying out one subtask of the global task [3], it means that the network can be decomposed into two or more modules (sub-systems) that operate on distinct inputs without communicating with each other [2, 7] (each neural network works independently in its own domain). The final decision is based on the results of the individual networks, called agents or experts [11]. Modular artificial neural networks are especially efficient for certain classes of regression and classification problems, as compared to the conventional monolithic artificial neural networks. The modular neural networks are comprised of modules which can be categorized on the basis of both distinct structure and functionality which are integrated together via an integrating unit. With functional categorization, each module is a neural network which carries out a distinct identifiable subtask [2].

2.3 Type-1 Fuzzy Logic

Zadeh introduced the term fuzzy logic in his seminal work “Fuzzy sets” which described the mathematics of fuzzy set theory (1965). FL techniques have been used in image-understanding applications such as detection of edges, feature extraction, classification, and clustering. Fuzzy logic poses the ability to mimic the human mind to effectively employ modes of reasoning that are approximate rather than exact [5]. Fuzzy logic is a useful tool for modeling complex systems and deriving useful fuzzy relations or rules [16]. However, it is often difficult for human experts to define the fuzzy sets and fuzzy rules used by these systems [23]. The basic structure of a fuzzy inference system consists of three conceptual components: a rule base, which contains a selection of fuzzy rules, a database (or dictionary) which defines the membership functions used in the rules, and a reasoning mechanism that performs the inference procedure [6, 24, 25].

3 General Architecture of the Proposed Method

The proposed method combines modular neural networks (MNNs) and type-1 fuzzy logic as integration techniques to perform the time series prediction. As it known, any kind of neural network (artificial, ensemble or modular neural network) has two main phases; training and testing phase. In the artificial or ensemble neural networks, all the data points used for the training phase are learning for the module or sub modules (depending on the king of neural network) [8, 17, 18]. In the proposed method the idea is to divide the data points used for the training phase into sub module and so on, each sub module will be an expert of a part of the time series. The number of data points for each module will be randomly established. An example of this is shown in Fig. 1, where 3 sub modules are used and each module learns a part of the data points used for the training phase.

Fig. 1
figure 1

Example of the proposed method

Full size image

The question would be, how to obtain a final prediction? Firstly, each module has a prediction as Fig. 2 shows, where each module has an individual prediction.

Fig. 2
figure 2

Example of the proposed method showing the three modules

Full size image

To obtain a final prediction these predictions are integrated. In the proposed method type-1 fuzzy logic is used. In this case, 3 modules are used in all the trainings for this reason the fuzzy integrator has 3 inputs, each input represents a module. The fuzzy integrator has an output to obtain a final prediction. An example of the fuzzy integrator is shown in Fig. 3.

Fig. 3
figure 3

Example of the fuzzy integrator

Full size image

The fuzzy integrator is Mamdani type, and each variable (inputs and outputs) has 3 Gaussian membership functions. In Fig. 4, an example of these variables is shown. The fuzzy rules used in the fuzzy inference system are shown in Fig. 5.

Fig. 4
figure 4

Example of variable of the fuzzy integrator

Full size image
Fig. 5
figure 5

Fuzzy rules

Full size image

3.1 Time Series

To prove the effectiveness of the proposed method, 800 data points of the Mackey-Glass time series are used that correspond to a period from 09/11/05 to 15/01/09, 500 data points are used for the training phase and 300 for the testing phase. The Mackey-Glass time series [9, 10] is shown in Fig. 6. The division of the data points for training and testing phase is represented in Fig. 7.

Fig. 6
figure 6

Mackey-Glass time series

Full size image
Fig. 7
figure 7

Data points for training and testing phase

Full size image

4 Experimental Results

The Different trainings performed using different modular neural network architectures (number of hidden layers and neurons) using the type-1 fuzzy integrator is shown in this section. In this work 3 tests were performed, in each case 30 trainings were performed where the number of neurons is randomly established.

4.1 Using One Hidden Layer in Each Module (Test #1)

In this test, the modules have one hidden layer in all the modules. In Table 1, the 5 trainings with the lowest prediction error are shown. The training #26 is the best training in this test. The individual prediction of each module of the best training is presented in Fig. 8.

Table 1 Best results using one hidden layer in each module
Full size table
Fig. 8
figure 8

Individual prediction of each module (best training, test #1)

Full size image

The final prediction can be observed in Fig. 9. In this training 0.058734 of prediction error was obtained.

Fig. 9
figure 9

Best prediction for the test #1

Full size image

4.2 Using Two Hidden Layer in Each Module

In this test, the modules have two hidden layer in all the modules. In Table 2, the 5 trainings with the lowest prediction error are shown. The training #29 is the best training in this test. The individual prediction of each module of the best training is presented in Fig. 10.

Table 2 Results using two hidden layer in each module
Full size table
Fig. 10
figure 10

Individual prediction of each module (best training, test #2)

Full size image

The final prediction can be observed in Fig. 11. In this training 0.061378 of prediction error was obtained.

Fig. 11
figure 11

Best prediction for the test #2

Full size image

4.3 Using One and Two Hidden Layer in Each Module

In this test, the modules have one and two hidden layer, the number in this case was randomly established. In Table 3, the 5 trainings with the lowest prediction error are shown. The training #16 is the best training in this test. The individual prediction of each module of the best training is presented in Fig. 12.

Table 3 Results using one and two hidden layer in each module
Full size table
Fig. 12
figure 12

Individual prediction of each module (best training, test #3)

Full size image

The final prediction can be observed in Fig. 13. In this training 0.058308 of prediction error was obtained.

Fig. 13
figure 13

Best prediction for the test #3

Full size image

4.4 Comparison of Results

In Table 4, a comparison of the results obtained is presented, where it can be observed that the lowest prediction error was obtained in the test #3. This training uses one hidden layer in the first and the second module and two hidden layers in the third module and has a prediction error of 0.058308. But, the best average of the 30 trainings was obtained in the test #1.

Table 4 Comparison of results
Full size table

5 Conclusions

In this work a new method combining modular neural networks and type-1 fuzzy logic used for time series prediction is proposed, where the main idea is to divide the data points for the training phase into sub modules, and each sub module learns a part of the time series and has a prediction. The final prediction is obtained using a type-1 fuzzy integrator Mamdani Type with 3 Gaussian membership functions in each variable (inputs and outputs) with 27 fuzzy rules. 3 tests were performed, where the number of hidden layers and neurons were varied, each test has 30 non-optimized trainings. The best prediction was obtained in the test #3, but the best average was obtained with the test #1. As the difference among the different tests is small, obtaining optimal architectures is better. For this reason as future work the optimization of some parameters of the neural networks is proposed as the number of data points used for the training phase, the number of modules, hidden layers and their neurons. Also the optimization of some parameters of the fuzzy integrator is proposed as the type of membership function, the number of membership functions with their parameters and the fuzzy rules.