Keywords

1 Introduction

The development of robots for production cyber-physical systems has reached a level where it is highly important to organize their joint work. The coalition is viewed as the most effective form of joint work organization [1]. Robots in swarms or flocks are limited in actions by strong rules and actions of nearest neighbors. In contrast, robots in coalitions calculate next steps based on the common goal reaching according to the current coalition state and set of alternatives provided by norms of coalition. Existing models of task solving in coalition define that a robot can receive a reward for the successful problem solving according to its contribution. The independency of robots makes it important to develop an approach to organize coalition formation and interaction between robots for making joint decision during joint problem solving.

Usually production cyber ephysical systems consist of independent robots with different equipment and software that can perform various tasks and expect different levels of reward [2]. In case of production system, the reward is introduced as measure of robot contribution to the success of production and viewed as way to “motivate” robot in doing actions. It is important to consider the heterogeneity and provide common model to consensus reaching during task decomposition and resolving. Each robot is an independent agent with own competencies and goals. The coalition can be considered then as a union of such an agents, which through the negotiation make a decision on a joint solution of the production problem and the distribution of the reward.

The coalition is characterized by dynamic nature and implies a change in its structure, depending on changes in the conditions of the problem being solved. New robots should be quickly familiarized with the current state of the problem solution and provide description of own competences to help to solve the problem. At the same time, existing coalition members should operate without any changes according to the plan. This problem is usually solved by external or internal knowledge bases for storing the history of interaction between coalition members [3, 4].

Most of the existing approaches to coalition formation are characterized by the exponential nature of the computations and communications complexity. To reduce complexity to polynomial the number of agents in one coalition, the number of coalitions, and the rationality of agents are usually limited [5]. The additional complexity is caused by the inability to accurately estimate the size of the gain received after problem solving, which introduces fuzziness into the formulation of the problem.

To solve the problems described above this paper proposes to use the cooperative games with fuzzy core to form a coalition of robots. This model provides the following advantages: robots share own competencies and their expected individual benefit while coalition actions are controlled by one of its member with calculated coalition core. The core coordinates the actions of individual members to achieve a common goal, as well as to evaluate and distribute the overall benefit. The change of environmental conditions leads to a dynamic change in the composition of the coalition, if necessary. A negotiation algorithm and architecture of robot interaction framework has been developed in previous work by the authors [6]. In this paper, it is proposed to use of blockchain technology to store and process fuzzy coalition game’s rules for robot negotiation during the coalition formation.

To store the game rules, competencies and requirements of robots, as well as information about the current state of coalitions and tasks, smart contracts over blockchain technology are used. With smart contracts, two types of the rules are defined: (i) for forming a coalition, and (ii) for changing the composition of the coalition, defined with the theory of fuzzy sets. Smart contracts and the current state of the problem solution, are stored in the blockchain.

The rest of the paper is organized as follows. Related work is revised in the following section. A fuzzy cooperative game (FCG) model with core is described in Sect. 3. In Sect. 4, different criteria of dynamic robot coalition formation are analyzed. Finally, the implementation scheme of a FCG over blockchain-based smart contracts is proposed following by conclusions.

2 Related Work

The cooperative nature of modern robotic complexes in production cyber-physical systems causes necessity of considering them within the context of cooperative game theory in order to model and understand their cooperative behavior. The main questions of coalition formation are: what coalitions will be formed, how the common winnings will be distributed among members and if the obtained coalition structure is stable. Once coalitions are formed and they have a feasible set of payoffs available to its members, the question is the identification of final payoffs awarded to each player. That is, given a collection of feasible sets of payoffs, one for each coalition, can one predict or recommend a payoff (or set of payoffs) to be awarded to each player?

The payoff distribution should guarantee the stability of the coalition structure when no one player has an intention to leave a coalition because of the expectation to increase its payoff. The benefit distribution among the coalition members has proved to be fuzzy, uncertain, and ambiguous [7]. Using the theory of fuzzy cooperative games (FCGs), the uncertainty is processed by means of the introduction of a fuzzy benefit concept through the bargaining process to the conclusion about the corresponding fuzzy distribution of individual benefits among the coalition members [8].

The predictions or recommendations of payment distribution are embodied in different solution concepts. According to [9], cooperative games are divided into two classes based on the way a solution is obtained: games with a solution set and games with a single solution. Games with core considered in this paper, belong to the former class and represent a mechanism for analyzing the possible set of stable outcomes of cooperative games with transferable utilities [10]. The concept of a core tends to maximize the sum of coalition utilities in the particular coalition structure (C-stable coalition). The core of a game with respect to a given coalition structure is defined as a set of such imputations that prevent the players from forming small coalitions by paying off all the subsets an amount, which is at least as much they would get if they form a coalition. Thus the core of a game is a set of imputations which are stable.

The drawbacks of the core is that, on the one hand, the computational complexity of finding the optimal structure is high since for the game with n players at least \( 2^{n} - 1 \) of the total \( n^{n/2} \) coalition structures should be tested. On the other hand, for particular classes of the game a core can be empty. Because of these problems, using the C-stable coalition structures was quite unpopular in practical applications [11] and only recently has attracted more attention of the researchers, when the concept of fuzzy cooperative games with core was introduced [12, 13]. For realistic applications like collaborative work of groups of robots, additive environments and the absence of the restrictions on the type of membership functions should be considered [14].

With regard to the organization of robots interaction, the blockchain is mostly used as immutable storage for information exchange and platform for smart contracts. Information stored in the blockchain could contain records about task and consumables distribution [15, 16], smart contracts and reward transactions [17], as well as global knowledge about coalition previous actions [18]. In combination with cooperative games blockchain technology can provide more trust for communication between robots, due to the storing information about transactions in immutable log that can be verified by every coalition participant. In contrary to existing approaches, blockchain does not require central authority that provide trust for all nodes. All nodes negotiate with each other coming to consensus with one of possible mechanisms: Proof of Work, Proof of Stake, or practical byzantine fault tolerance [19].

The combination of the peer-to-peer network and the cryptographic algorithms used in blockchain technology allow for a negotiation process and consensus building without any controlling authorities. Blockchain had already been used in swarm robotics to store global knowledge about swarm actions [18]. The availability of a distributed transaction ledger also allows new robots to join the swarm and gain all the knowledge prior to the moment of inclusion by downloading and analyzing the transaction history.

3 Fuzzy Cooperative Game Model with Core

A generalized model of a fuzzy cooperative game (FCG) with core was proposed in [14, 20, 21]. As shown in [14], the concept of a core tends to maximize the sum of coalition utilities in the particular coalition structure. The core of a game is a set of imputations, which are stable. The proposed model helped solving the problems of the computational complexity of finding the optimal structure and of the empty core, which enabled its use in practical applications of selecting robots in coalitions.

A FCG is defined as a pair \( \left( {Robot, w} \right) \), where \( Robot \) is nonempty and finite set of players, subsets of \( Robot \) joining together to fulfil some task \( T_{i} \) are called coalitions K, and w is called a characteristic function of the game, being \( w:2^{n} \to {\Re }^{ + } \) a mapping connecting every coalition \( K \subset Robot \) with a fuzzy quantity \( w\left( K \right) \in {\Re }^{ + } \), with a membership function \( \mu_{K} :R \to \left[ {0,1} \right] \). A modal value of \( w\left( K \right) \) corresponds to the characteristic function of the crisp game \( v\left( K \right) \): \( \hbox{max} \,\mu_{K} \left( {w\left( K \right)} \right) = \mu_{K} \left( {\nu \left( K \right)} \right) \). For an empty coalition \( w\left( \emptyset \right) = 0 \). A fuzzy core for the game \( \left( {Robot, w} \right) \) with the imputation \( X = \left( {x_{ij} } \right)_{i \in I,j \in Robot} \in {\Re }^{ + } \) is a fuzzy subset \( C_{F} \) of \( {\Re }^{ + } \):

$$ C_{F} = \left\{ {x_{ij} \in {\Re }^{ + } :\nu f = \left( {w\left( {Robot} \right),\mathop \sum \nolimits_{{\begin{array}{*{20}c} {i \in I,} \\ {j \in Robot} \\ \end{array} }} x_{ij} \varphi_{ij} } \right),\mathop {\hbox{min} }\nolimits_{{\begin{array}{*{20}c} {K_{i} \in k} \\ {j \in Robot} \\ \end{array} }} \left( {\nu f = \left( {\mathop \sum \nolimits_{{j \in K_{i} }} x_{ij} \varphi_{ij} ,w\left( {K_{i} } \right)} \right)} \right) } \right\} , $$
(1)

where \( x_{ij} \) is the fuzzy payment of a robot j participating in a coalition \( i \), \( i = 1, 2, \ldots , I, j = 1, 2, \ldots , N, \bar{k} = \left[ {K_{1} , K_{2} , \ldots , K_{l} } \right] \) is the ordered structure of effective coalitions; \( \varphi \) is a fuzzy partial order relation with a membership function \( \nu f = :R \times R \to \left[ {0,1} \right] \), and \( \varphi_{ij} \) is a binary variable such that:

$$ \varphi_{ij} = \left\{ {\begin{array}{*{20}c} {1, if\, robot\, j \,participates\, in\, a\, coalition\, i;} \\ {0, otherwise.} \\ \end{array} } \right. $$

The \( \varphi_{ij} \) is considered here as a result of some robot’s strategy on joining a coalition.

A fuzzy partial order relation is defined as follows (for more details see  [22]). Let a, b be fuzzy numbers with membership functions \( \mu_{a} \) and \( \mu_{b} \) respectively, then the possibility of partial order \( a\phi = b \) is defined as \( \nu \phi = \left( {a,b} \right) \in \left[ {0,1} \right] \) as follows:

$$ \nu \phi = \left( {a,b} \right) = \mathop {\sup }\nolimits_{{\begin{array}{*{20}c} {x,y \in R} \\ {x \ge y} \\ \end{array} }} \left( {\hbox{min} \left( {\mu_{a} \left( x \right),\mu_{b} \left( y \right)} \right)} \right) . $$
(2)

The core \( C_{F} \) is the set of possible distributions of the total payment achievable by the coalitions, and none of coalitions can offer to its members more than they can obtain accepting some imputation from the core. The first argument of the core \( C_{F} \) indicates that the payments for the grand coalition are less than the characteristic function of the game. The second argument reflects the property of group rationality of the players, that there is no other payoff vector, which yields more to each player. The membership function \( \mu_{{C_{F} }} :R \to \left[ {0,1} \right] \), is defined as:

$$ \mu_{{C_{F} }} \left( x \right) = \hbox{min} \left\{ {\nu f = \left( {w\left( {Robot} \right), \mathop \sum \nolimits_{{\begin{array}{*{20}c} {i \in I} \\ {j \in Robot} \\ \end{array} }} x_{ij} \varphi_{ij} } \right),\mathop {\hbox{min} }\nolimits_{{\begin{array}{*{20}c} {K_{i} \in k} \\ {j \in Robot} \\ \end{array} }} \left( {\nu f = \left( {\mathop \sum \nolimits_{{j \in K_{i} }} x_{ij} \varphi_{ij} , w\left( {K_{i} } \right)} \right)} \right)} \right\} $$
(3)

The solution of a cooperative game is a coalition configuration \( \left( {S, x} \right) \) which consists of (i) a partition \( S \) of \( Robot \), the so-called coalition structure, and (ii) an efficient payoff distribution x which assigns each robot in Robot its payoff out of the utility of the coalition it is member of in a given coalition structure S. A coalition configuration (S, x) is called stable if no robot has an incentive to leave its coalition in S due to its assigned payoff xi.

It was proved that the fuzzy set of coalition structures forming the game core represents a subset of the fuzzy set formed by the structure of effective coalitions. In turn, this inference allows us to specify the upper possibility bound for the core, which is a very important condition for the process of solution searching, because in this case, the presence of a solution that meets the efficiency condition may serve as the signal to terminate the search algorithm [21].

The game purpose is to generate an effective structure of robot coalitions for executing some production task. In turn, the generated structure of robot coalitions represents the optimal configuration of the grand coalition. Individual robots use the technique of nonlinear fuzzy regression to estimate the parameters of utility functions for their payments [23]. A “coalition robot” is enabled for constructing membership functions (MF) of coalitions and generating the game core (fuzzy-number generator). The sum operation is based on Zadeh extension principle [22] for fuzzy numbers a and b (which are convex sets normalized in R):

$$ \mu_{a\left( *\right)b} \left( Z \right) = \mathop {\sup }\nolimits_{z = x *y} \hbox{min} \left( {\mu_{a} \left( x \right),\mu_{b} \left( y \right)} \right) $$
(4)

where * can designate the sum ⊕ or the product • of fuzzy numbers. Each fuzzy set is decomposed into two segments, a non-decreasing and non-increasing one. The operation * is performed for every group of n segments (one segment for each fuzzy set) that belong to the same class (non-decreasing or non-increasing one). Thus, a fuzzy set is generated for every group of n segments.

4 Criteria for FCG in Dynamic Robot Coalition Formation

Group problem solving in cyberphysical production systems requires a well-coordinated interaction of the participants’ actions during the coalition formation for production task. Regardless of the coalition model used, its formation can be considered as three types of interrelated actions that are performed sequentially:

  • Generation of a coalition structure; a framework in which agents within each coalition coordinate their activities;

  • Optimization of each coalition; agents have to be united by competencies for effective problem solving;

  • Profit sharing between coalition members.

If these actions are performed before problem solving, a static coalition formation is considered. The structure of static coalitions does not change over time. At the time of optimization of the coalition, also a plan for solving the problem is calculated as well as all possible deviations from the plan. In case of failure, the correction of the plan is carried out with coalition members that are still operate, taking into account the changed conditions in order to get back to the original plan with minimal losses.

A more complex case of a coalition formation is the dynamic coalition. In this case a plan of problem solving is formed during the optimization stage same as for the static coalition. However, in case of deviation from the plan, a fix is based on changing the structure of the coalition, for example, by adding a new participant. To do this, the rules for the formation of the coalition should describe actions for unknown situations with fuzzy parameters, and the overall benefit of the coalition and plan are dynamically recalculated considering the context of the task has changed.

To evaluate the coalition efficiency the following parameters can be used:

  • Energy spent to task solving. The solution of each task or sub-task can be estimated by the energy (charge of the battery) \( E_{k} \left( {T_{i} } \right) \) of the robot k that was spent to solve it:

    $$ E_{k} \left( {T_{i} } \right) = \mathop \sum \nolimits_{j} f_{{T_{i} }} \left( {b_{j}^{k} } \right) \cdot \varphi \left( {T_{i} ,k,j} \right) $$
    (5)

    Based on aggregated data over similar problems, it is possible to obtain an approximate estimation, which, however, introduces fuzziness into the final coalition formation. The coalition efficiency can be estimated as the total energy expended:

    $$ v\left( {K_{{T_{i} }} } \right) = Payoff\left( {T_{i} } \right) - \mathop {\hbox{min} }\nolimits_{{K_{i} \in k}} \mathop \sum \nolimits_{{j \in K_{i} }} E_{j} \left( {T_{i} } \right) $$
    (6)

  • An analogue to the estimated energy expended is a robot uptime based on failure probability. The estimation of failure probability is the ratio of difference between the time of the node work and the average uptime of the same kind of robot units: \( P_{{C_{i} }} = T_{{C_{i} }}^{w} - T_{{C_{i} }}^{m} /T_{{C_{i} }}^{avg} \), where \( P_{{C_{i} }} \) – failure probability of unit \( c_{i} \) by robot \( {\text{r}}_{\text{j}} {{\rm r}}_{\text{j}} \), \( T_{{c_{i} }}^{w} \) – total work duration of the unit \( c_{i} \), \( T_{{c_{i} }}^{m} \) – last service time point. The probability of entire robot failure is evaluated according to the maximum probability of nodes failure \( {\text{P}}_{\text{r}} = \max_{\text{i}} {{\rm P}}_{{{\text{c}}_{\text{i}} }} \). The efficiency criterion in this case will be the maximum duration of the coalition’s overall work to the next maintenance.

  • Maximizing the coalition benefit. For example, in relation to cyberphysical production system, the coalition’s benefit is the cumulative product of the system. The value of the solution of the problem decreases with the passage of time: the longer the task is postponed, the less benefit it can provide.

5 Robot Interaction over Smart Contracts with FCG

In this Section, the scheme of interaction of robots during the coalition formation, is proposed. The ability of smart contracts within the scope of blockchain technology to describe complex algorithms by using the Turing-complete programming language is used to define game rules in smart contracts. Examples of blockchain platforms with support of smart contracts include the Ethereum platform and Hyperledger Fabric.

Figure 1 shows the scheme of interaction of robots in the coalition through a blockchain. Two kinds of chains for storing resources and contracts is proposed to use in the blockchain for robot interaction. All system resources including consumables, energy, reward, which are represented by tokens, are stored in the resources chains. In the contracts chain the rules of cooperative game are stored, which are used by the robots coordinators during the coalition forming and the distribution of tasks. The contracts also contains rules for processing tasks and assigning coalition core. The need for the presence of coalition core, which will assess the complexity of the solution of the problem or the effectiveness of its solution and then distribute tasks among the performers is noted in previous works [20, 21]. New tasks can be formed both inside and outside of the system and are recorded in the contract chain, where they become available for all coalition cores. Tasks contain a formalized description of the goal, the initial parameters and the amount of reward for the solution.

The robot coordinator (RobotCORE1 or RobotCORE2 on Fig. 1) selects robots using smart contracts based on their competencies and reward expectations, as well as the rules of the cooperative game, defined for the subject area to which the task belongs. If the robot can participate in several coalitions, each robot coordinator calculates the cooperative game core and winning for each of the coalitions, as well as the availability of sufficient resources for the robot successful work. If there are enough resources for robot’s operations, it can participate in several coalitions. Otherwise, the robot is assigned to a coalition for which it can bring the largest benefit. The reward for the successful solution of the problem is distributed among the robots participating in the coalition based on the reward rules for the cooperative game, described in the code of the relevant contract.

The presented approach provides the following advantages. First, the use of the blockchain guarantees the authorship of the information and its immutability, which is an important aspect in dynamic coalitions of robots. Built-in authentication algorithms and digital signatures allow to get rid of the complex infrastructure of certification centers, which simplifies the deployment and operation of the system. Separation of chains with resources and contracts provides an independent storage and exchange of the state of the coalition between performers and control robots. This provides a trusted knowledge base for robots to share coalition state including task and reward distribution. Since the data in the blockchain is bounded by the blocks’ hash, no block can be changed. It makes process logs immutable and history of operations can be traced back, to find a failure points, and enhance the effectiveness of future organized coalitions. The Hyperledger platform that is proposed to be used for the organization of interaction provides a high speed of processing and exchange of transactions (up to 3500 transactions per seconds [24]).

Fig. 1.
figure 1

Robot interaction in coalition through blockchain and smart contracts.

Full size image

Secondly, the architecture of Hyperledger platform, which includes three levels (clients, peers and orderers), reflects the interaction structure presented on Fig. 1: the clients correspond to the coalition robots, peers is robots with the coalition core and orderer is a cloud service for storing chains. Each chain is corresponds to channel in Hyperledger. Due to this, the computation of the core and the exchange of tasks and the solutions obtained is carried out without the involvement of robots with a small computing power. Turing-complete language can be used to formalize a contract of any complexity.

6 Conclusion

In this paper, the integrated model of application of games with fuzzy coalitions and fuzzy smart contracts, which can be applied to coalition formation between multiple robots has been described. Fuzziness serves as the fundamental component of realistic cooperation models when there exist fuzzy expectations of player and coalition benefits. When an effective solution is found, individual benefits for players (the agreement efficiency) increase, as well as the capability of the coalition to find an effective and stable agreement. Fuzzy payments may have any utility function, which enables the use of the model in real-world applications. The blockchain model allows to avoid the synchronization problem, which is critical for distributed negotiation algorithms with large robot populations.

The integration of this FCG model with smart contracts can make robot coalitions more transparent. The use of Internet of Things (IoT) sensors, which track robots’ actions though the chain, provides opportunity to control and analyze all process of joint problem solving. If any robot in coalition fails to meet the terms of the contract, for instance if it did not finish an operation on time, it would be clear for every party to see and new coalitions can be arranged dynamically. The blockchain can be used to safe and trustiness logging of robots’ task and reward distribution and for task solving.